Introduction to Markov Chains

Introduction

Markov Chains are just a fancy way of saying what the probability between two or more things would be using current present state date. They are ran on thousands of pieces of data to estimate what the next "state" will be. Let's take below as an example, 


Cite: Medium
Suppose we feed the past 5 years' data on the weather into this simulation.
The three big circles that we have identified are what we call "states." The arrows are the possibility for one state to go to another in a certain time interval you choose. We will be doing a one day interval. In our case, we have three states of the weather: rainy, sunny, and cloudy. The chance for the weather going from sunny to cloudy is 40% (as represented by the arrow). The chance for the weather going from sunny to rainy is 10%. It could also be sunny the next day, and that chance is 50%. Notice that the chances of the Sunny state all add up to 100%. This happens for all the other states as well.  

This could also be represented in a table, 

Why?

Markov Chains also have an embedded way that accounts "uncertainty." For example, stocks can be extremely volatile in some cases (take for example the 2008 Stock Market Crash). Markov Chains can be good in accounting this extreme volatility. And, they are the basis of stochastic modeling and predicting. They are found in several different places!

1. Finance/Economics
2. Betting (e.g. sports betting)
3. Biology (e.g. making predictive modeling on what the next sequence of DNA could be)
4. What lunch the cafeteria is gong to have (👀)
5. Game Theory
6. Natural Language Processing (e.g. guess what the next word is going to be in a paragraph) 
7. Chemistry (e.g. modeling chemical reactions)

Pros/Cons

Pros

1. Easy to use out of the get-go and simple. 
2. Accounts for uncertainty better than most other models.
3. Has flexibility in its modeling of the transitions.
4. Markov chains have a more natural way of expressing probabilities than do most other combinatorial methods (representing how objects are counted). 

Cons

1. It isn't good with "understanding" stuff (it's another reason why you would use an attention mechanism).
2. Can't know all possible states/predict all real-world scenarios (e.g. in Biology perhaps it doesn't account for a pathogen that is in the bloodstream). 
3. Requires a lot of data to properly predict transitions. 
4. Creating/Forming data in a manner that accounts for the states and the "lines" (or transition) to the next state. It can be hard to differentiate between them. 
5. Markov proprieties are only good for the example you have created. They can not be used for multiple different types of probabilistic modeling (e.g. attempting to ask how do you make a bridge to an AI trained to assist you for low-level tasks such as creating a to-do list). 
6. The simplicity part also means that modeling complex processes can be extremely difficult (and, are better not to be modeled by Markov Chains in most cases). 



Comments

Post a Comment

Popular posts from this blog

What is a Multimodal LLM?

Image
Introduction Multimodal is the ability for a model to process several different types of inputs. For example, ChatGPT can support text input as well as image input as one combined input. You could also have a model take in .pdf, .md, and input text as inputs (yes, these count as different modalities).  This allows for users to give represent different ideas in its rawest form possible, such as an image, instead of describing it. If you try to describe it, you could hit hurdles in trying to properly explain it and making sure it interpreted the description correctly.  How do Multimodal LLMs work?  Typically, these modalities are "fused" together. Here's an architecture diagram,  Image from NVIDIA In this case, we have an encoder for a specific modality (e.g. video encoder, image encoder, audio encoder). This encodes them into embeddings the LLM can understand and use. After encoding each specific modality, we then fuse them together. Then, the output is projected and ...
Read more

Top 3 Breakthroughs in Computer Vision in 2024

Image
Introduction Over the past few year, AI has become popular; however, no year matches the pace of the past few years. 2024 was an extremely crucial year for Computer Vision, with new models and breakthroughs occurring rapidly. Today, I'll be talking about the 3 biggest advancements in the Computer Vision field during 2024.  3. Vision Language Modeling Simply put, Vision Language Models, or VLMs for short, are Large Language Models (LLMs) that take an image and/or text as input. Think of ChatGPT, where you import an image and then type out a question that you have over the image. It will be able to understand both and output an answer. These are also what you would call a "multimodal model," which is a model that can understand multiple different types of sources/inputs.  Although Vision Language Models have existed for a while, they have recently become better and more accurate. However, this doesn't mean it's all sunshine an rainbows: there are still several barri...
Read more

A Mathematical Explanation of Gradient Descent

Image
Example of a close-to-optimal Gradient descent for a quadratic function. Credit: Dylan Introduction Although I wrote an article recently explaining Gradient Descent, I feel like a mathematical explanation of how it works would be beneficial not only to understand how it works, but also how you derive the functions.  Explanation First, we need a way to calculate how far off the model is. For our example, we will use the L2 Loss Function,  Let N represent the total amount of data points we have, i the iteration we are on, yáµ¢ represent the expected output, and Å· represent the output predicted by the model.  Let's look more into Å·. Since Å· is the predicted value, this is the function that needs to have their parameters changed. Our goal is to change these parameters enough to minimize the loss of Å·. For our example, let's set Å· to be a linear line.  The values in "a" and "b" are our parameters. These will be the values we will have to change within f(x) to chang...
Read more