transition matrix

equilibrium state

adjacency matrix

\(A v=\lambda v\)

 

有从A指向B的箭头，则说明有a non-zero transition probability from state A to state B.

we leave state 0 and we never come back again.

 

The probability of revisiting state 0 in a random walk that started from state 0 is less than 1. This type of state for which the probability of coming back to itself is less than 1 is called a transient state.

从1出发，We can clearly see that after some time we are bound to revisit state 1. So, in this case, the probability of coming back to state 1 is 1. This type of state is called a recurrent state.

从2出发，也是和1一样，一定会回到2，所以2也是recurrent state

0不是recurrent state，因为there's no way to go from state 1 or state 2 to state 0.

In this type of situation where some states are unreachable from others, we say the Markov chain is reducible.

 

如果连接2到0的箭头，那么0也变成了recurrent state。This type of chain where we can go to any state from any given state is called an irreducible chain.

And yes, these classes are known as communicating classes if you haven't guessed that already.