Probabilities of Arbitrary Transitions by Chapman-Kolmogorov Equations

Probabilities of Arbitrary Transitions by Chapman-Kolmogorov Equations is Availale on Medium

Problems modeled as discrete homogenous (stationary or time independent) Markov chains can often be solved iteratively using conditional probabilities or conditional expectations. However, it is not uncommon for a process to begin at a specific state (or a series of states given some arbitrary probabilities) where the desired state is some n arbitrary steps forward in the process.

Given the relative size of n it may be challenging analytically to explicitly write out the necessary chain of conditional probabilities or conditional expectations for use in the law of total expectation or law of total probability to derive the value at step n.

The Chapman-Kolmogorov equations remedy this by proving the probability of transitioning from a given state to an arbitrary future state n steps forward is an element in the given transition matrix raised to the power of n.

This article will be broke up into the following sections…

- Motivation for the Chapman-Kolmogorov equations

- Proof of the Chapman-Kolmogorov equations

- Application of the Chapman-Kolmogorov equations