SUBJECT
Markov chains in discrete and continuous time
lecture
master
2
Semesters 1-4
Autumn/Spring semester
Markov property and strong Markov property for stochastic processes. Discrete time Markov chains with stationary transition probabilities: definitions, transition probability matrix. Classification of states, periodicity, recurrence. The basic limit theorem for the transition probabilities. Stationary probability distributions. Law of large numbers and central limit theorem for the functionals of positive recurrent irreducible Markov chains. Transition probabilities with taboo states. Regular measures and functions. Doeblin’s ratio limit theorem. Reversed Markov chains.
Absorption probabilities. The algebraic approach to Markov chains with finite state space. Perron-Frobenius theorems.
- Karlin – Taylor: A First Course in Stochastic Processes, Second Edition. Academic Press, 1975
- Chung: Markov Chains With Stationary Transition Probabilities. Springer, 1967.
- Isaacson – Madsen: Markov Chains: Theory and Applications. Wiley, 1976.