# Markov Chain Monte Carlo

Markov chain Monte Carlo (MCMC) and closely related stochastic algorithms become indispensable when the objective functions of interest are intractable. In this approach one can design an algorithm with a random source (also known as a Markov kernel) and run it for a relatively long time, seeking a sample from the stationary distribution (of the Markov kernel). We begin our discussion with the review of Monte Carlo simulations, Markov chains, and random algorithms in a general setting, preparing the stage for the study of various implementations of stochastic algorithms. We also explore other interesting topics related to Markov chains and random algorithms such as hidden Markov models and EM algorithms. The presentations will cover the following topics:

PDF Files | Contents | Planned Date |

lecture1.pdf | Motivation for MCMC | September 28 |

lecture2.pdf | Stochastic Models and MCMC | October 5 |

lecture3.pdf | Discrete Structures and Gibbs Sampler | October 14 (Thursday) |

lecture4.pdf | Hidden Markov Models: An Introduction to Dynamic Decision Making | November 11 (Thursday) |

lecture5.pdf | Bayesian Statistics and Data Mining | November 23 |

These topics are also interspersed with demonstrations in R. R is free software/platform for statistical computing and graphics, and our source code is made available for anyone interested in running it.

There are a large number of expository and research papers on MCMC methods and their applications. Quite a few of them are made available at

This presentation has been developed for a series of lectures at Tokyo Institute of Technology in 2010.

© TTU Mathematics