Preface; 1 Introduction; 1.1. Some Basic Concepts of Stochastic Processes and Examples; 1.2. Markovian and Non-Markovian Processes, Markov Chains and Examples; 1.3. Diffusion Processes and Examples; 1.4. State Space Models and Hidden Markov Models; 1.5. The Scope of the Book; 1.6. Complements and Exercises; References; 2 Discrete Time Markov Chain Models in Genetics and Biomedical Systems; 2.1. Examples fromGenetics and AIDS; 2.2. The Transition Probabilities and Computation; 2.3. The Structure and Decomposition of Markov Chains

2.4. Classification of States and the Dynamic Behavior of Markov Chains2.5. The Absorption Probabilities of Transient States; 2.5.1. The case when CT is finite; 2.5.2. The case when CT is infinite; 2.6. The Moments of First Absorption Times; 2.6.1. The case when CT is finite; 2.7. Some Illustrative Examples; 2.8. Finite Markov Chains; 2.8.1. The canonical form of transition matrix; 2.8.2. Absorption probabilities of transient states in finite Markov chains; 2.9. Stochastic Difference Equation for Markov Chains With Discrete Time; 2.9.1. Stochastic difference equations for finite Markov chains

2.9.2. Markov chains in the HIV epidemic in homosexual or IV drug user populations2.10.Complements and Exercises; 2.11. Appendix; 2.11.1. The Hardy-Weinberg law in population genetics; 2.11.1.1. The Hardy-Weinberg law for a single locus in diploid populations; 2.11.1.2. The Hardy-Weinberg law for linked loci in diploid populations; 2.11.2. The inbreeding mating systems; 2.11.3. Some mathematical methods for computing An, the nth power of a square matrix A; References; 3 Stationary Distributions and MCMC in Discrete Time Markov Chains; 3.1. Introduction

3.2. The Ergodic States and Some Limiting Theorems3.3. Stationary Distributions and Some Examples; 3.4. Applications of Stationary Distributions and Some MCMC Methods; 3.4.1. The Gibbs sampling method; 3.4.2. The weighted bootstrap method for generating random samples; 3.4.3. The Metropolis-Hastings algorithm; 3.5. Some Illustrative Examples; 3.6. Estimation of Linkage Fraction by Gibbs Sampling Method; 3.7. Complements and Exercises; 3.8. Appendix: A Lemma for Finite Markov Chains; References; 4 Continuous-Time Markov Chain Models in Genetics, Cancers and AIDS; 4.1. Introduction

4.2. The Infinitesimal Generators and an Embedded Markov Chain4.3. The Transition Probabilities and Kolmogorov Equations; 4.4. Kolmogorov Equations for Finite Markov Chains with Continuous Time; 4.5. Complements and Exercises; References; 5 Absorption Probabilities and Stationary Distributions in Continuous-Time Markov Chain Models; 5.1. Absorption Probabilities and Moments of First Absorption Times of Transient States; 5.1.1. The case when CT is finite; 5.2. The Stationary Distributions and Examples; 5.3. Finite Markov Chains and the HIV Incubation Distribution