Table of Contents |
| List of Figures | xv |
| Preface | xvii |
| Introduction | xix |
1. | Probability Space | 1 |
1.1. | Introduction/Purpose of the Chapter | 1 |
1.2. | Vignette/Historical Notes | 2 |
1.3. | Notations and Definitions | 2 |
1.4. | Theory and Applications | 4 |
1.4.1. | Algebras | 4 |
1.4.2. | Sigma Algebras | 5 |
1.4.3. | Measurable Spaces | 7 |
1.4.4. | Examples | 7 |
1.4.5. | The Borel σ-Algebra | 9 |
1.5. | Summary | 12 |
| Exercises | 12 |
2. | Probability Measure | 15 |
2.1. | Introduction/Purpose of the Chapter | 15 |
2.2. | Vignette/Historical Notes | 16 |
2.3. | Theory and Applications | 17 |
2.3.1. | Definition and Basic Properties | 17 |
2.3.2. | Uniqueness of Probability Measures | 22 |
2.3.3. | Monotone Class | 24 |
2.3.4. | Examples | 26 |
2.3.5. | Monotone Convergence Properties of Probability | 28 |
2.3.6. | Conditional Probability | 31 |
2.3.7. | Independence of Events and σ-Fields | 39 |
2.3.8. | Borel---Cantelli Lemmas | 46 |
2.3.9. | Fatou's Lemmas | 48 |
2.3.10. | Kolmogorov's Zero---One Law | 49 |
2.4. | Lebesgue Measure on the Unit Interval (0,1 | 50 |
| Exercises | 52 |
3. | Random Variables: Generalities | 63 |
3.1. | Introduction/Purpose of the Chapter | 63 |
3.2. | Vignette/Historical Notes | 63 |
3.3. | Theory and Applications | 64 |
3.3.1. | Definition | 64 |
3.3.2. | The Distribution of a Random Variable | 65 |
3.3.3. | The Cumulative Distribution Function of a Random Variable | 67 |
3.3.4. | Independence of Random Variables | 70 |
| Exercises | 71 |
4. | Random Variables: The Discrete Case | 79 |
4.1. | Introduction/Purpose of the Chapter | 79 |
4.2. | Vignette/Historical Notes | 80 |
4.3. | Theory and Applications | 80 |
4.3.1. | Definition and Basic Facts | 80 |
4.3.2. | Moments | 84 |
4.4. | Examples of Discrete Random Variables | 89 |
4.4.1. | The (Discrete) Uniform Distribution | 89 |
4.4.2. | Bernoulli Distribution | 91 |
4.4.3. | Binomial (n, p) Distribution | 92 |
4.4.4. | Geometric (p) Distribution | 95 |
4.4.5. | Negative Binomial (r, p) Distribution | 101 |
4.4.6. | Hypergeometric Distribution (N, m, n) | 102 |
4.4.7. | Poisson Distribution | 104 |
| Exercises | 108 |
5. | Random Variables: The continuous case | 119 |
5.1. | Introduction/Purpose of the Chapter | 119 |
5.2. | Vignette/Historical Notes | 119 |
5.3. | Theory and Applications | 120 |
5.3.1. | Probability Density Function (p.d.f.) | 120 |
5.3.2. | Cumulative Distribution Function (c.d.f.) | 124 |
5.3.3. | Moments | 127 |
5.3.4. | Distribution of a Function of the Random Variable | 128 |
5.4. | Examples | 130 |
5.4.1. | Uniform Distribution on an Interval [a,b] | 130 |
5.4.2. | Exponential Distribution | 133 |
5.4.3. | Normal Distribution (μ, σ2) | 136 |
5.4.4. | Gamma Distribution | 139 |
5.4.5. | Beta Distribution | 144 |
5.4.6. | Student's t Distribution | 147 |
5.4.7. | Pareto Distribution | 149 |
5.4.8. | The Log-Normal Distribution | 151 |
5.4.9. | Laplace Distribution | 153 |
5.4.10. | Double Exponential Distribution | 155 |
| Exercises | 156 |
6. | Generating Random Variables | 177 |
6.1. | Introduction/Purpose of the Chapter | 177 |
6.2. | Vignette/Historical Notes | 178 |
6.3. | Theory and Applications | 178 |
6.3.1. | Generating One-Dimensional Random Variables by Inverting the Cumulative Distribution Function (c.d.f.) | 178 |
6.3.2. | Generating One-Dimensional Normal Random Variables | 183 |
6.3.3. | Generating Random Variables. Rejection Sampling Method | 186 |
6.3.4. | Generating from a Mixture of Distributions | 193 |
6.3.5. | Generating Random Variables. Importance Sampling | 195 |
6.3.6. | Applying Importance Sampling | 198 |
6.3.7. | Practical Consideration: Normalizing Distributions | 201 |
6.3.8. | Sampling Importance Resampling | 203 |
6.3.9. | Adaptive Importance Sampling | 204 |
6.4. | Generating Multivariate Distributions with Prescribed Covariance Structure | 205 |
| Exercises | 208 |
7. | Random Vectors in Rn | 210 |
7.1. | Introduction/Purpose of the Chapter | 210 |
7.2. | Vignette/Historical Notes | 210 |
7.3. | Theory and Applications | 211 |
7.3.1. | The Basics | 211 |
7.3.2. | Marginal Distributions | 212 |
7.3.3. | Discrete Random Vectors | 214 |
7.3.4. | Multinomial Distribution | 219 |
7.3.5. | Testing Whether Counts are Coming from a Specific Multinomial Distribution | 220 |
7.3.6. | Independence | 221 |
7.3.7. | Continuous Random Vectors | 223 |
7.3.8. | Change of Variables. Obtaining Densities of Functions of Random Vectors | 229 |
7.3.9. | Distribution of Sums of Random Variables. Convolutions | 231 |
| Exercises | 236 |
8. | Characteristic Function | 255 |
8.1. | Introduction/Purpose of the Chapter | 255 |
8.2. | Vignette/Historical Notes | 255 |
8.3. | Theory and Applications | 256 |
8.3.1. | Definition and Basic Properties | 256 |
8.3.2. | The Relationship Between the Characteristic Function and the Distribution | 260 |
8.4. | Calculation of the Characteristic Function for Commonly Encountered Distributions | 265 |
8.4.1. | Bernoulli and Binomial | 265 |
8.4.2. | Uniform Distribution | 266 |
8.4.3. | Normal Distribution | 267 |
8.4.4. | Poisson Distribution | 267 |
8.4.5. | Gamma Distribution | 268 |
8.4.6. | Cauchy Distribution | 269 |
8.4.7. | Laplace Distribution | 270 |
8.4.8. | Stable Distributions. Levy Distribution | 271 |
8.4.9. | Truncated Levy Flight Distribution | 274 |
| Exercises | 275 |
9. | Moment-Generating Function | 280 |
9.1. | Introduction/Purpose of the Chapter | 280 |
9.2. | Vignette/Historical Notes | 280 |
9.3. | Theory and Applications | 281 |
9.3.1. | Generating Functions and Applications | 281 |
9.3.2. | Moment-Generating Functions. Relation with the Characteristic Functions | 288 |
9.3.3. | Relationship with the Characteristic Function | 292 |
9.3.4. | Properties of the MGF | 292 |
| Exercises | 294 |
10. | Gaussian random vectors | 300 |
10.1. | Introduction/Purpose of the Chapter | 300 |
10.2. | Vignette/Historical Notes | 301 |
10.3. | Theory and Applications | 301 |
10.3.1. | The Basics | 301 |
10.3.2. | Equivalent Definitions of a Gaussian Vector | 303 |
10.3.3. | Uncorrelated Components and Independence | 309 |
10.3.4. | The Density of a Gaussian Vector | 313 |
10.3.5. | Cochran's Theorem | 316 |
10.3.6. | Matrix Diagonalization and Gaussian Vectors | 319 |
| Exercises | 325 |
11. | Convergence Types. Almost Sure Convergence. LP-Convergence. Convergence in Probability | 338 |
11.1. | Introduction/Purpose of the Chapter | 338 |
11.2. | Vignette/Historical Notes | 339 |
11.3. | Theory and Applications: Types of Convergence | 339 |
11.3.1. | Traditional, Deterministic Convergence Types | 339 |
11.3.2. | Convergence of Moments of an r.v.---Convergence in LP | 341 |
11.3.3. | Almost Sure (a.s.) Convergence | 342 |
11.3.4. | Convergence in Probability | 344 |
11.4. | Relationships Between Types of Convergence | 346 |
11.4.1. | a.s. and LP | 347 |
11.4.2. | Probability and a.s.ILP? | 351 |
11.4.3. | Uniform Integrability | 357 |
| Exercises | 359 |
12. | Limit Theorems | 372 |
12.1. | Introduction/Purpose of the Chapter | 372 |
12.2. | Vignette/Historical Notes | 372 |
12.3. | Theory and Applications | 375 |
12.3.1. | Weak Convergence | 375 |
12.3.2. | The Law of Large Numbers | 384 |
12.4. | Central Limit Theorem | 401 |
| Exercises | 409 |
13. | Appendix A: Integration Theory, general Expectations | 421 |
13.1. | Integral of Measurable Functions | 422 |
13.1.1. | Integral of Simple (Elementary) Functions | 422 |
13.1.2. | Integral of Positive Measurable Functions | 424 |
13.1.3. | Integral of Measurable Functions | 428 |
13.2. | General Expectations and Moments of a Random Variable | 429 |
13.2.1. | Moments and Central Moments. LP Space | 430 |
13.2.2. | Variance and the Correlation Coefficient | 431 |
13.2.3. | Convergence Theorems | 433 |
14. | Appendix B: Inequalities Involving Random Variables and Their Expectations | 434 |
14.1. | Functions of Random Variables. The Transport Formula | 441 |
| Bibliography | 445 |
| Index | 447 |