This book is devoted to the systematic exposition of the contemporary theory of controlled Markov processes with discrete time parameter or in another termi­ nology multistage Markovian decision processes. We discuss the applications of this theory to various concrete problems. Particular attention is paid to mathe­ matical models of economic planning, taking account of stochastic factors. The authors strove to construct the exposition in such a way that a reader interested in the applications can get through the book with a minimal mathe­ matical apparatus. On the other hand, a mathematician will find, in the appropriate chapters, a rigorous theory of general control models, based on advanced measure theory, analytic set theory, measurable selection theorems, and so forth. We have abstained from the manner of presentation of many mathematical monographs, in which one presents immediately the most general situation and only then discusses simpler special cases and examples. Wishing to separate out difficulties, we introduce new concepts and ideas in the simplest setting, where they already begin to work. Thus, before considering control problems on an infinite time interval, we investigate in detail the case of the finite interval. Here we first study in detail models with finite state and action spaces-a case not requiring a departure from the realm of elementary mathematics, and at the same time illustrating the most important principles of the theory.

Springer Book Archives

I: Control on a Finite Time Interval.- Chaper 1. Finite and Denumerable Models.- 2. Semicontinuous Models.- 3. General (Borel) Models.- II: Control on an Infinite Time Interval.- 4. Discrete Models.- 5. Borel Models.- 6. Homogeneous Models.- 7. Maximization of the Average Reward Per Unit Time.- III: Some Applications.- 8. Models with Incomplete Information.- 9. Concave Models. Models of Economic Development.- Appendix 3: Theorems on Measurable Selection.-
1. The Lemma of Yankov.-
2. The Theorem of Blackwell and Ryll-Nardzewski.-
3. Example of a Correspondence Not Admitting a Measurable Selection.- Appendix 4: Conditional Distributions.-
1. Introduction.-
2. Conditional Mathematical Expectations.-
3. Support Systems.-
4. Existence of Conditional Distributions.- Appendix 5: Some Lemmas on Measurability.-
1. The Lemma on Multiplicative Systems.-
2. Measurable Structure in the Space of Probability Measures.- Historical-Bibliographical Notes.