Markov Chains and Stochastic Stability

Author: Sean P. Meyn
Publisher: Springer Science & Business Media
ISBN: 144713267X
Category : Technology & Engineering
Languages : en
Pages : 559

Book Description
Markov Chains and Stochastic Stability is part of the Communications and Control Engineering Series (CCES) edited by Professors B.W. Dickinson, E.D. Sontag, M. Thoma, A. Fettweis, J.L. Massey and J.W. Modestino. The area of Markov chain theory and application has matured over the past 20 years into something more accessible and complete. It is of increasing interest and importance. This publication deals with the action of Markov chains on general state spaces. It discusses the theories and the use to be gained, concentrating on the areas of engineering, operations research and control theory. Throughout, the theme of stochastic stability and the search for practical methods of verifying such stability, provide a new and powerful technique. This does not only affect applications but also the development of the theory itself. The impact of the theory on specific models is discussed in detail, in order to provide examples as well as to demonstrate the importance of these models. Markov Chains and Stochastic Stability can be used as a textbook on applied Markov chain theory, provided that one concentrates on the main aspects only. It is also of benefit to graduate students with a standard background in countable space stochastic models. Finally, the book can serve as a research resource and active tool for practitioners.

Markov Chains and Stochastic Stability

Author: Sean Meyn
Publisher: Cambridge University Press
ISBN: 1139477978
Category : Mathematics
Languages : en
Pages : 595

Book Description
Meyn and Tweedie is back! The bible on Markov chains in general state spaces has been brought up to date to reflect developments in the field since 1996 - many of them sparked by publication of the first edition. The pursuit of more efficient simulation algorithms for complex Markovian models, or algorithms for computation of optimal policies for controlled Markov models, has opened new directions for research on Markov chains. As a result, new applications have emerged across a wide range of topics including optimisation, statistics, and economics. New commentary and an epilogue by Sean Meyn summarise recent developments and references have been fully updated. This second edition reflects the same discipline and style that marked out the original and helped it to become a classic: proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background.

Markov Chains and Stochastic Stability

Author: Sean Meyn
Publisher: Cambridge University Press
ISBN: 0521731828
Category : Mathematics
Languages : en
Pages : 623

Book Description
New up-to-date edition of this influential classic on Markov chains in general state spaces. Proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background. New commentary by Sean Meyn, including updated references, reflects developments since 1996.

Markov Chains and Stochastic Stability

Author: Sean P. Meyn
Publisher:
ISBN:
Category :
Languages : en
Pages : 594

Book Description

Markov Chains and Stochastic Stability

Author: Sean P Meyn
Publisher:
ISBN: 9781447132684
Category :
Languages : en
Pages : 572

Book Description

Markov Chains

Author: Pierre Bremaud
Publisher: Springer Science & Business Media
ISBN: 1475731248
Category : Mathematics
Languages : en
Pages : 456

Book Description
Primarily an introduction to the theory of stochastic processes at the undergraduate or beginning graduate level, the primary objective of this book is to initiate students in the art of stochastic modelling. However it is motivated by significant applications and progressively brings the student to the borders of contemporary research. Examples are from a wide range of domains, including operations research and electrical engineering. Researchers and students in these areas as well as in physics, biology and the social sciences will find this book of interest.

Continuous-Time Markov Chains and Applications

Author: G. George Yin
Publisher: Springer Science & Business Media
ISBN: 1461443466
Category : Mathematics
Languages : en
Pages : 442

Book Description
This book gives a systematic treatment of singularly perturbed systems that naturally arise in control and optimization, queueing networks, manufacturing systems, and financial engineering. It presents results on asymptotic expansions of solutions of Komogorov forward and backward equations, properties of functional occupation measures, exponential upper bounds, and functional limit results for Markov chains with weak and strong interactions. To bridge the gap between theory and applications, a large portion of the book is devoted to applications in controlled dynamic systems, production planning, and numerical methods for controlled Markovian systems with large-scale and complex structures in the real-world problems. This second edition has been updated throughout and includes two new chapters on asymptotic expansions of solutions for backward equations and hybrid LQG problems. The chapters on analytic and probabilistic properties of two-time-scale Markov chains have been almost completely rewritten and the notation has been streamlined and simplified. This book is written for applied mathematicians, engineers, operations researchers, and applied scientists. Selected material from the book can also be used for a one semester advanced graduate-level course in applied probability and stochastic processes.

Discrete-Time Markov Chains

Author: George Yin
Publisher: Springer Science & Business Media
ISBN: 9780387219486
Category : Business & Economics
Languages : en
Pages : 372

Book Description
Focusing on discrete-time-scale Markov chains, the contents of this book are an outgrowth of some of the authors' recent research. The motivation stems from existing and emerging applications in optimization and control of complex hybrid Markovian systems in manufacturing, wireless communication, and financial engineering. Much effort in this book is devoted to designing system models arising from these applications, analyzing them via analytic and probabilistic techniques, and developing feasible computational algorithms so as to reduce the inherent complexity. This book presents results including asymptotic expansions of probability vectors, structural properties of occupation measures, exponential bounds, aggregation and decomposition and associated limit processes, and interface of discrete-time and continuous-time systems. One of the salient features is that it contains a diverse range of applications on filtering, estimation, control, optimization, and Markov decision processes, and financial engineering. This book will be an important reference for researchers in the areas of applied probability, control theory, operations research, as well as for practitioners who use optimization techniques. Part of the book can also be used in a graduate course of applied probability, stochastic processes, and applications.

Stability Problems for Stochastic Models: Theory and Applications

Author: Alexander Zeifman
Publisher: MDPI
ISBN: 3036504524
Category : Mathematics
Languages : en
Pages : 370

Book Description
The aim of this Special Issue of Mathematics is to commemorate the outstanding Russian mathematician Vladimir Zolotarev, whose 90th birthday will be celebrated on February 27th, 2021. The present Special Issue contains a collection of new papers by participants in sessions of the International Seminar on Stability Problems for Stochastic Models founded by Zolotarev. Along with research in probability distributions theory, limit theorems of probability theory, stochastic processes, mathematical statistics, and queuing theory, this collection contains papers dealing with applications of stochastic models in modeling of pension schemes, modeling of extreme precipitation, construction of statistical indicators of scientific publication importance, and other fields.

Essentials of Stochastic Processes

Author: Richard Durrett
Publisher: Springer
ISBN: 3319456148
Category : Mathematics
Languages : en
Pages : 282

Book Description
Building upon the previous editions, this textbook is a first course in stochastic processes taken by undergraduate and graduate students (MS and PhD students from math, statistics, economics, computer science, engineering, and finance departments) who have had a course in probability theory. It covers Markov chains in discrete and continuous time, Poisson processes, renewal processes, martingales, and option pricing. One can only learn a subject by seeing it in action, so there are a large number of examples and more than 300 carefully chosen exercises to deepen the reader’s understanding. Drawing from teaching experience and student feedback, there are many new examples and problems with solutions that use TI-83 to eliminate the tedious details of solving linear equations by hand, and the collection of exercises is much improved, with many more biological examples. Originally included in previous editions, material too advanced for this first course in stochastic processes has been eliminated while treatment of other topics useful for applications has been expanded. In addition, the ordering of topics has been improved; for example, the difficult subject of martingales is delayed until its usefulness can be applied in the treatment of mathematical finance.