Abstract
In this note we exploit the knowledge embodied in infinitesimal generators of Markov processes to compute efficiently and economically the transient solution of continuous time Markov processes. We consider the Krylov subspace approximation method which has been analysed by Gallopoulos and Saad for solving partial differential equations and linear ordinary differential equations [7, 17]. We place special emphasis on error bounds and stepsize control. We illustrate the usefulness of the approach by providing some application examples.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
H. Abdallah and R. Marie. The uniformized power method for transient solutions of Markov processes. Computers Ops Res., 20(5):515–526, 1993.
S. Agmon, A. Douglas, and L. Nirenberg. Estimates near the boundary for solutions of elliptic partial differential equations satisfying general boundary conditions I. Comm. Pure Appl. Math., 12:623–727, 1959.
A. Berman and R. J. Plemmons. Nonnegative Matrices in the Mathematical Sciences. Academic Press, New York, 1979.
C. A. Clarotti. The Markov approach to calculating system reliability: Computational problems. In A. Serra and R. E. Barlow, editors, International School of Physics «Enrico Fermi», Varenna, Italy, pages 54–66, Amsterdam, 1984. North-Holland Physics Publishing.
W. J. Cody, G. Meinardus, and R. S. Varga. Chebyshev rational approximation to exp(—x) in [0, +∞) and applications to heat conduction problems. J. Approx. Theory,2:50–65, 1969.
I. S. Duff and J. A. Scott. Computing selected eigenvalues of sparse un-symmetric matrices using subspace iteration. ACM Trans. Math. Softw., 19(2):137–159, June 1993.
E. Gallopoulos and Y. Saad. Efficient solution of parabolic equations by Krylov approximation methods. SIAM J. Sci. Stat. Comput.,13(5):1236–1264, September 1992.
G. H. Golub and C. F. Van Loan. Matrix Computations. The Johns Hopkins University Press, Baltimore, second edition, 1989.
K. Gustafsson. Control theoretic techniques for stepsize selection in explicit Runge-Kutta methods. ACM Trans. Math. Softw., 17(4):533–554, December 1991.
M. Malhotra and K. S. Trivedi. Higher-order methods for transient analysis of stiff Markov chains. Submitted, June 1991.
C. B. Moler and C. F. Van Loan. Nineteen dubious ways to compute the exponential of a matrix. SIAM Review.,20(4):801–836, October 1978.
M. F. Neuts. Matrix Geometric Solutions in Stochastic Models: An Algorithmic Approach. The Johns Hopkins Univeristy Press, Baltimore, 1981.
B. N. Parlett. A recurrence among the elements of functions of triangular matrices. Linear Algebra Appl., 14:117–121, May 1976.
B. N. Parlett and K. C. Ng. Development of an accurate algorithm for exp(bτ). Technical Report PAM-294, Center for Pure and Applied Mathematics, University of California, Berkeley, August 1985.
B. Philippe and M. Sadkane. Improving the reliability of a deflated non hermitian generalized cigenvalue problem. In Fifth SIAM Conference on Applied Linear Algebra, June 1994.
A. Riebman and K. Trivedi. Transient analysis of cumulative measures of Markov model behavior. Commu. Statist.-Stochastic Models, 5(4):683–710, 1989.
Y. Saad. Analysis of some Krylov subspace approximations to the matrix exponential operator. SIAM J. Numer. Anal., 29(1):208–227, February 1992.
Y. Saad. Numerical Methods for Large Eigenvalue Problems. John Wiley & Sons, Manchester Univ. Press, 1992.
E. Seneta. Non-negative Matrices and Markov Chains. Springer-Verlag, New York, second edition, 1981.
R. B. Sidje. Parallel Algorithms for Large Sparse Matrix Exponentials: application to numerical transient analysis of Markov processes. PhD thesis, University of Rennes 1, July 1994.
C. F. Van Loan. The sensitivity of the matrix exponential. SIAM J. Numer. Anal., 14(6):971–981, December 1977.
R. S. Varga. Nonnegatively posed problems and completely monotonic functions. Linear Algebra and Appl., 1:329–347, 1968.
R. C. Ward. Numerical computation of the matrix exponential with accuracy estimate. SIAM J. Numer. Anal., 14(4):600–610, September 1977.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1995 Springer Science+Business Media New York
About this paper
Cite this paper
Philippe, B., Sidje, R.B. (1995). Transient Solutions of Markov Processes by Krylov Subspaces. In: Stewart, W.J. (eds) Computations with Markov Chains. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-2241-6_7
Download citation
DOI: https://doi.org/10.1007/978-1-4615-2241-6_7
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-5943-2
Online ISBN: 978-1-4615-2241-6
eBook Packages: Springer Book Archive