5,505 Result(s)
-
Chapter
Almost closed and sojourn sets
In this last section of Part I we return to general considerations of the evolution of the M. C. The new results will concern primarily non-recurrent (essential or inessential) states. Logically speaking, this...
-
Chapter
Definitions and measure-theoretic foundations
For general definitions, conventions and notation we refer to § I.1 and § II.1.
-
Chapter
Standard transition matrix
According to Theorem 1.3, the analytic study of (p ij ) is reduced to that of (П IJ if the set of indices F is ignored. In fact, the curtailed...
-
Chapter
Continuity properties of sample functions
In what follows we shall give further theorems concerning almost all (3) sample functions. It is important to distinguish between two kinds of assertions: (i) an assertion about x (t, ω) for a. a.ω at a fixed t w...
-
Chapter
Optional random variable
We begin by recalling the definition of a conditional expectation relative to a given field. Given the probability triple (Ω, ℱ, P), a random variable ζ with E(ζ) < ∞ and an augmented Borel subfield ℊ of ℱ, any ω
-
Chapter
Classification of states
In this section the continuous parameter analogues of the main developments of §§ I.3–7 will be given. The corresponding discrete parameter results in Part I will be used; on the other hand the content of this...
-
Chapter
Functionals
In this section we indicate how the developments of Part I, §§ 14 to 16 can be extended to the continuous parameter case. The main idea of recurrence to a fixed state and the consequent sectioning of the time ...
-
Chapter
Ratio limit theorems
In this section we apply the Laplace transform to some instances of the first entrance formulas (Theorem 11.8) to obtain analytical results which extend those of §§ I.9–10.
-
Chapter
Imbedded renewal process
In the preceding section we were concerned with the transition from a stable state i; in this section we are concerned with the transition to a stable state j. From the analytical point of view, the contrast is t...
-
Chapter
The minimal solution
In the sequel we shall use the alternative notation $$ q_{ii} = - q_i . $$
-
Chapter
Fundamental defintions
The precise definition of the term “Markov chain” as used in this monograph will be given below. However, the following remarks will help clarify our usage for the benefit of those readers who have had previou...
-
Chapter
Examples
In this section we give a number of examples to illustrate the various possibilities of sample function behavior. They imply that certain general propositions in the preceding sections are not vacuous; for ins...
-
Chapter
Classification of states
In this section certain classifications of the states with regard to their basic transition properties will be given. Further properties leading to finer classifications will appear as we proceed.
-
Chapter
The generating function
Let {a n , n≧0} be a sequence of real numbers. Its generating function is the power series $$A\left(...
-
Chapter
Criteria and examples
We start with an elementary lemma which will be useful on several occasions. Although it is but one half of the well known theorem on the regularity of Nörlund means we give its proof here.
-
Chapter
A random walk example
In this section we study in some detail a random walk scheme which is more general than the one we studied in § 5. First we develop a general method.
-
Chapter
Various complements
The quantities πi defined in §6 satisfy a certain system of linear homogeneous equations. This determining system is not only of theoretical importance but furnishes a practical way of computing these quantities.
-
Chapter
Functionals and associated random variables
In this and the next two sections we consider a M. C. {x n ,n≧0} on (Ω, ℱ, P) with the state space I forming a recurrent class. Thus for almost all ω∈Ω, the sequence {x ...
-
Chapter
Taboo probabilities
For a deeper study of the M. C. {x n , n ≧ 0} we now introduce transition probabilities with taboo states. Let H be an arbitrary set of states. We define
-
Chapter
Further limit theorems
In this section we give several more limit theorems about S n including the central limit theorem and the law of the iterated logarithm. The state space I will now be assumed to...
