Search
Search Results
-
Markov Processes
A Markov processMarkov process is a dynamical system where movement within the system consists of transitions between a finite set of states. These...
-
Markov processes on quasi-random graphs
We study Markov population processes on large graphs, with the local state transition rates of a single vertex being a linear function of its...
-
Constrained Markov Decision Processes with Non-constant Discount Factor
This paper studies constrained Markov decision processes under the total expected discounted cost optimality criterion, with a state-action dependent...
-
Mixing times and hitting times for general Markov processes
The hitting and mixing times are two fundamental quantities associated with Markov chains. In Peres and Sousi [PS15] and Oliveira [Oli12], the...
-
Continuous Parameter Jump Markov Processes
Let \(\{X_t:t\ge 0\}\) be a Markov process on a measurable...
-
Risk-Sensitive Markov Decision Processes
Traditionally, Markov decision processes are Markov processes whose transition law is controlled by a decision maker aiming at a maximization of...
-
The structure of the local time of Markov processes indexed by Lévy trees
We construct an additive functional playing the role of the local time—at a fixed point x —for Markov processes indexed by Lévy trees. We start by...
-
Markov decision processes with risk-sensitive criteria: an overview
The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term ’risk-sensitive’ refers here to...
-
Risk-Sensitivity Vanishing Limit for Controlled Markov Processes
In this paper, we prove that the optimal risk-sensitive reward for Markov decision processes with compact state space and action space converges to...
-
Variational Principles for Asymptotic Variance of General Markov Processes
A variational formula for the asymptotic variance of general Markov processes is obtained. As application, we get an upper bound of the mean exit...
-
Continuous–Time Markov Control Processes
As noted in Remark 4.7(b), the solution \(x(\cdot )\) of...
-
Asymptotic Properties of Extremal Markov Processes Driven by Kendall Convolution
This paper is devoted to the analysis of the finite-dimensional distributions and asymptotic behavior of extremal Markov processes connected with the...
-
Strong Feller semigroups and Markov processes: a counterexample
The aim of this note is to show, by providing an elementary way to construct counterexamples, that the strong Feller and the joint (space-time)...
-
A general approach for Parisian stop** times under Markov processes
We propose a method based on continuous-time Markov chain (CTMC) approximation to compute the distribution of Parisian stop** times and to price...
-
Path continuity of Markov processes and locality of Kolmogorov operators
We prove that if we are given a generator of a right Markov process with càdlàg paths and an open domain G in the state space, on which the generator...
-
Existence and Uniqueness of Denumerable Markov Processes with Instantaneous States
Based on the resolvent decomposition theorems presented very recently by Chen (J Theor Probab 33:2089–2118, 2020), in this paper we focus on...
-
On maximizing probabilities for over-performing a target for Markov decision processes
This paper studies the dual relation between risk-sensitive control and large deviation control of maximizing the probability for out-performing a...
-
A General Family of Branching Markov Processes
Recall that, in Chap. 2 , we introduced the notion of a general Markov process on E which is taken to...
-
The Central Limit Theorem for Markov Processes that are Exponentially Ergodic in the Bounded-Lipschitz Norm
In this paper, we establish a version of the central limit theorem for Markov–Feller continuous time processes (with a Polish state space) that are...
-
On the Existence and Uniqueness of Stationary Distributions for Some Piecewise Deterministic Markov Processes with State-Dependent Jump Intensity
In this paper, we consider a subclass of piecewise deterministic Markov processes with a Polish state space that involve a deterministic motion...