-
Article
Revisiting the ODE Method for Recursive Algorithms: Fast Convergence Using Quasi Stochastic Approximation
Several decades ago, Profs. Sean Meyn and Lei Guo were postdoctoral fellows at ANU, where they shared interest in recursive algorithms. It seems fitting to celebrate Lei Guo’s 60th birthday with a review of the O...
-
Chapter
Fundamental Design Principles for Reinforcement Learning Algorithms
Along with the sharp increase in visibility of the field, the rate at which new reinforcement learning algorithms are being proposed is at a new peak. While the surge in activity is creating excitement and opp...
-
Chapter
Distributed Control Design for Balancing the Grid Using Flexible Loads
Inexpensive energy from the wind and the sun comes with unwanted volatility, such as ramps with the setting sun or a gust of wind. Controllable generators manage supply-demand balance of power today, but this ...
-
Article
Large deviations for the empirical mean of an \(M/M/1\) queue
Let \((Q(k):k\ge 0)\) be an \(M/M/1\) ...
-
Chapter
Dynamic Competitive Equilibria in Electricity Markets
This chapter addresses the economic theory of electricity markets, viewed from an idealized competitive equilibrium setting, taking into account volatility and the physical and operational constraints inherent...
-
Article
On exponential ergodicity of multiclass queueing networks
One of the key performance measures in queueing systems is the decay rate of the steady-state tail probabilities of the queue lengths. It is known that if a corresponding fluid model is stable and the stochast...
-
Chapter and Conference Paper
Model-Based Real-Time Estimation of Building Occupancy During Emergency Egress
This paper provides a viable and practical solution to the challenge of real-time estimation of the number of people in areas of a building, during an emergency egress situation. Such estimates would be extrem...
-
Article
Coding and control for communication networks
The purpose of this paper is to survey techniques for constructing effective policies for controlling complex networks, and to extend these techniques to capture special features of wireless communication netw...
-
Article
Dynamic Safety-Stocks for Asymptotic Optimality in Stochastic Networks
This paper concerns control of stochastic networks using state-dependent safety-stocks.
-
Article
In Search of Sensitivity in Network Optimization
This paper concerns the following questions regarding policy synthesis in large queueing networks: (i) It is well known that an understanding of variability is important in the determination of safety stocks t...
-
Article
Value iteration and optimization of multiclass queueing networks
This paper considers in parallel the scheduling problem for multiclass queueing networks, and optimization of Markov decision processes. It is shown that the value iteration algorithm may perform poorly when t...
-
Article
Algorithms for optimization and stabilization of controlled Markov chains
This article reviews some recent results by the author on the optimal control of Markov chains. Two common algorithms for the construction of optimal policies are considered: value iteration and policy iteration.
