Search
Search Results
-
A fast non-monotone line search for stochastic gradient descent
We give an improved non-monotone line search algorithm for stochastic gradient descent (SGD) for functions that satisfy interpolation conditions. We...
-
Polyak’s Method Based on the Stochastic Lyapunov Function for Justifying the Consistency of Estimates Produced by a Stochastic Approximation Search Algorithm under an Unknown-but-Bounded Noise
AbstractIn 1976–1977, Polyak published in the journal Avtomatica i Telemekhanika (Automation and Remote Control) two remarkable papers on how to...
-
A Line Search Based Proximal Stochastic Gradient Algorithm with Dynamical Variance Reduction
Many optimization problems arising from machine learning applications can be cast as the minimization of the sum of two functions: the first one...
-
Expected complexity analysis of stochastic direct-search
This work presents the convergence rate analysis of stochastic variants of the broad class of direct-search methods of directional type. It...
-
Improved variance reduction extragradient method with line search for stochastic variational inequalities
In this paper, we investigate the numerical methods for solving stochastic variational inequalities. Using line search scheme, we propose an improved...
-
Sample complexity analysis for adaptive optimization algorithms with stochastic oracles
Several classical adaptive optimization algorithms, such as line search and trust-region methods, have been recently extended to stochastic settings...
-
Adaptive Sampling Stochastic Multigradient Algorithm for Stochastic Multiobjective Optimization
In this paper, we propose an adaptive sampling stochastic multigradient algorithm for solving stochastic multiobjective optimization problems....
-
Stochastic Optimization Methods
This chapter introduces some methods aimed at solving difficult optimization problems arising in many engineering fields. By difficult optimization...
-
Adaptive Sampling line search for local stochastic optimization with integer variables
We consider optimization problems with an objective function that is estimable using a Monte Carlo oracle, constraint functions that are known...
-
Stochastic Regularized Newton Methods for Nonlinear Equations
In this paper, we study stochastic regularized Newton methods to find zeros of nonlinear equations, whose exact function information is normally...
-
The continuous stochastic gradient method: part I–convergence theory
In this contribution, we present a full overview of the continuous stochastic gradient (CSG) method, including convergence results, step size rules...
-
Hesitant adaptive search with estimation and quantile adaptive search for global optimization with noise
Adaptive random search approaches have been shown to be effective for global optimization problems, where under certain conditions, the expected...
-
An efficient scenario penalization matheuristic for a stochastic scheduling problem
We propose a new scenario penalization matheuristic for a stochastic scheduling problem based on both mathematical programming models and local...
-
Multistart algorithm for identifying all optima of nonconvex stochastic functions
We propose a multistart algorithm to identify all local minima of a constrained, nonconvex stochastic optimization problem. The algorithm uniformly...
-
Online model adaptation in Monte Carlo tree search planning
We propose a model-based reinforcement learning method using Monte Carlo Tree Search planning. The approach assumes a black-box approximated model of...
-
Stochastic mesh adaptive direct search for blackbox optimization using probabilistic estimates
We present a stochastic extension of the mesh adaptive direct search (MADS) algorithm originally developed for deterministic blackbox optimization....
-
Self-adaptive heuristic algorithms for the dynamic and stochastic orienteering problem in autonomous transportation system
This paper studies a task execution and routing problem in the autonomous transportation system that maps to the famous orienteering problem (OP) in...
-
Reconstructing Unknown Coefficients of Stochastic Differential Equations and Intelligently Predicting Random Processes with Directed Learning
AbstractA way of intelligently predicting random processes is described, based on more complete use of information about statistical patterns of the...
-
Inequality constrained stochastic nonlinear optimization via active-set sequential quadratic programming
We study nonlinear optimization problems with a stochastic objective and deterministic equality and inequality constraints, which emerge in numerous...
-
Stochastic Steffensen method
Is it possible for a first-order method, i.e., only first derivatives allowed, to be quadratically convergent? For univariate loss functions, the...
