Continuum Armed Bandit Problem of Few Variables in High Dimensions

Abstract

We consider the stochastic and adversarial settings of continuum armed bandits where the arms are indexed by [0,1]^d. The reward functions r:[0,1]^d → ℝ are assumed to intrinsically depend on at most k coordinate variables implying $r(x_1,\dots,x_d) = g(x_{i_1},\dots,x_{i_k})$ for distinct and unknown i ₁,…,i _k  ∈ {1,…,d} and some locally Hölder continuous g:[0,1]^k → ℝ with exponent α ∈ (0,1]. Firstly, assuming (i ₁,…,i _k ) to be fixed across time, we propose a simple modification of the CAB1 algorithm where we construct the discrete set of sampling points to obtain a bound of $O(n^{\frac{\alpha+k}{2\alpha+k}} (\log n)^{\frac{\alpha}{2\alpha+k}} C(k,d))$ on the regret, with C(k,d) depending at most polynomially in k and sub-logarithmically in d. The construction is based on creating partitions of {1,…,d} into k disjoint subsets and is probabilistic, hence our result holds with high probability. Secondly we extend our results to also handle the more general case where (i ₁,…,i _k ) can change over time and derive regret bounds for the same.