On the Choice of Regression Basis Functions and Machine Learning

Abstract

As is known, regression-analysis tools are widely used in machine-learning problems to establish the relationship between the observed variables and to store information in a compact manner. Most often, a regression function is described by a linear combination of some given functions f_j(X),  j = 1, …, m, X ∈ D ⊂ R^s. If the observed data contain a random error, then the regression function reconstructed from the observations contains a random error and a systematic error depending on the selected functions  f_j. This article indicates the possibility of an optimal, in the sense of a given functional metric, choice of  f_j, if it is known that the true dependence obeys some functional equation. In some cases (a regular grid, s ≤ 2), close results can be obtained using a technique for random-process analysis. The numerical examples given in this work illustrate significantly broader opportunities for the assumed approach to regression problems.