Approximate Modeling

Abstract

Data modeling using a linear static model class leads to an unstructured low-rank approximation problem. When the approximation criterion is the Frobenius norm, the problem can be solved by the singular value decomposition. In general, however, unstructured low-rank approximation is a difficult nonconvex optimization problem. Section 4.1 presents a local optimization algorithm, based on alternating projections. An advantage of the alternating projections algorithm is that it is easily generalizable to missing data, nonnegativity, and other constraints on the data. approximate modeling problem leads to structured low-rank approximation. Section 4.2 presents a variable projection algorithm for affine structured low-rank approximation. This method is well suited for problems where one dimension of the matrix is small and the other one is large. This is the case in system identification, where the small dimension is determined by the model’s complexity and the large dimension is determined by the number of samples. The alternating projections and variable projection algorithms perform local optimization. An alternative approach for structured low-rank approximation is to solve a convex relaxation of the problem. A convex relaxation based on replacement of the rank constraint by a constraint on the nuclear norm is shown in Sect. 4.3. The nuclear norm heuristic is also applicable for matrix completion (missing data estimation). Section 4.4 generalize the variable projection method to solve low-rank matrix completion and approximation problems.

In general no simple relationships are satisfied exactly by the data. This discrepancy between observed data and simple relationships is often modelled by introducing stochastics. However, instead of stochastic uncertainty it is in our opinion primarily the complexity of reality which often prevents existence of simple exact models. In this case model errors do not reflect chance, but arise because a simple model can only give an approximate representation of complex systems.

Heij (1989)