SpringerLink
Log in

Universal output prediction and nonparametric regression for arbitrary data

  • Part C Modeling, Identification And Estimation
  • Conference paper
  • First Online:
Learning, control and hybrid systems

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 241))

  • 108 Accesses

Abstract

We construct a class of elementary nonparametric output predictors of an unknown discrete-time nonlinear fading memory system. Our algorithms predict asymptotically well for every bounded input sequence, every disturbance sequence in certain classes, and every linear or nonlinear system that is continuous and asymptotically time-invariant, causal, and with fading memory. The predictor is based on k n -nearest neighbor estimators from nonparametric statistics. It uses only previous input and noisy output data of the system without any knowledge of the structure of the unknown system, the bounds on the input, or the properties of noise. Under additional smoothness conditions we provide rates of convergence for the time-average errors of our scheme. Finally, we apply our results to the special case of stable LTI systems.

This work was partially supported by the National Science Foundation under NYI grant IRI-9457645.

This research was completed while Posner was at the Department of Electrical Engineering, Princeton University and the Department of Statistics, University of Toronto.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. T.M. Cover, “Estimation by the nearest neighbor rule,” IEEE Trans. Information Theory, vol. IT-14, pp. 50–55, Jan. 1968.

    Article  Google Scholar 

  2. M.A. Dahleh, E.D. Sontag, D.N.C. Tse, and J.N. Tsitsiklis, “Worst-case identification of nonlinear fading memory systems,” Automatica, vol. 31, pp. 503–508, 1995.

    Article  MATH  MathSciNet  Google Scholar 

  3. M.A. Dahleh, T. Theodosopoulos, and J.N. Tsitsiklis, “The sample complexity of worst-case identification for f.i.r. linear systems,” Syst. Contr. Lett., vol. 20, pp. 157–166, March 1993.

    Article  MATH  MathSciNet  Google Scholar 

  4. L. Devroye, “Necessary and sufficient conditions for the pointwise convergence of nearest neighbor regression function estimates,” Z. Wahrscheinlichkeitstheorie verw. Gebiete vol. 61, pp. 467–481, 1982.

    Article  MATH  MathSciNet  Google Scholar 

  5. R.M. Dudley, Real Analysis and Probability, Chapman & Hall, 1989.

    Google Scholar 

  6. J.D. Farmer and J.J. Sidorowich, “Exploiting chaos to predict the future and reduce noise,” Evolution, Learning, and Cognition, pp. 265–289, World Scientific, Singapore, 1988.

    Google Scholar 

  7. M. Feder, N. Merhav, and M. Gutman, “Universal prediction of individual sequences,” IEEE Trans. Information Theory, vol. 38, pp. 1258–1270, July 1992.

    Article  MATH  MathSciNet  Google Scholar 

  8. W. Greblicki, “Nonparametric identification of Wiener systems by orthogonal series,” IEEE Trans. Automatic Control, vol. 30, pp. 2077–2086, 1994.

    Article  MathSciNet  Google Scholar 

  9. W. Greblicki and M. Pawlak, “Dynamic system identification with order statistics,” IEEE Trans. Information Theory, vol. 40, pp. 1474–1489, 1994.

    Article  MATH  Google Scholar 

  10. L. Györfi, “The rate of convergence of k n -NN regression estimates and classification rules,” IEEE Trans. Information Theory, vol. IT-27, pp. 362–364, May 1981.

    Article  Google Scholar 

  11. A.J. Helmicki, C.A. Jacobson, C.N. Nett, “Control Oriented System Identification: A Worst-Case/Deterministic Approach in H ∞,” IEEE Trans. Automatic Control, vol. 36, Oct. 1991.

    Google Scholar 

  12. A. Krzyżak, “On estimation of a class of nonlinear systems by the kernel regression estimate,” IEEE Trans. Information Theory, vol. 36, pp. 141–152, 1990.

    Article  Google Scholar 

  13. A. Krzyżak, “Identification of nonlinear systems by recursive kernel regression estimates,” Int. J. Systems Sci., vol. 24, pp. 577–598, 1993.

    Article  Google Scholar 

  14. S.R. Kulkarni, “Data-dependent nearest neighbor and kernel estimators consistent for arbitrary processes,” preprint.

    Google Scholar 

  15. S.R. Kulkarni and S.E. Posner, “Rates of convergence of nearest neighbor estimation under arbitrary sampling,” IEEE Trans. Information Theory, pp. 1028–1039, July 1995.

    Google Scholar 

  16. S.R. Kulkarni and S.E. Posner, “Nonparametric output prediction for nonlinear fading memory systems,” to appear IEEE Trans. Automatic Control, Nov., 1998.

    Google Scholar 

  17. L. Ljung, System Identification: Theory for the User, Prentice-Hall, 1987.

    Google Scholar 

  18. P.M. Mäkilä, “Robust identification and Galois sequences,” Int. J. Contr., vol. 54, pp. 1189–1200, 1991.

    Article  MATH  Google Scholar 

  19. G. Morvai, S. Yakowitz, and L. Györfi, “Nonparametric inferences for ergodic stationary time series,” preprint.

    Google Scholar 

  20. K. Poolla and A. Tikku, “On the time complexity of worst-case system identification,” IEEE Trans. Automatic Control, vol. 39, pp. 944–950, May 1994.

    Article  MATH  MathSciNet  Google Scholar 

  21. S.E. Posner, “Nonparametric estimation, regression, and prediction under minimal regularity conditions,” Ph.D. Thesis, Department of Electrical Engineering, Princeton University, 1995.

    Google Scholar 

  22. C.J. Stone, “Consistent nonparametric regression,” Ann. Stat., vol. 8, pp. 1348–1360, 1977.

    Google Scholar 

  23. D.N.C. Tse, M.A. Dahleh, J.N. Tsitsiklis, “Optimal asymptotic identification under bounded disturbances,” IEEE Trans. Automatic Control, vol. 38, pp. 1176–1190, Aug. 1993.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Princeton University, 08544, Princeton, NJ

    S. R. Kulkarni

  2. ING Baring Securities, Inc., 10021, New York, NJ

    S. E. Posner

Authors
  1. S. R. Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. E. Posner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Yutaka Yamamoto PhD Shinji Hara PhD

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag London Limited

About this paper

Cite this paper

Kulkarni, S.R., Posner, S.E. (1999). Universal output prediction and nonparametric regression for arbitrary data. In: Yamamoto, Y., Hara, S. (eds) Learning, control and hybrid systems. Lecture Notes in Control and Information Sciences, vol 241. Springer, London. https://doi.org/10.1007/BFb0109733

Download citation

  • DOI: https://doi.org/10.1007/BFb0109733

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-85233-076-7

  • Online ISBN: 978-1-84628-533-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation