SpringerLink
Log in

Prologue

  • Chapter
  • First Online:
Exercises in Applied Mathematics

Part of the book series: Chapman Mathematical Notes ((CMN))

  • 227 Accesses

Abstract

Abstract

The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Claude Shannon [387], A mathematical theory of communication (1948) There is, essentially, only one problem in statistical thermodynamics: the distribution of a given amount of energy E over N identical systems or, perhaps better, to determine the distribution of an assembly of N identical systems over the possible states in which this assembly can find itself, given that the energy of the assembly is a constant E. Erwin Schrödinger, [381, p. 1] …reconcile the time-reversibility of classical mechanics and the irreversibility of thermodynamics. Peter Whittle, on Ehrenfest model, [439, pp. 136–137]

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Thailand)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 85.59
Price includes VAT (Thailand)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
EUR 99.99
Price excludes VAT (Thailand)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Neither the original theorem nor its algebraic counterpart is discussed in the present book.

  2. 2.

    Often called positive in the book.

  3. 3.

    We will use often the terminology positive rather than positive semi-definite here; see Remark 3.5.2.

  4. 4.

    We refer to the articles [373] and [403], which set the records straight on some misconceptions on the life and death of Evariste Galois.

  5. 5.

    Two obviously contradicting goals

  6. 6.

    When the noise can be neglected, this is noiseless coding theory, a topic not considered here.

References

  1. Abbott, D., Davis, B., Phillips, N., Eshraghian, K.: Simple derivation of the thermal noise formula using window-limited fourier transforms and other conundrums. IEEE Trans. Educ. 39(1), 1–13 (1996)

    Article  Google Scholar 

  2. Ackley, D., Hinton, G., Sejnowski, T.: A learning algorithm for boltzmann machines. Cognit. Sci. 9(1), 147–169 (1985)

    Google Scholar 

  3. Aggarwal, C.: Neural Networks and Deep Learning. Springer, Berlin (2018)

    Book  Google Scholar 

  4. Alpay, D.: A Complex Analysis Problem Book. Birkhäuser/Springer Basel AG, Basel (2011)

    Google Scholar 

  5. Alpay, D.: An advanced complex analysis problem book. Topological Vector Spaces, Functional Analysis, and Hilbert Spaces of Analytic Functions. Birkhäuser/Springer Basel AG, Basel (2015)

    Google Scholar 

  6. Alpay, D.: A Complex Analysis Problem Book, 2nd edn. Birkhäuser/Springer, Cham (2016)

    Google Scholar 

  7. Alpay, D., Salomon, G.: On algebras which are inductive limits of Banach spaces. Integr. Equ. Oper. Theory 83(2), 211–229 (2015)

    Article  MathSciNet  Google Scholar 

  8. Amit, D.: Modeling Brain Function: The World of Attractor Neural Networks. Cambridge University Press, Cambridge (1992)

    Google Scholar 

  9. Artin, E.: Zur Arithmetik hyperkomplexer Zahlen. Abh. Math. Sem. Univ. Hamburg 5(1), 261–289 (1927)

    Article  MathSciNet  Google Scholar 

  10. Ash, R.B.: Information Theory. Dover Publications Inc., New York (1990). Corrected reprint of the 1965 original

    Google Scholar 

  11. Bacry, H.: Introduction aux concepts de la physique statistique. Ellipses, 32 rue Bargue, Paris 75015 (1991)

    Google Scholar 

  12. Bargmann, V.: On a Hilbert space of analytic functions and an associated integral transform. Comm. Pure Appl. Math. 14, 187–214 (1961)

    Article  MathSciNet  Google Scholar 

  13. Bargmann, V.: Remarks on a Hilbert space of analytic functions. Proc. Natl. Acad. Arts 48, 199–204 (1962)

    Article  MathSciNet  Google Scholar 

  14. Ben-Naim, A.: Information, Entropy, Life and the Universe: What We Know and What We Do Not Know. World Scientific Publishing Co., Inc., Singapore (2015)

    Book  Google Scholar 

  15. Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. Roy. Stat. Soc. Ser. B 36, 192–236 (1974)

    Article  MathSciNet  Google Scholar 

  16. Blahut, R.: Theory and Practice of Error Control Codes. Addison-Wesley, Boston (1983)

    Google Scholar 

  17. Borel, E.: Sur les principes de la théorie cinétique des gaz. Ann. Sci. École Norm. Sup. (3) 23, 9–32 (1906)

    Google Scholar 

  18. Bose, R.C., Ray-Chaudhuri, D.K.: On a class of error correcting binary group codes. Inf. Control 3, 68–79 (1960)

    Article  MathSciNet  Google Scholar 

  19. Brown, R.: XXVII. A brief account of microscopical observations made in the months of June, July and August 1827, on the particles contained in the pollen of plants; and on the general existence of active molecules in organic and inorganic bodies. Philos. Mag. 4(21), 161–173 (1828)

    Google Scholar 

  20. Brush, S.G.: A history of random processes. Arch. History Exact Sci. 5(1), 1–36 (1968). I. Brownian movement from Brown to Perrin

    Google Scholar 

  21. Choi, M.D.: Completely positive linear maps on complex matrices. Linear Algebra Appl. 10, 285–290 (1975)

    Article  MathSciNet  Google Scholar 

  22. Cohen, G., Dornstetter, J.-L., Godlewski, Ph.: Codes correcteurs d’erreurs. Une introduction au codage algébrique. Masson, Issy-les-Moulineaux (1992)

    Google Scholar 

  23. Cohen-Tannoudji, C., Diu, B., Laloë, F.: Mécanique quantique. Tome I., vol. 31. Hermann, Paris (1977)

    Google Scholar 

  24. Diu, B., Guthmann, C., Lederer, D., Roulet, B.: Thermodynamique. Hermann, Paris (2007)

    Google Scholar 

  25. Duda, R., Hart, P.: Pattern Classification and Scene Analysis, vol. 3. Wiley, New York (1973)

    Google Scholar 

  26. Einstein, A.: Eine neue bestimmung der moleküldimensionen. PhD thesis, ETH Zurich (1905)

    Google Scholar 

  27. Einstein, A.: Über die von der molekularkinetischen Theorie der Wärme geforderte Bewegung von in ruhenden Flüssigkeiten suspendierten Teilchen. Ann. Phys. 322(8), 549–560 (1905)

    Article  Google Scholar 

  28. Einstein, A.: Eine neue Bestimmung der Moleküldimensionen. Section 2: Berechnung des Reibungskoeffizienten einer Flüssigkeit, in welcher sehr viele kleine Kugeln in regelloser Verteilung suspendiert sind. Annalen der Physik IV, 297–306 (1906)

    Google Scholar 

  29. Einstein, A.: Investigations on the theory of the Brownian movement. Dover Publications, Inc., New York (1956). Edited with notes by R. Fürth, Translated by A. D. Cowper

    Google Scholar 

  30. Finger, S.: Origins of neuroscience: a history of explorations into brain function. Oxford University Press, Oxford (2001)

    Google Scholar 

  31. Gallager, R.G.: Information Theory and Reliable Communication, vol. 2. Springer, Berlin (1968)

    Google Scholar 

  32. Gibbs, J.W.: Elementary Principles in Statistical Mechanics: Developed with a Special Reference to the Rational Foundation of Thermodynamics. Dover Publications, Inc., New York (1960). First published 1902

    Google Scholar 

  33. Golay, M.J.E.: Notes on digital coding. Proc. I.R.E. 37, 657 (1949)

    MathSciNet  Google Scholar 

  34. Gouy, L.: Sur le mouvement brownien. CR Acad. Sci. Paris 109, 102–105 (1889)

    Google Scholar 

  35. Grimmett, G.R.: A theorem about random fields. Bull. London Math. Soc. 5, 81–84 (1973)

    Article  MathSciNet  Google Scholar 

  36. Guiaşu, S., Shenitzer, A.: The principle of maximum entropy. Math. Intell. 7(1), 42–48 (1985)

    Article  MathSciNet  Google Scholar 

  37. Hamming, R.W.: Error detecting and error correcting codes. Bell Syst. Tech. J. 29(2), 147–160 (1950)

    Article  MathSciNet  Google Scholar 

  38. Hamming, R.W.: Coding and Information Theory. Prentice-Hall Inc., Englewood Cliffs (1980)

    Google Scholar 

  39. Jaynes, E.T.: Where do we stand on maximum entropy? In: Maximum Entropy Formalism (Conference, Massachusetts Institute of Technology, Cambridge, MA, 1978), pp. 15–118. MIT Press, Cambridge (1979)

    Google Scholar 

  40. Joyner, D., Kim, J.L.: Selected Unsolved Problems in Coding Theory. Applied and Numerical Harmonic Analysis. Birkhäuser/Springer, New York (2011)

    Google Scholar 

  41. Kindermann, R., Snell, J.L.: Markov Random Fields and Their Applications, vol. 1. Contemporary Mathematics. American Mathematical Society, Providence (1980)

    Google Scholar 

  42. Kobayashi, M.: Hyperbolic Hopfield neural networks. IEEE Trans. Neural Netw. Learn. Syst. 24(2), 335–341 (2013)

    Article  Google Scholar 

  43. Kolmogorov, A.N.: Combinatorial foundations of information theory and the calculus of probabilities. Uspekhi Mat. Nauk. 38(4(232)), 27–36 (1983)

    Google Scholar 

  44. Langevin, P.: Sur la théorie du mouvement brownien. Compt. Rendus 146, 530–533 (1908)

    Google Scholar 

  45. Li, M., Vitányi, P.: An Introduction to Kolmogorov Complexity and Its Applications. Texts in Computer Science. Springer, Cham (2019). Fourth edition of [ MR1238938]

    Google Scholar 

  46. Lozanov Crvenković, Z., Perišić, D.: Hermite expansions of elements of Gelfand Shilov spaces in quasianalytic and non quasianalytic case. Novi Sad J. Math. 37(2), 129–147 (2007)

    MathSciNet  Google Scholar 

  47. Lumbroso, H.: Problèmes résolus de thermodynamique: physique de la matière: mathématiques supérieures et spéciales premier cycle universitaire: programme 1987. Ediscience International, Québec (1992)

    Google Scholar 

  48. Merhav, N.: Statistical physics and information theory. Found. Trends® Commun. Inf. Theory 6(1–2), 1–212 (2010)

    Google Scholar 

  49. Minsky, M.L., Papert, S.A.: Perceptrons, expanded edition. MIT Press, London (1988)

    Google Scholar 

  50. Moussa, A., Ponsonnet, P.: Cours de physique VII, Exercises de thermodynamique. Éditions André Desvigne, Lyon (1984)

    Google Scholar 

  51. Neveu, J.: Processus aléatoires gaussiens. Number 34 in Séminaires de mathématiques supérieures. Les presses de l’université de Montréal (1968)

    Google Scholar 

  52. Nilsson, N.J.: Learning Machines. Foundations of Trainable Pattern-Classifying Systems. Mc Graw Hill Book Company, New York (1965)

    Google Scholar 

  53. Ohya, M., Petz, D.: Quantum Entropy and Its Use. Texts and Monographs in Physics. Springer-Verlag, Berlin (1993)

    Google Scholar 

  54. Olazaran, M.: A sociological study of the official history of the perceptrons controversy. Social Stud. Sci. 26(3), 611–659 (1996)

    Article  Google Scholar 

  55. Parzen, E.: Modern Probability Theory and Its Applications. John Wiley & Sons Inc., Hoboken (1960)

    Book  Google Scholar 

  56. Peters, M.A., Iglesias, P.A.: Minimum Entropy Control for Time-Varying Systems. Birkhäuser Boston Inc., Boston (1997)

    Book  Google Scholar 

  57. Pless, V.: The Theory and Practice of Error Correcting Codes. Wiley, Hoboken (1989)

    Google Scholar 

  58. Reif, F.: Physique statistique. Cours de physique, Berkeley (1972)

    Google Scholar 

  59. Reza, F.M.: An Introduction to Information Theory. Dover Publications, Inc., New York (1994). Reprint of the 1961 original [Dover, New York; MR 23 # B 2082]

    Google Scholar 

  60. Rogers, A.: Supermanifolds. Theory and Applications. World Scientific, Singapore (2007)

    Google Scholar 

  61. Rothman, T.: Genius and biographers: the fictionalization of Évariste Galois. Am. Math. Monthly 89(2), 84–106 (1982)

    Article  Google Scholar 

  62. Saitta, L., Giordana, A., Cornuejols, A.: Phase Transitions in Machine Learning. Cambridge University Press, Cambridge (2011)

    Book  Google Scholar 

  63. Schrödinger, E.: Statistical Thermodynamics. A Course of Seminar Lectures Delivered in January–March 1944, at the School of Theoretical Physics, Dublin Institute for Advanced Studies, 2nd edn., reprinted. Cambridge University Press, New York (1962)

    Google Scholar 

  64. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948)

    Article  MathSciNet  Google Scholar 

  65. Sloane, N.J.A.: A Short Course on Error Correcting Codes. Springer-Verlag, Vienna (1975). International Centre for Mechanical Sciences, Courses and Lectures, No. 188

    Google Scholar 

  66. Stochel, J.B.: Representation of generalised creation and annihilation operators in Fock space. Univ. Iagel. Acta Math. 34, 135–148 (1997)

    MathSciNet  Google Scholar 

  67. Szilard, L.: Über die Entropieverminderung in einem thermodynamischen System bei Eingriffen intelligenter Wesen. Zeitschrift für Physik 53(11–12), 840–856 (1929)

    Article  Google Scholar 

  68. Taton, R.: Sur les relations scientifiques d’Augustin Cauchy et d’Evariste Galois. Rev. Histoire Sci. Appl. 24(2), 123–148 (1971)

    Article  MathSciNet  Google Scholar 

  69. Touchette, H.: The large deviation approach to statistical mechanics. Phys. Rep. 478(1–3), 1–69 (2009)

    Article  MathSciNet  Google Scholar 

  70. Tversky, A.: The Essential Tversky. MIT Press, Cambridge (2018)

    Google Scholar 

  71. Van der Pas, P.W.: The discovery of the Brownian motion. Scientiarum Historia: Tijdschrift voor de Geschiedenis van de Wetenschappen en de Geneeskunde 13(1), 27–35 (1971)

    MathSciNet  Google Scholar 

  72. van Enk, S.J.: The Brandeis Dice problem and statistical mechanics. Stud. Hist. Philos. Sci. B Stud. Hist. Philos. Mod. Phys. 48(part A), 1–6 (2014)

    Google Scholar 

  73. Vershynin, R.: High-Dimensional Probability: An Introduction with Applications in Data Science, vol. 47. Cambridge University Press, Cambridge (2018)

    Google Scholar 

  74. Von Neumann, J.: Les fondements mathématiques de la mécanique quantique, (translation from 1932 German book). Librairie Félix Alcan, Paris (1946). Éditions Jacques Gabay, 1988

    Google Scholar 

  75. Wehrl, A.: The many facets of entropy. Rep. Math. Phys. 30(1), 119–129 (1991)

    Article  MathSciNet  Google Scholar 

  76. Whittle, P.: Probability. John Wiley & Sons, Hoboken (1976)

    Google Scholar 

Download references

Acknowledgements

First of all, special thanks are due to President Daniele Struppa, for providing me the wonderful work conditions at Chapman University. It is a pleasure to thank Doctor Thomas Hempfling from Birkhäuser for his constant support and advices during the writing of this book. Special thanks are due to Professor Hesham El-Askary, Program Director of the Computational and Data Sciences program (CADS) at Chapman, for giving me the opportunity of develo** numerous courses (such as error-correcting codes, information theory, machine learning) on which is based the present work.

It is a pleasure to take this opportunity to thank my thesis advisor, Professor Harry Dym, for his help since my student days at the Weizmann Institute of Sciences. His influence can be felt all over the book.

My friend and former student, Doctor Haim Attia, from the Sami Shamoon College of Engineering, Beer-Sheva, Israel, read the manuscript and found quite a few errors and misprints. It is a pleasure to thank him for his dedication.

Numerous people have helped me during the writing of this book. Influence of my various collaborations percolates throughout the book. A special thank is due to Professor Tryphon Georgiou from University of California at Irvine for very fruitful discussions. Professor Dan Volok provided in particular the proof of Exercise 5.2.7, 2.4.4, but I owe him much more due to our collaboration over the years. Professor Mihaela Vajiac contributed Exercise 4.1.10 in particular and much more, thanks to our collaboration and countless discussions, which began in 2012. Professors Mihaela Vajiac and Adrian Vajiac introduced me to Artin’s work [50], and I wish to thank Prof. Adrian Vajiac for his help with Exercise 2.2.13 and much more along the years since 2012.

Three teachers had a huge influence during my formative years before graduate school: Monsieur Venet, in fifth grade during the academic year 1966–1967 at the École Pihet, Paris; Monsieur Jean-Daniel Bloch, who was my teacher in Mathématiques supérieures at the Lycée Louis-Le-Grand during the academic year 1973–1974; and Professor Hayri Korezlioglu, who introduced me to stochastic processes, and much more, at the École Nationale Supérieure des Télécommunications during the years 1976–1978. They are not among us anymore, but I wish to thank them heartfully.

Finally, it is a pleasure to thank my wife, Liora Mayats-Alpay, for her help and patience along the years and during the writing of this book in particular.

Author information

Authors and Affiliations

  1. Schmid College of Science and Technology, Chapman University, Orange, CA, USA

    Daniel Alpay

Authors
  1. Daniel Alpay
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Alpay, D. (2024). Prologue. In: Exercises in Applied Mathematics. Chapman Mathematical Notes. Birkhäuser, Cham. https://doi.org/10.1007/978-3-031-51822-5_1

Download citation

Publish with us

Policies and ethics

Search

Navigation