Abstract
The proliferation of business data and on-demand computing have propelled the use of artificial intelligence methods in quantitative forecasting. Machine learning has a prominent role in solving clustering and classification problems as well as dimensionality reduction. Nevertheless, traditional statistical methods of forecasting continue to perform well in competitions and many practical applications. The chapter considers critically the successes of machine learning in forecasting, using some case studies as well as theoretical considerations, including limitations on machine learning and other techniques for forecasting. It also discusses weaknesses of the Vapnik–Chervonenkis theory. The main aim of the chapter is to stimulate scholarly dialogue on the role of machine learning in forecasting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Q.v. Thomas Burnett at the American Association for the Advancement of Science (AAAS) https://www.aaas.org/programs/dialogue-science-ethics-and-religion/what-scientism for example.
- 3.
Vegard Flovik writing on Medium at https://towardsdatascience.com/how-not-to-use-machine-learning-for-time-series-forecasting-avoiding-the-pitfalls-19f9d7adf424 for example.
References
Abu-Mostafa, Yaser S. 1989. The Vapnik-Chervonenkis Dimension: Information Versus Complexity in Learning. Neural Computation 1 (3): 312–317. https://doi.org/10.1162/neco.1989.1.3.312.
Ackermann, Klaus, Lauren Haynes, Rayid Ghani, Joe Walsh, Adolfo De Unánue, Hareem Naveed, Andrea Navarrete Rivera, et al. 2018. Deploying Machine Learning Models for Public Policy. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining—KDD 18. ACM Press. https://doi.org/10.1145/3219819.3219911.
Ahmed, Nesreen K., Amir F. Atiya, Neamat El Gayar, and Hisham El-Shishiny. 2010. An Empirical Comparison of Machine Learning Models for Time Series Forecasting. Econometric Reviews 29 (5–6): 594–621. https://doi.org/10.1080/07474938.2010.481556.
Amat, Fernando, Ashok Chandrashekar, Tony Jebara, and Justin Basilico. 2018. Artwork Personalization at Netflix. In Proceedings of the 12th ACM Conference on Recommender Systems—RecSys 18. ACM Press. https://doi.org/10.1145/3240323.3241729.
Armstrong, J. Scott, and Kesten C. Green. 2019. Why Didn’t Experts Pick M4-Competition Winner? https://repository.upenn.edu/marketing_papers/431.
Armstrong, J. Scott, Kesten C. Green, and Andreas Graefe. 2015. Golden Rule of Forecasting: Be Conservative. Journal of Business Research 68 (8): 1717–1731. https://doi.org/10.1016/j.jbusres.2015.03.031.
Aziz, Saqib, and Michael Dowling. 2019. Machine Learning and AI for Risk Management. In Disrupting Finance: FinTech and Strategy in the 21st Century, ed. Theo Lynn, John G. Mooney, Pierangelo Rosati, and Mark Cummins, 33–50. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-02330-0_3.
Baziar, Sadegh, Mehdi Tadayoni, Majid Nabi-Bidhendi, and Mohsen Khalili. 2014. Prediction of Permeability in a Tight Gas Reservoir by Using Three Soft Computing Approaches: A Comparative Study. Journal of Natural Gas Science and Engineering 21 (November): 718–724. https://doi.org/10.1016/j.jngse.2014.09.037.
Beam, Andrew L., and Isaac S. Kohane. 2018. Big Data and Machine Learning in Health Care. JAMA 319 (13): 1317–1318. https://doi.org/10.1001/jama.2017.18391.
Ben-David, Shai, Pavel Hrubeš, Shay Moran, Amir Shpilka, and Amir Yehudayoff. 2019. Author Correction: Learnability Can Be Undecidable. Nature Machine Intelligence 1 (2): 121. https://doi.org/10.1038/s42256-019-0023-6.
Bequé, Artem, and Stefan Lessmann. 2017. Extreme Learning Machines for Credit Scoring: An Empirical Evaluation. Expert Systems with Applications 86 (November): 42–53. https://doi.org/10.1016/j.eswa.2017.05.050.
Bontempi, Gianluca, Souhaib Ben Taieb, and Yann-Aël Le Borgne. 2013. Machine Learning Strategies for Time Series Forecasting. In Business Intelligence: Second European Summer School, EBISS 2012, Brussels, Belgium, July 15–21, 2012, Tutorial Lectures, ed. Marie-Aude Aufaure, and Esteban Zimányi, 62–77. Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-36318-4_3.
Brattka, Vasco, Stéphane Le Roux, Joseph S. Miller, and Arno Pauly. 2016. The Brouwer Fixed Point Theorem Revisited. In Pursuit of the Universal, 58–67. Springer International Publishing. https://doi.org/10.1007/978-3-319-40189-8_6.
Butaru, Florentin, Qingqing Chen, Brian Clark, Sanmay Das, Andrew W. Lo, and Akhtar Siddique. 2016. Risk and Risk Management in the Credit Card Industry. Journal of Banking & Finance 72 (November): 218–239. https://doi.org/10.1016/j.jbankfin.2016.07.015.
Buzaglo, Sarit, Rom Pinchasi, and Günter Rote. 2012. Topological Hypergraphs. In Thirty Essays on Geometric Graph Theory, 71–81. New York: Springer. https://doi.org/10.1007/978-1-4614-0110-0_6.
Einhorn, Hillel J. 1972. Alchemy in the Behavioral Sciences. Public Opinion Quarterly 36 (3): 367. https://doi.org/10.1086/268019.
Esteva, Andre, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean. 2019. A Guide to Deep Learning in Healthcare. Nature Medicine 25 (1): 24–29. https://doi.org/10.1038/s41591-018-0316-z.
Fouché, W.L., and P.H. Potgieter. 1998. Kolmogorov Complexity and Symmetric Relational Structures. The Journal of Symbolic Logic 63 (03): 1083–1094. https://doi.org/10.2307/2586728.
Gneiting, Tilmann, and Martin Schlather. 2004. Stochastic Models That Separate Fractal Dimension and the Hurst Effect. SIAM Review 46 (2): 269–282. https://doi.org/10.1137/s0036144501394387.
Gogas, Periklis, Theophilos Papadimitriou, and Anna Agrapetidou. 2018. Forecasting Bank Failures and Stress Testing: A Machine Learning Approach. International Journal of Forecasting 34 (3): 440–455. https://doi.org/10.1016/j.ijforecast.2018.01.009.
Guyon, I., N. Matić, and V. Vapnik. 1994. Discovering Informative Patterns and Data Cleaning. In Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining, 145–156. AAAIWS’94. Seattle, WA: AAAI Press. http://dl.acm.org/citation.cfm?id=3000850.3000866.
Howell, Bronwyn E., and Petrus H. Potgieter. 2018. Bundles of Trouble: Can Competition Law Adapt to Digital Pricing Innovation? Competition and Regulation in Network Industries 19 (1–2): 3–24. https://doi.org/10.1177/1783591718801102.
Hyndman, Rob J., and Anne B. Koehler. 2006. Another Look at Measures of Forecast Accuracy. International Journal of Forecasting 22 (4): 679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001.
Kim, Hyun Hak, and Norman R. Swanson. 2018. Mining Big Data Using Parsimonious Factor, Machine Learning, Variable Selection and Shrinkage Methods. International Journal of Forecasting 34 (2): 339–354. https://doi.org/10.1016/j.ijforecast.2016.02.012.
Klimek, Peter, Sebastian Poledna, and Stefan Thurner. 2019. Quantifying Economic Resilience from Input–output Susceptibility to Improve Predictions of Economic Growth and Recovery. Nature Communications 10 (1). https://doi.org/10.1038/s41467-019-09357-w.
Lázaro, Jorge López, Álvaro Barbero Jiménez, and Akiko Takeda. 2018. Improving Cash Logistics in Bank Branches by Coupling Machine Learning and Robust Optimization. Expert Systems with Applications 92 (February): 236–255. https://doi.org/10.1016/j.eswa.2017.09.043.
Lessmann, Stefan, Bart Baesens, Hsin-Vonn Seow, and Lyn C. Thomas. 2015. Benchmarking State-of-the-Art Classification Algorithms for Credit Scoring: An Update of Research. European Journal of Operational Research 247 (1): 124–136. https://doi.org/10.1016/j.ejor.2015.05.030.
Madden, Gary, Nicholas Apergis, Paul Rappoport, and Aniruddha Banerjee. 2017. An Application of Nonparametric Regression to Missing Data in Large Market Surveys. Journal of Applied Statistics 45 (7): 1292–1302. https://doi.org/10.1080/02664763.2017.1369498.
Makridakis, Spyros, Evangelos Spiliotis, and Vassilios Assimakopoulos. 2018a. Statistical and Machine Learning Forecasting Methods: Concerns and Ways Forward. Edited by Alejandro Raul Hernandez Montoya. PLOS ONE 13 (3): e0194889. https://doi.org/10.1371/journal.pone.0194889.
Makridakis, Spyros, Evangelos Spiliotis, and Vassilios Assimakopoulos. 2018b. The M4 Competition: Results, Findings, Conclusion and Way Forward. International Journal of Forecasting 34 (4): 802–808. https://doi.org/10.1016/j.ijforecast.2018.06.001.
Metcalf, Evan B. 1975. Secretary Hoover and the Emergence of Macroeconomic Management. Business History Review 49 (01): 60–80. https://doi.org/10.2307/3112962.
Moon, Jihoon, **woong Park, Eenjun Hwang, and Sanghoon Jun. 2017. Forecasting Power Consumption for Higher Educational Institutions Based on Machine Learning. The Journal of Supercomputing 74 (8): 3778–3800. https://doi.org/10.1007/s11227-017-2022-x.
Mullainathan, Sendhil, and Jann Spiess. 2017. Machine Learning: An Applied Econometric Approach. Journal of Economic Perspectives 31 (2): 87–106. https://doi.org/10.1257/jep.31.2.87.
Narayanan, Arvind, and Vitaly Shmatikov. 2006. How to Break Anonymity of the Netflix Prize Dataset. CoRR abs/cs/0610105. http://arxiv.org/abs/cs/0610105.
Nikolopoulos, Konstantinos, and Fotios Petropoulos. 2018. Forecasting for Big Data: Does Suboptimality Matter? Computers & Operations Research 98 (October): 322–329. https://doi.org/10.1016/j.cor.2017.05.007.
Sutcliffe, John, Stephen Hurst, Ayman G. Awadallah, Emma Brown, and Khaled Hamed. 2016. Harold Edwin Hurst: The Nile and Egypt, Past and Future. Hydrological Sciences Journal 61 (9): 1557–1570. https://doi.org/10.1080/02626667.2015.1019508.
White, Charles P. 1928. Industrial Forecasting. The ANNALS of the American Academy of Political and Social Science 139 (1): 109–125. https://doi.org/10.1177/000271622813900115.
Acknowledgements
This project has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 731143 for Computing with Infinite Data.
The author is grateful to the late Gary Madden for innumerable stimulating conversations on many topics that have influenced the author’s thinking and also to Patricia Madden for much kindness and hospitality in Perth and to both of them for friendship and good cheer in many locations around the world.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Author(s)
About this chapter
Cite this chapter
Potgieter, P.H. (2020). Machine Learning and Forecasting: A Review. In: Alleman, J., Rappoport, P., Hamoudia, M. (eds) Applied Economics in the Digital Era. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-40601-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-40601-1_8
Published:
Publisher Name: Palgrave Macmillan, Cham
Print ISBN: 978-3-030-40600-4
Online ISBN: 978-3-030-40601-1
eBook Packages: Economics and FinanceEconomics and Finance (R0)