Abstract
Errors are always present in predictions produced by machine learning models. Producing a quantitative estimate of the uncertainty in a model’s output is crucial for many fields, especially those where predictive models drive important decisions. In this paper we discuss two methods for producing prediction intervals for neural network, random forest, and gradient boosted tree models. We then evaluate the prediction intervals produced by each algorithm by predicting the expected ride length for a NYC taxi trip dataset. We show that inductive conformal prediction produces the most reliable intervals for all machine learning models investigated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., et al.: Tensorflow: Large-scale machine learning on heterogeneous distributed systems. ar**v preprintar**v:1603.04467 (2016)
Scott Armstrong, J.: Long-Range Forecasting: From Crystal Ball to Computer, 2nd edn. Wiley, New York (1985)
Boström, H., Linusson, H., Löfström, T., Johansson, U.: Accelerating difficulty estimation for conformal regression forests. Ann. Math. Artif. Intell. 81(1–2), 125–144 (2017)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
François, C. et al.: Keras. https://keras.io (2015)
Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat., 1189–1232 (2001)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059 (2016)
Makridakis, Spyros: Accuracy measures: theoretical and practical concerns. Int. J. Forecast. 9(4), 527–529 (1993)
Nix, D.A., Weigend, A.S.: Estimating the mean and variance of the target probability distribution. In: Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94), vol. 1, pp. 55–60. IEEE (1994)
Regression conformal prediction with nearest neighbours: Papadopoulos, Harris., Vovk, Vladimir, Gammerman, Alexander. J. Artif. Intell. Res. 40, 815–840 (2011)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Zhu, L., Laptev, N.: Deep and confident prediction for time series at uber. In: 2017 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 103–110. IEEE (2017)
Acknowledgements
The project was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Morgan, E., Zhou, R., Feng, W. (2021). Prediction Intervals of Machine Learning Models for Taxi Trip Length. In: Kilgour, D.M., Kunze, H., Makarov, R., Melnik, R., Wang, X. (eds) Recent Developments in Mathematical, Statistical and Computational Sciences. AMMCS 2019. Springer Proceedings in Mathematics & Statistics, vol 343. Springer, Cham. https://doi.org/10.1007/978-3-030-63591-6_65
Download citation
DOI: https://doi.org/10.1007/978-3-030-63591-6_65
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63590-9
Online ISBN: 978-3-030-63591-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)