Abstract
Recommender Systems (RS) pervade many aspects of our everyday digital life. Proposed to work at scale, state-of-the-art RS allow the modeling of thousands of interactions and facilitate highly individualized recommendations. Conceptually, many RS can be viewed as instances of statistical regression models that incorporate complex feature effects and potentially non-Gaussian outcomes. Such structured regression models, including time-aware varying coefficients models, are, however, limited in their applicability to categorical effects and inclusion of a large number of interactions. Here, we propose Factorized Structured Regression (FaStR) for scalable varying coefficient models. FaStR overcomes limitations of general regression models for large-scale data by combining structured additive regression and factorization approaches in a neural network-based model implementation. This fusion provides a scalable framework for the estimation of statistical models in previously infeasible data settings. Empirical results confirm that the estimation of varying coefficients of our approach is on par with state-of-the-art regression techniques, while scaling notably better and also being competitive with other time-aware RS in terms of prediction performance. We illustrate FaStR’s performance and interpretability on a large-scale behavioral study with smartphone user data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abadi, M., et al.: TensorFlow: a system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 265–283 (2016)
Al-Hadi, I., Sharef, N.M., Sulaiman, M.N., Mustapha, N.: Review of the temporal recommendation system with matrix factorization. Int. J. Innov. Comput. Inf. Control 13(5), 1579–1594 (2017)
Baltrunas, L., Ricci, F.: Experimental evaluation of context-dependent collaborative filtering using item splitting. User Model. User-Adap. Inter. 24, 7–34 (2013). https://doi.org/10.1007/s11257-012-9137-9
Blondel, M., Fu**o, A., Ueda, N., Ishihata, M.: Higher-order factorization machines. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016). https://doi.org/10.5555/3157382.3157473
Campos, P.G., Díez, F., Cantador, I.: Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols. User Model. User-Adap. Inter. 24, 67–119 (2013). https://doi.org/10.1007/s11257-012-9136-x
Chen, T., Yin, H., Nguyen, Q.V.H., Peng, W.C., Li, X., Zhou, X.: Sequence-aware factorization machines for temporal predictive analytics. In: 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp. 1405–1416. IEEE (2020). https://doi.org/10.1109/ICDE48307.2020.00125
Chua, F.C.T., Oentaryo, R.J., Lim, E.P.: Modeling temporal adoptions using dynamic matrix factorization. In: 2013 IEEE 13th International Conference on Data Mining, pp. 91–100 (2013). https://doi.org/10.1109/ICDM.2013.25
Condli, M.K., Lewis, D.D., Madigan, D., Posse, C.: Bayesian mixed-effects models for recommender systems. In: ACM SIGIR, vol. 99 (1999)
Currie, I.D., Durban, M., Eilers, P.H.: Generalized linear array models with applications to multidimensional smoothing. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 68(2), 259–280 (2006). https://doi.org/10.1111/j.1467-9868.2006.00543.x
Guo, Y., Su, Y., Yang, Z., Zhang, A.: Explainable recommendation systems by generalized additive models with manifest and latent interactions (2020)
Harper, F.M., Konstan, J.A.: The MovieLens datasets: history and context. ACM Trans. Interact. Intell. Syst. (TIIS) 5(4), 1–19 (2015). https://doi.org/10.1145/2827872
Hastie, T., Tibshirani, R.: Varying-coefficient models. J. Roy. Stat. Soc.: Ser. B (Methodol.) 55(4), 757–779 (1993). https://doi.org/10.1111/j.2517-6161.1993.tb01939.x
Imaizumi, M., Hayashi, K.: Tensor decomposition with smoothness. In: International Conference on Machine Learning, pp. 1597–1606. PMLR (2017). https://doi.org/10.5555/3305381.3305546
Koren, Y.: Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 426–434 (2008). https://doi.org/10.1145/1401890.1401944
Koren, Y.: Collaborative filtering with temporal dynamics. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 447–456 (2009). https://doi.org/10.1145/1721654.1721677
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009). https://doi.org/10.1109/MC.2009.263
Nelder, J.A., Wedderburn, R.W.: Generalized linear models. J. R. Stat. Soc. Set. A (Gen.) 135(3), 370–384 (1972). https://doi.org/10.2307/2344614
R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2021)
Rendle, S.: Factorization machines. In: 2010 IEEE International Conference on Data Mining, pp. 995–1000. IEEE (2010). https://doi.org/10.1109/ICDM.2010.127
Rendle, S., Krichene, W., Zhang, L., Anderson, J.: Neural collaborative filtering vs. matrix factorization revisited. In: Fourteenth ACM Conference on Recommender Systems, pp. 240–248 (2020). https://doi.org/10.1145/3383313.3412488
Rendle, S., Zhang, L., Koren, Y.: On the difficulty of evaluating baselines: a study on recommender systems. ar**v preprint ar**v:1905.01395 (2019). https://doi.org/10.48550/ARXIV.1905.01395
Rügamer, D., Kolb, C., Klein, N.: Semi-structured deep distributional regression: a combination of additive models and deep learning. ar**v preprint ar**v:2002.05777 (2020). https://doi.org/10.48550/ARXIV.2002.05777
Stachl, C., et al.: Predicting personality from patterns of behavior collected with smartphones. Proc. Natl. Acad. Sci. 117, 17680–17687 (2020). https://doi.org/10.1073/pnas.1920484117
Thorat, P.B., Goudar, R., Barve, S.: Survey on collaborative filtering, content-based filtering and hybrid recommendation system. Int. J. Comput. Appl. 110(4), 31–36 (2015). https://doi.org/10.5120/19308-0760
Wood, S.N.: Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. J. R. Stat. Soc. (B) 73(1), 3–36 (2011). https://doi.org/10.1111/j.1467-9868.2010.00749.x
Wood, S.N.: Generalized Additive Models: An Introduction with R. Chapman and Hall/CRC, New York (2017). https://doi.org/10.1201/9781315370279
Wood, S.N.: Rejoinder on: Inference and computation with Generalized Additive Models and their extensions. TEST 29(2), 354–358 (2020). https://doi.org/10.1007/s11749-020-00716-0
Wood, S.N., Li, Z., Shaddick, G., Augustin, N.H.: Generalized additive models for GigaData: modeling the u.k. black smoke network daily data. J. Am. Stat. Assoc. 112(519), 1199–1210 (2017). https://doi.org/10.1080/01621459.2016.1195744
Wu, C., Lian, D., Ge, Y., Zhu, Z., Chen, E., Yuan, S.: Fight fire with fire: towards robust recommender systems via adversarial poisoning training. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1074–1083, SIGIR 2021. Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3404835.3462914
Yu, H.F., Rao, N., Dhillon, I.S.: Temporal regularized matrix factorization for high-dimensional time series prediction. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016)
Zhang, X., Zhou, Y., Ma, Y., Chen, B.C., Zhang, L., Agarwal, D.: GLMix: generalized linear mixed models for large-scale response prediction. In: KDD 2016, pp. 363–372. Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2939672.2939684
Zhang, Y., Bi, X., Tang, N., Qu, A.: Dynamic tensor recommender systems. J. Mach. Learn. Res. 22(65), 1–35 (2021). https://doi.org/10.11159/icsta19.09
Zhao, W.X., et al.: RecBole: towards a unified, comprehensive and efficient framework for recommendation algorithms (2020). https://doi.org/10.1145/3459637.3482016
Acknowledgement
This work has been partially supported by the German Federal Ministry of Education and Research (BMBF) under Grant No. 01IS18036A. We also thank four anonymous reviewers for their helpful suggestions and comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rügamer, D. et al. (2023). Factorized Structured Regression for Large-Scale Varying Coefficient Models. In: Amini, MR., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., Tsoumakas, G. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2022. Lecture Notes in Computer Science(), vol 13717. Springer, Cham. https://doi.org/10.1007/978-3-031-26419-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-26419-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26418-4
Online ISBN: 978-3-031-26419-1
eBook Packages: Computer ScienceComputer Science (R0)
