Abstract
Recent work on dissimilarity-based hierarchical clustering has led to the introduction of global objective functions for this classical problem. Several standard approaches, such as average linkage clustering, as well as some new heuristics have been shown to provide approximation guarantees. Here, we introduce a broad new class of objective functions which satisfy desirable properties studied in prior work. Many common agglomerative and divisive clustering methods are shown to be greedy algorithms for these objectives, which are inspired by related concepts in phylogenetics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
We do not analyse or generate any datasets, because our work proceeds within a theoretical and mathematical approach.
Notes
Our results can also be adapted to the case where the input are similarities Throughout, we confine ourselves to dissimilarities for simplicity.
Note that we are not imposing that estimated edge lengths be positive.
Because the Gaussian and Laplace distributions allow for negative values, these models do not in fact produce a valid dissimilarity. The resulting \(\hat{h}\) however is of interest.
References
Alon, N., Azar, Y., & Vainstein, D. (2020). Hierarchical clustering: A 0.585 revenue approximation. In Proceedings of Thirty Third Conference on Learning Theory, pp. 153–162. PMLR, July 2020. ISSN: 2640-3498
Atteson, K. (1999). The performance of neighbor-joining methods of phylogenetic reconstruction. Algorithmica, 25(2), 251–278, June 1999
Bryant, D. (2005). On the uniqueness of the selection criterion in neighbor-joining. Journal of Classification, 22(1), 3–15, June 2005
Cohen-Addad, V., Kanade, V., & Mallmann-Trenn, F. (2017). Hierarchical clustering beyond the worst-case. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems 30 (pp. 6201–6209). Curran Associates Inc.
Charikar, M., & Chatziafratis, V. (2017). Approximate hierarchical clustering via sparsest cut and spreading metrics. In Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’17, pp. 841–854, Philadelphia, PA, USA, Society for Industrial and Applied Mathematics
Castro, R. M., Coates, M. J., & Nowak R. D. (2004). Likelihood based hierarchical clustering. IEEE Transactions on Signal Processing, 52(8), 2308–2321, Aug 2004
Charikar, M., Chatziafratis, V., & Niazadeh, R. (2019). Hierarchical clustering better than average-linkage. In Proceedings of the 2019 Annual ACM–SIAM Symposium on Discrete Algorithms (SODA), Proceedings, pp. 2291–2304. Society for Industrial and Applied Mathematics, January 2019
Cohen-Addad, V., Kanade, V., Mallmann-Trenn, F., & Mathieu, C. (2018). Hierarchical clustering: Objective functions and algorithms. In Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2018, New Orleans, LA, USA, January 7-10, 2018, pp. 378–397
Chatziafratis, V., Niazadeh, R., & Charikar, M. (2018). Hierarchical clustering with structural constraints. In Proceedings of the 35th International Conference on Machine Learning, pp. 774–783. PMLR, July 2018. ISSN: 2640–3498
Dasgupta, S. (2016). A cost function for similarity-based hierarchical clustering. In Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2016, Cambridge, MA, USA, June 18-21, 2016, pp. 118–127
Degens, P. O. (1983). Hierarchical cluster methods as maximum likelihood estimators (pp. 249–253). Berlin Heidelberg, Berlin, Heidelberg: Springer.
Eickmeyer, K., Huggins, P., Pachter, L., & Yoshida, R. (2008). On the optimality of the neighbor-joining algorithm. Algorithms for Molecular Biology, 3(1), 5, April 2008
Fiorini, S., & Joret, G. (2012). Approximating the balanced minimum evolution problem. Operations Research Letters, 40(1), 31–35.
Gascuel, O., & Steel, M. (2006). Neighbor-joining revealed. Molecular Biology and Evolution, 23(11), 1997–2000.
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning: Data mining, Inference, and Prediction, 2nd Edition, Springer Science & Business Media, August 2009 Google-Books-ID: tVIjmNS3Ob8C
Lacey, M. R., & Chang, J. T. (2006). A signal–to–noise analysis of phylogeny estimation by neighbor-joining: Insufficiency of polynomial length sequences. Mathematical Biosciences, 199(2), 188–215, February 2006
Mihaescu, R., Levy, D., & Pachter, Lior. (2009). Why neighbor-joining works. Algorithmica, 54(1), 1–24, May 2009
Mihaescu, R., & Pachter, L. (2008). Combinatorics of least-squares trees. Proceedings of the National Academy of Sciences, 105(36), 13206–13211, September 2008. Publisher: Proceedings of the National Academy of Sciences
Manghiuc, B.-A., & Sun, H. (2021). Hierarchical clustering: O(1)–approximation for well-clustered graphs. In advances in neural information processing systems, volume 34, pp. 9278–9289. Curran Associates, Inc
Murphy, K. P. (2012). Machine learning: A probabilistic perspective. MIT Press, September 2012. Google–Books–ID: RC43AgAAQBAJ
Pardi, F., & Gascuel, O. (2012). Combinatorics of distance-based tree inference. Proceedings of the National Academy of Sciences, 109(41), 16443–16448, October 2012. Publisher: Proceedings of the national academy of sciences
Roch, S. (2010). Toward extracting all phylogenetic information from matrices of evolutionary distances. Science, 327(5971), 1376–1379.
Roy, A., & Pokutta, S. (2016). Hierarchical clustering via spreading metrics. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, & R. Garnett (Eds.), Advances in neural information processing systems 29 (pp. 2316–2324). Curran Associates Inc.
Saitou, N., & Nei, M. (1987). The neighbor-joining method: A new method for reconstructing phylogenetic trees. Molecular Biology and Evolution, 4(4), 406–425.
Sokal, R. R. (1958). A statiscal method for evaluating systematic relationships. Univ Kans sci bull, 38, 1409–1438.
Semple, C., & Steel, M. (2003). Phylogenetics, volume 22 of mathematics and its applications series. Oxford University Press
Steel, M. (2016). Phylogeny—Discrete and random processes in evolution, volume 89 of CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA
Warnow, T. (2017). Computational phylogenetics: An introduction to designing methods for phylogeny estimation. Cambridge University Press, USA, 1st edition
Willson, S. J. (2005). Minimum evolution using ordinary least-squares is less robust than neighbor-joining. Bulletin of Mathematical Biology, 67(2), 261–279, March 2005
Acknowledgements
The author’s work was supported by NSF grants DMS-1149312 (CAREER), DMS-1614242, CCF-1740707 (TRIPODS Phase I), DMS-1902892, DMS-1916378, and DMS-2023239 (TRIPODS Phase II), as well as a Simons Fellowship and a Vilas Associates Award. Part of this work was done at MSRI and the Simons Institute for the Theory of Computing. I thank Sanjoy Dasgupta, Varun Kanade, Harrison Rosenberg, Garvesh Raskutti and Cécile Ané for helpful comments.
Funding
The author’s work was supported by NSF grants DMS-1149312 (CAREER), DMS-1614242, CCF-1740707 (TRIPODS Phase I), DMS-1902892, DMS-1916378, and DMS-2023239 (TRIPODS Phase II), as well as a Simons Fellowship and a Vilas Associates Award.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Roch, S. Expanding the Class of Global Objective Functions for Dissimilarity-Based Hierarchical Clustering. J Classif 40, 513–526 (2023). https://doi.org/10.1007/s00357-023-09447-x
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-023-09447-x