Abstract
In this paper, we address the Multi-Instance-Learning (MIL) problem when bag labels are naturally represented as ordinal variables (Multi-Instance-Ordinal Regression). Moreover, we consider the case where bags are temporal sequences of ordinal instances. To model this, we propose the novel Multi-Instance Dynamic Ordinal Random Fields (MI-DORF). In this model, we treat instance-labels inside the bag as latent ordinal states. The MIL assumption is modelled by incorporating a high-order cardinality potential relating bag and instance-labels, into the energy function. We show the benefits of the proposed approach on the task of weakly-supervised pain intensity estimation from the UNBC Shoulder-Pain Database. In our experiments, the proposed approach significantly outperforms alternative non-ordinal methods that either ignore the MIL assumption, or do not model dynamic information in target data.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Total number of observations T can vary across different sequences.
- 2.
The potential with the Multinomial Logistic Regession model is defined as \(\log ( \frac{ \exp (\beta ^T_l x)}{ \sum _{ l^\prime \in L} \exp (\beta ^T_{l^\prime } x) } )\). Where all \(\mathbf {\beta _l}\) defines a linear projection for each possible ordinal value l [32].
References
Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1619–1632 (2011)
Wu, J., Zhao, Y., Zhu, J.Y., Luo, S., Tu, Z.: Milcut: a swee** line multiple instance learning paradigm for interactive image segmentation. In: Proceedings of the Computer Vision and Pattern Recognition (2014)
Ruiz, A., Van de Weijer, J., Binefa, X.: Regularized multi-concept MIL for weakly-supervised facial behavior categorization. In: Proceedings of the British Machine Vision Conference (2014)
Maron, O., Lozano-Pérez, T.: A framework for multiple-instance learning. In: Advances in Neural Information Processing Systems (1998)
Ray, S., Page, D.: Multiple instance regression. In: Proceedings of the International Conference on Machine Learning (2001)
Lucey, P., Cohn, J.F., Prkachin, K.M., Solomon, P.E., Matthews, I.: Painful data: the UNBC-McMaster shoulder pain expression archive database. In: International Conference on Automatic Face and Gesture Recognition (2011)
Aung, M.S., Kaltwang, S., Romera-Paredes, B., Martinez, B., Singh, A., Cella, M., Valstar, M.F., Meng, H., Kemp, A., Shafizadeh, M., Elkins, A.C., Kanakam, N., rothschild, A.D., Tyler, N., Watson, P.J., Williams, A.C., Pantic, M., Bianchi-berthouze, N.: The automatic detection of chronic pain-related expression: requirements, challenges and a multimodal dataset. IEEE Trans. Affect. Comput. (2015, to appear)
Hjermstad, M.J., Fayers, P.M., Haugen, D.F., Caraceni, A., Hanks, G.W., Loge, J.H., Fainsinger, R., Aass, N., Kaasa, S., EPCRC, E.P.C.R.C., et al.: Studies comparing numerical rating scales, verbal rating scales, and visual analogue scales for assessment of pain intensity in adults: a systematic literature review. J. Pain Symptom Manag. 41, 1073–1093 (2011)
Rudovic, O., Pavlovic, V., Pantic, M.: Automatic pain intensity estimation with heteroscedastic conditional ordinal random fields. In: Bebis, G., et al. (eds.) ISVC 2013. LNCS, vol. 8034, pp. 234–243. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41939-3_23
Kaltwang, S., Rudovic, O., Pantic, M.: Continuous pain intensity estimation from facial expressions. In: Bebis, G., et al. (eds.) ISVC 2012. LNCS, vol. 7432, pp. 368–377. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33191-6_36
Wu, C., Wang, S., Ji, Q.: Multi-instance hidden Markov model for facial expression recognition. In: International Conference on Automatic Face and Gesture Recognition (2015)
Sikka, K., Dhall, A., Bartlett, M.: Weakly supervised pain localization using multiple instance learning. In: International Conference on Automatic Face and Gesture Recognition (2013)
Kim, M., Pavlovic, V.: Hidden conditional ordinal random fields for sequence classification. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS (LNAI), vol. 6322, pp. 51–65. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15883-4_4
Liu, J., Chen, C., Zhu, Y., Liu, W., Metaxas, D.N.: Video classification via weakly supervised sequence modeling. Comput. Vis. Image Underst. 152, 79–87 (2015)
Barber, D.: Bayesian Reasoning and Machine Learning. Cambridge University Press, Cambridge (2012)
Amores, J.: Multiple instance classification: review, taxonomy and comparative study. Artif. Intell. 201, 81–105 (2013)
Gärtner, T., Flach, P.A., Kowalczyk, A., Smola, A.J.: Multi-instance kernels. In: Proceedings of the International Conference on Machine Learning (2002)
Chen, Y., Bi, J., Wang, J.Z.: Miles: Multiple-instance learning via embedded instance selection. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1931–1947 (2006)
Zhou, Z.H., Sun, Y.Y., Li, Y.F.: Multi-instance learning by treating instances as non-I.I.D. samples. In: Proceedings of the International Conference on Machine Learning (2009)
Wagstaff, K.L., Lane, T., Roper, A.: Multiple-instance regression with structured data. In: International Conference on Data Mining (2008)
Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Advances in Neural Information Processing Systems (2002)
Zhang, C., Platt, J.C., Viola, P.A.: Multiple instance boosting for object detection. In: Advances in Neural Information Processing Systems (2005)
Hsu, K.J., Lin, Y.Y., Chuang, Y.Y.: Augmented multiple instance regression for inferring object contours in bounding boxes. IEEE Trans. Image Process. 23, 1722–1736 (2014)
Hajimirsadeghi, H., Li, J., Mori, G., Zaki, M., Sayed, T.: Multiple instance learning by discriminative training of Markov networks. In: Uncertainty in Artificial Intelligence (2013)
Quattoni, A., Wang, S., Morency, L.P., Collins, M., Darrell, T.: Hidden conditional random fields. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1848–1852 (2007)
Rabiner, L.R., Juang, B.H.: An introduction to hidden Markov models. ASSP Mag. 3, 4–16 (1986)
Winkelmann, R., Boes, S.: Analysis of Microdata. Springer Science Business Media, Berlin (2006)
Gupta, R., Diwan, A.A., Sarawagi, S.: Efficient inference with cardinality-based clique potentials. In: Proceedings of the International Conference on Machine Learning (2007)
Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasi-newton matrices and their use in limited memory methods. Math. Program. 63, 129–156 (1994)
Tarlow, D., Swersky, K., Zemel, R.S., Adams, R.P.: Fast exact inference for recursive cardinality models. In: Conference on Uncertainty in Artificial Intelligence (2012)
Ray, S., Craven, M.: Supervised versus multiple instance learning: an empirical comparison. In: Proceedings of the International Conference on Machine Learning (2005)
Walecki, R., Rudovic, O., Pavlovic, V., Pantic, M.: Variable-state latent conditional random fields for facial expression recognition and action unit detection. In: International Conference on Automatic Face and Gesture Recognition (2015)
Prkachin, K.M.: The consistency of facial expressions of pain: a comparison across modalities. Pain 51, 297–306 (1992)
Xuehan-**ong, D., la Torre, F.: Supervised descent method and its application to face alignment. In: Proceedings of the Computer Vision and Pattern Recognition (2013)
Rudovic, O., Pavlovic, V., Pantic, M.: Context-sensitive dynamic ordinal regression for intensity estimation of facial action units. IEEE Trans. Pattern Anal. Mach. Intell. 37, 944–958 (2015)
Acknowledgement
This paper is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grants agreement no. 645012 (KRISTINA), no. 645094 (SEWA) and no. 688835 (DE-ENIGMA). Adria Ruiz would also like to acknowledge Spanish Government to provide support under grant FPU13/01740.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Ruiz, A., Rudovic, O., Binefa, X., Pantic, M. (2017). Multi-Instance Dynamic Ordinal Random Fields for Weakly-Supervised Pain Intensity Estimation. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10112. Springer, Cham. https://doi.org/10.1007/978-3-319-54184-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-54184-6_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54183-9
Online ISBN: 978-3-319-54184-6
eBook Packages: Computer ScienceComputer Science (R0)