Abstract
In this paper, we advance the state of the art in variational image segmentation through the fusion of bottom-up segmentation and top-down classification of object behavior over an image sequence. Such an approach is beneficial for both tasks and is carried out through a joint optimization, which enables the two tasks to cooperate, such that knowledge relevant to each can aid in the resolution of the other, thereby enhancing the final result. In particular, classification offers dynamic probabilistic priors to guide segmentation, while segmentation supplies its results to classification, ensuring that they are consistent with prior knowledge. The prior models are learned from training data and updated dynamically, based on segmentations of earlier images in the sequence. We demonstrate the potential of our approach in a hand gesture recognition application, where the combined use of segmentation and classification improves robustness in the presence of occlusion and background complexity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. International Journal of Computer Vision 1, 321–331 (1987)
Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Communications in Pure and Applied Mathematics 42, 577–685 (1989)
Osher, S., Sethian, J.: Fronts propagating with curvature-dependent speed: Algorithms based on the Hamilton-Jacobi formulation. Journal of Computational Physics 79, 12–49 (1988)
Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. In: Proc. IEEE Intl. Conf. on Comp. Vis., Boston, USA, pp. 694–699. IEEE Computer Society Press, Los Alamitos (1995)
Kichenassamy, S., et al.: Gradient flows and geometric active contour models. In: Proc. IEEE Intl. Conf. on Comp. Vis., pp. 810–815. IEEE Computer Society Press, Los Alamitos (1995)
Malladi, R., Sethian, J., Vemuri, B.: Shape modeling with front propagation: A level set approach. IEEE PAMI 17, 158–175 (1995)
Vese, L., Chan, T.: A multiphase level set framework for image segmentation using the Mumford and Shah model. International Journal of Computer Vision 50(3), 271–293 (2002)
Paragios, N., Deriche, R.: Geodesic active regions and level set methods for supervised texture segmentation. International Journal of Computer Vision 46(3), 223–247 (2002)
Cootes, T., et al.: Unified framework for atlas matching using active appearance models. In: Int’l Conf. Inf. Proc. in Med. Imaging, pp. 322–333 (1999)
Leventon, M.E., Grimson, W.E.L., Faugeras, O.: Statistical shape influence in geodesic active contours. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 316–323. IEEE Computer Society Press, Los Alamitos (2000)
Tsai, A., et al.: Model-based curve evolution technique for image segmentation. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 463–468. IEEE Computer Society Press, Los Alamitos (2001)
Chen, Y., et al.: Using prior shapes in geometric active contours in a variational framework. International Journal of Computer Vision 50(3), 315–328 (2002)
Rousson, M., Paragios, N.: Shape Priors for Level Set Representations. In: Heyden, A., et al. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 78–92. Springer, Heidelberg (2002)
Cremers, D., Osher, S., Soatto, S.: Kernel density estimation and intrinsic alignment for knowledge-driven segmentation: Teaching level sets to walk. Pattern Recognition 3175, 36–44 (2004)
Cremers, D., Sochen, N., Schnör, C.: Multiphase dynamic labeling for variational recognition-driven image segmentation. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 74–86. Springer, Heidelberg (2004)
Bresson, X., Vandergheynst, P., Thiran, J.-P.: A variational model for object segmentation using boundary information and shape prior driven by the Mumford-Shah functional. International Journal of Computer Vision 28(2), 145 (2006)
Paragios, N., Deriche, R.: Geodesic active regions and level set methods for motion estimation and tracking. Computer Vision and Image Understanding 97, 259–282 (2005)
Terzopoulos, D., Szeliski, R.: Tracking with Kalman snakes. In: Active vision, pp. 3–20 (1993)
Rathi, Y., et al.: Particle filtering for geometric active contours with application to tracking moving and deforming objects. In: Proc. CVPR, vol. 2, pp. 2–9 (2005)
Cremers, D., Funka-Lea, G.: Dynamical statistical shape priors for level set based tracking. In: Paragios, N., et al. (eds.) VLSM 2005. LNCS, vol. 3752, pp. 210–221. Springer, Heidelberg (2005)
Tu, Z., et al.: Image parsing: Segmentation, detection, and recognition. In: ICCV, pp. 18–25 (2003)
Leibe, B., Leonardis, A., Schiele, B.: Combined Object Categorization and Segmentation With An Implicit Shape Model. In: ECCV Workshop on SLCV (2004)
Van Gool, L., Ferrari, V., Tuytelaars, T.: Simultaneous Object Recognition and Segmentation by Image Exploration. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 40–54. Springer, Heidelberg (2004)
Kokkinos, I., Maragos, P.: An Expectation Maximization approach to the synergy between image segmentation and object categorization. In: ICCV, pp. 617–624 (2005)
Bishop, C.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)
Rabiner, L.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of the IEEE 77(2) (1989)
Chan, T., Vese, L.: Active contours without edges. IEEE Transactions on Image Processing 10(2), 266–277 (2001)
Stenger, B., et al.: Topology-free Hidden Markov Models: Application to background modeling. In: ICCV, pp. 294–301 (2001)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Gui, L., Thiran, JP., Paragios, N. (2007). A Variational Framework for the Simultaneous Segmentation and Object Behavior Classification of Image Sequences. In: Sgallari, F., Murli, A., Paragios, N. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2007. Lecture Notes in Computer Science, vol 4485. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72823-8_56
Download citation
DOI: https://doi.org/10.1007/978-3-540-72823-8_56
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72822-1
Online ISBN: 978-3-540-72823-8
eBook Packages: Computer ScienceComputer Science (R0)