SpringerLink
Log in

A robust hybrid image-based modeling system

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper presents a new robust image-based modeling system for creating high-quality 3D models of complex objects from a sequence of unconstrained photographs. The images can be acquired by a video camera or hand-held digital camera without the need of camera calibration. In contrast to previous methods, we integrate correspondence-based and silhouette-based approaches, which significantly enhances the reconstruction of objects with few visual features (e.g., uni-colored objects) and improves surface smoothness. Our solution uses a mesh segmentation and charting approach in order to create a low-distortion mesh parameterization suitable for objects of arbitrary genus. A high-quality texture is produced by first parameterizing the reconstructed objects using a segmentation and charting approach, projecting suitable sections of input images onto the model, and combining them using a graph-cut technique. Holes in the texture due to surface patches without projecting input images are filled using a novel exemplar-based inpainting method which exploits appearance space attributes to improve patch search, and blends patches using Poisson-guided interpolation. We analyzed the effect of different algorithm parameters, and compared our system with a laser scanning-based reconstruction and existing commercial systems. Our results indicate that our system is robust, superior to other image-based modeling techniques, and can achieve a reconstruction quality visually not discernible from that of a laser scanner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Nguyen, M.H., Wunsche, B., Delmas, P., Lutteroth, C.: A hybrid image-based modelling algorithm. In: Proceedings of 36th Australasian Computer Science Conference (ACSC 2013) (2013)

  2. Nguyen, H.M., Wunsche, B., Delmas, P., Lutteroth, C.: 3D models from the black box: investigating the current state of image-based modeling. In: WSCG 2012 Communication Proceedings, pp. 249–258 (2012)

  3. Hernandez, C., Vogiatzis, G., Cipolla, R.: Multi-view photometric stereo. IEEE Trans. Pattern Recognit. Mach. Intell. 30, 548–554 (2008)

    Article  Google Scholar 

  4. Franco, J.-S., Lapierre, M., Boyer, E.: Visual shapes of silhouette sets. In: 3D Data Processing, Visualization and Transmission, pp. 397–404 (2006)

  5. Matusik, W., Buehler, C., Raskar, R., Gortler, S., McMillan, L.: Image-based visual hulls. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 369–374 (2000)

  6. Nguyen, M.H., Wunsche, B., Delmas, P., Lutteroth, C.: Realistic 3D scene reconstruction from unconstrained and uncalibrated images. In: Proceedings of GRAPP 2011, Algarve, Portugal, vol. 31, pp. 67–75 (2011)

  7. Baumgart, B.G.: Geometric modeling for computer vision. Doctoral Dissertation, Stanford University (1974)

  8. Grauman, K., Shakhnarovich, G., Darrell, T.: A Bayesian approach to image-based visual hull reconstruction. In: IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 187–194 (2003)

  9. Cheung, K., Baker, S., Kanade, T.: Shape-from-silhouette across time part 1: theory and algorithms. Int. J. Comput. Vis. 62(1), 221–247 (2005)

    Article  Google Scholar 

  10. Cheung, K., Baker, S., Kanade, T.: Shape-from-silhouette across time part 2: applications to human modeling and markerless motion tracking. Int. J. Comput. Vis. 63(1), 225–245 (2005)

    Article  Google Scholar 

  11. Franco, J.S., Boyer, E.: Exact polyhedral visual hulls. In: British Machine Vision Conference, pp. 329–338 (2003)

  12. Debevec, P.E., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: a hybrid geometry and image-based approach. ACM Trans Graph. pp. 11–20 (1996)

  13. Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., Kang, S.B.: Image-based plant modeling. ACM Trans. Graph. 25(3), 599–604 (2006)

    Article  Google Scholar 

  14. Brown, M., Lowe, D.G.: Unsupervised 3D object recognition and reconstruction in unordered datasets. In: Fifth International Conference on 3D Digital Imaging and Modeling, pp. 56–63 (2005)

  15. Snavely, N., Seitz, S., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. 25(3), 835–846 (2006)

    Article  Google Scholar 

  16. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  17. Lowe, D.G.: Object recognition from local scale-invariant features. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999)

  18. Nguyen, H.M.: Accelerated 3D Content Creation using Stereo from Motion. Master Thesis. The University of Auckland, New Zealand (2012)

  19. Remondino, F., El-Hakim, S.: Image-based 3D modelling: a review. Photogramm. Rec. 21, 269–291 (2006)

    Article  Google Scholar 

  20. Garai, G., Chaudhuri, B.B.: A split and merge procedure for polygonal border detection of dot pattern. Image Vis. Comput. 17, 75–82 (1999)

    Article  Google Scholar 

  21. Amenta, N., Choi, S., Kolluri, R.K.: The power crust. Int. J. Comput. Geom. Theory Appl. 19, 127–153 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  22. Edelsbrunner, H., Mucke, E.P.: Three-dimensional alpha shapes. ACM Trans. Graph. 13, 43–72 (1994)

    Article  MATH  Google Scholar 

  23. Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C., Taubin, G.: The ball-pivoting algorithm for surface reconstruction. IEEE Trans. Vis. Comput. Graph. 5(4), 349–359 (1999)

    Article  Google Scholar 

  24. Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the Fourth Eurographics Symposium on Geometry Processing, pp. 61–70 (2006)

  25. Zhang, E., Mischaikow, K., Turk, G.: Feature-based surface parameterization and texture map**. ACM Trans. Graph. 24(1), 1–27 (2005)

    Article  Google Scholar 

  26. Reeb, Georges: Sur les points singuliers dune forme de pfaff completement integrable ou diune fonction numerique [on the (singular points of a completely integrable pfaff form or of a numerical function)]. Comptes Randus Acad. Sci. Paris 222, 847–849 (1946)

    MathSciNet  MATH  Google Scholar 

  27. Hilaga, M., Shinagawa, Y., Komura, T., Kunii, T.L.: Topology matching for fully automatic similarity estimation of 3D shapes. In: Computer Graphics Proceedings, Annual Conference Series (SIGGRAPH 2001), pp. 203–212 (2001)

  28. Eck, M., DeRose, T., Duchamp, T., Hoppey, H., Lounsberyz, M., Stuetzle, W.: Multi-resolution analysis of arbitrary meshes. In: Computer Graphics Proceedings, Computer Graphics Proceedings, Annual Conference Series (SIGGRAPH 1995), pp. 173–182 (1995)

  29. Floater, S.M.: Parametrization and smooth approximation of surface triangulations. Comput. Aided Geom. Des. 14(3), 231–250 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  30. Sander, P.V., Gortler, S.J., Snyder, J., Hoppe, H.: Signal-specialized parameterization, In: Proceedings of the 13th Eurographics Workshop on Rendering, pp. 87–100 (2002)

  31. Kwata, V., Schodl, A., Essaa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003)

    Article  Google Scholar 

  32. Clark, X.B., Finlay, J., Wilson, A., Milburn, K., Nguyen, M.H., Lutteroth, C., Wunsche, B.C.: An investigation into graphcut parameter optimisation for image-fusion applications. In: Proceedings of Image and Vision Computing New Zealand (IVCNZ 2012), pp. 480–485 (2012)

  33. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceeding SIGGRAPH ’00 Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424 (2000)

  34. Telea, A.: An image inpainting technique based on the fast marching method. J. Graph. Tools 9(1), 23–34 (2004)

    Article  Google Scholar 

  35. Perez, P., Gangnet, M., Blake, A.: Poisson image editing. J. ACM Trans. Graph. 22(3), 313–318 (2003)

    Article  Google Scholar 

  36. Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. IEEE Trans. Image Process. 19(9), 1200–1212 (2004)

    Article  Google Scholar 

  37. Harrison, P.: A non-hierarchical procedure for re-synthesis of complex texture. In: Proceedings of International Conference on Graphics, Visualisation and Computer Vision, pp. 190–197 (2001)

  38. Manke, F., Wunsche, B.: Fast spatially controllable 2D/3D texture synthesis and morphing for multiple input textures. In: Proceedings of the 4th International Conference on Computer Graphics Theory and Applications (GRAPP 2009), pp. 5–12 (2009)

  39. Lefebvre, S., Hoppe, H.: Appearance-space texture synthesis. In: ACM SIGGRAPH, pp. 541–548 (2006)

  40. Nguyen, H.M., Wunsche, B., Delmas, P., Lutteroth, C.: Parameter optimisation for texture reconstruction. In: Proceedings of Image and Vision Computing New Zealand (IVCNZ 2013), pp. 226–230 (2013)

  41. Stanford Bunny. https://graphics.stanford.edu/data/3Dscanrep/. Accessed 1st May 2014

  42. Rakhmanov, E.A., Saff, E.B., Zhou, Y.M.: Minimal discrete energy on the sphere. Math. Res. Lett. 1(6), 647–662 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  43. Rusinkiewicz, S., Levoy, M.: Efficient Variants of the ICP Algorithm. In: Proceeding of Third International Conference on 3D Digital Imaging and Modeling (3DIM), pp. 145–152 (2001)

  44. Zambanini, S., Kampel, M.: A local image descriptor robust to illumination changes. In: Proceedings of the 18th Scandinavian Conference on Image Analysis, pp. 11–21 (2013)

Download references

Author information

Authors and Affiliations

  1. University of Auckland, Auckland, New Zealand

    Hoang Minh Nguyen, Burkhard Wünsche, Patrice Delmas & Christof Lutteroth

  2. Oregon State University, Corvallis, USA

    Eugene Zhang

Authors
  1. Hoang Minh Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Burkhard Wünsche
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrice Delmas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christof Lutteroth
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eugene Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hoang Minh Nguyen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nguyen, H.M., Wünsche, B., Delmas, P. et al. A robust hybrid image-based modeling system. Vis Comput 32, 625–640 (2016). https://doi.org/10.1007/s00371-015-1078-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-015-1078-y

Keywords

Search

Navigation