SpringerLink
Log in

Object panorama construction using large-parallax images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Conventional panorama techniques create a wide-angle image by stitching images taken from the same viewpoint. In contrast, the method proposed in this work produces an unwrapped surface image of a three-dimensional spherical object. Traditionally, in order to construct a panoramic image including multiple faces of an object, consecutive video frames must be captured around the object so that images with small parallax can be stitched together to avoid ghost artifacts. In this study, however, we use only two input images taken from different viewpoints to construct the panoramic surface image of a spherical object. This kind of constraint can occur when the cameras have limitation on changing their poses. The acquired two input images have a larger parallax than the video frames. Therefore, in order to align the overlap** regions of the large-parallax images, an image-morphing method with a curved interpolation line is proposed. The interpolation curve is designed for a spherical target object and it reduces dent distortion. As image morphing is highly vulnerable to feature mismatches, the corresponding features in the parallax images are paired by active feature matching using a structured light. During image composition, the seam boundary that minimizes ghost effects at the transition between images is determined based on image similarity. The experimental results for large-parallax images with an angle difference of 60° demonstrate the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Not applicable.

References

  1. Ahn B, Koo HI, Kim HI, Jeong J, Cho NI (2015) Efficient unwrap representation of faces for video editing. IEEE Signal Process Lett 22(10):1718–1722

    Article  Google Scholar 

  2. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  3. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522

    Article  Google Scholar 

  4. Bergen T, Wittenberg T (2014) Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform 20(1):304–321

    Article  Google Scholar 

  5. Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vis 74(1):59–73

    Article  Google Scholar 

  6. Chang C-H, Sato Y, Chuang Y-Y (2014) Shape-preserving half-projective warps for image stitching. IEEE Conference on Computer Vision and Pattern Recognition 3254–3261

  7. Dang TK, Worring M, Bui TD (2011) A semi-interactive panorama based 3D reconstruction framework for indoor scenes. Comput Vis Image Underst 115(11):1516–1524

    Article  Google Scholar 

  8. Delaunay B (1934) Sur la sphere vide. Otdelenie Matematicheskii i Estestvennyka Nauk 7(793–800):1–2

    MATH  Google Scholar 

  9. Dogan H, Ekinci M (2014) Automatic panorama with auto-focusing based on image fusion for microscopic imaging system. SIViP 8(1):5–20

    Article  Google Scholar 

  10. Dong S, Wang P, Abbasa K (2021) A survey on deep learning and its applications. Comput Sci Rev 40:100379

    Article  MathSciNet  Google Scholar 

  11. Dzwierzynska J (2016) Direct construction of an inverse panorama from a moving view point. Procedia Eng 161:1608–1614

    Article  Google Scholar 

  12. Dzwierzynska J (2017) A conical perspective image of an architectural object close to human perception. IOP Conf ies Mater Sci Eng 245:052099

    Article  Google Scholar 

  13. Dzwierzynska J (2019) Computer-aided inverse panorama on a conical projection surface. Inverse Probl Sci Eng 27(7):863–886

    Article  MathSciNet  Google Scholar 

  14. Fang X, Zhu J, Luo B (2012) Image mosaic with relaxed motion. SIViP 6(4):647–667

    Article  Google Scholar 

  15. Gao J, Kim SJ, Brown MS (2011) Constructing image panoramas using dual-homography war**. IEEE Conference on Computer Vision and Pattern Recognition 49–56

  16. Hernandez-Lopez FJ, Trejo-Sánchez JA, Rivera M (2020) Panorama construction using binary trees. SIViP 14:1–8

    Article  Google Scholar 

  17. Jung K, Hong J (2021) Quantitative assessment method of image stitching performance based on estimation of planar parallax. IEEE Access 9:6152–6163

    Article  Google Scholar 

  18. Jung K, Kang D, Kekatpure AL, Adikrishna A, Hong J, Jeon I (2016) A new wide-angle arthroscopic system: a comparative study with a conventional 30° arthroscopic system. Knee Surg Sports Traumatol Arthrosc 24(5):1722–1729

    Article  Google Scholar 

  19. Knorr M (2018) Self-Calibration of Multi-Camera Systems for Vehicle Surround Sensing, vol 41. KIT Scientific Publishing

    Google Scholar 

  20. Kong L (2019) Research on construction and implementation of panoramic multimedia video information space model in big data environment. Multimed Tools Appl 53(4):2533–2552

    Google Scholar 

  21. Kopf J, Uyttendaele M, Deussen O, Cohen MF (2007) Capturing and viewing gigapixel images. ACM Trans Graph 26(3):93-es

    Article  Google Scholar 

  22. Lanman D, Taubin G (2009) Build your own 3D scanner: optical triangulation for beginners. ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia 1–94

  23. Li J, Wang Z, Lai S, Zhai Y, Zhang M (2017) Parallax-tolerant image stitching based on robust elastic war**. IEEE Transactions on Multimedia 20(7):1672–1687

    Article  Google Scholar 

  24. Liao J, Lima RS, Nehab D, Hoppe H, Sander PV, Yu J (2014) Automating image morphing using structural similarity on a halfway domain. ACM Trans Graph 33(5):1–12

    Article  Google Scholar 

  25. Lin W-Y, Liu S, Matsushita Y, Ng T-T, Cheong L-F (2011) Smoothly varying affine stitching. IEEE Conference on Computer Vision and Pattern Recognition 345–352

  26. Lin C-C, Pankanti SU, Natesan Ramamurthy K, Aravkin AY (2015) Adaptive as-natural-as-possible image stitching. IEEE Conference on Computer Vision and Pattern Recognition 1155–1163

  27. Liu J, Wang B, Hu W, Sun P, Li J, Duan H, Si J (2015) Global and local panoramic views for gastroscopy: an assisted method of gastroscopic lesion surveillance. IEEE Trans Biomed Eng 62(9):2296–2307

    Article  Google Scholar 

  28. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  29. Matas J, Chum O, Urban M, Pajdla T (2004) Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput 22(10):761–767

    Article  Google Scholar 

  30. Microsoft Image Composite Editor, (n.d.) https://www.microsoft.com/en-us/research/product/computational-photography-applications/image-composite-editor/. Accessed April 22 2019

  31. Mikolajczyk K, Schmid C (2004) Scale & affine invariant interest point detectors. Int J Comput Vis 60(1):63–86

    Article  Google Scholar 

  32. Mikolajczyk K, Tuytelaars T, Schmid C, Zisserman A, Matas J, Schaffalitzky F, Kadir T, Van Gool L (2005) A comparison of affine region detectors. Int J Comput Vis 65(1–2):43–72

    Article  Google Scholar 

  33. Mishkin D, Matas J, Perdoch M (2015) MODS: fast and robust method for two-view matching. Comput Vis Image Underst 141:81–93

    Article  Google Scholar 

  34. Moreels P, Perona P (2007) Evaluation of features detectors and descriptors based on 3d objects. Int J Comput Vis 73(3):263–284

    Article  Google Scholar 

  35. Parke FI (1980) Adaptation of scan and slit-scan techniques to computer animation. 7th annual conference on Computer graphics and interactive techniques, 178–181

  36. Peleg S, Herman J (1997) Panoramic mosaics by manifold projection. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition 338–343

  37. Qi Z, Cooperstock J (2007) Overcoming parallax and sampling density issues in image mosaicing of non-planar scenes. British Machine Vision Conference

  38. Rav-Acha A, Kohli P, Rother C, Fitzgibbon A (2008) Unwrap mosaics: a new representation for video editing. ACM SIGGRAPH conference and exhibition on computer graphics and interactive techniques in Asia 1-11

  39. Seitz SM, Dyer CR (1996) View morphing. 23th Annual Conference on Computer Graphics and Interactive Techniques 21–30

  40. Szeliski R (2006) Image alignment and stitching: a tutorial. Found Trends® Comput Graph Vis 2(1):1–104

    MathSciNet  MATH  Google Scholar 

  41. Szeliski R, Shum H-Y (1997) Creating full view panoramic image mosaics and environment maps. 24th Annual Conference on Computer Graphics and Interactive Techniques 251–258

  42. Tzavidas S, Katsaggelos AK (2005) A multicamera setup for generating stereo panoramic video. IEEE Trans Multimedia 7(5):880–890

    Article  Google Scholar 

  43. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  44. Weibel T, Daul C, Wolf D, Rösch R, Guillemin F (2012) Graph based construction of textured large field of view mosaics for bladder cancer diagnosis. Pattern Recogn 45(12):4138–4150

    Article  Google Scholar 

  45. Williams L (2006) Performance-driven facial animation. ACM SIGGRAPH conference and exhibition on computer graphics and interactive techniques in Asia 16-es

  46. Wolberg G (1998) Image morphing: a survey. Vis Comput 14(8–9):360–372

    Article  Google Scholar 

  47. **ao J, Shah M (2004) Tri-view morphing. Comput Vis Image Underst 96(3):345–366

    Article  Google Scholar 

  48. **ong Y, Pulli K (2010) Fast panorama stitching for high-quality panoramic images on mobile phones. IEEE Trans Consumer Electronics 56(2):298–306

  49. Xue W, Zhang L, Mou X, Bovik AC (2013) Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans Image Process 23(2):684–695

    Article  MathSciNet  Google Scholar 

  50. Yu G, Morel J-M (2011) ASIFT: an algorithm for fully affine invariant comparison. Image Processing On Line 1:11–38

    Article  Google Scholar 

  51. Zaragoza J, Chin T-J, Brown MS, Suter D (2013) As-projective-as-possible image stitching with moving DLT. IEEE Conference on Computer Vision and Pattern Recognition 2339–2346

  52. Zhang Q, Jung J, Won J, Cho J (2011) Object panorama creation based on a general photographing environment. 5th international conference on ubiquitous information management and communication, 1-6

  53. Zheng J, Wang Y, Wang H, Li B, Hu H-M (2019) A novel projective-consistent plane based image stitching method. IEEE Trans Multimedia 21(10):2561–2575

    Article  Google Scholar 

  54. Zhu Z, Riseman EM, Hanson AR (2001) Parallel-perspective stereo mosaics. 8th IEEE international conference on computer vision 345-352

Download references

Code availability

Not applicable.

Funding

This work was supported by the Health and Medical R&D Program of the Ministry of Health and Welfare of Korea (HI13C1634) and by the National Research Foundation of Korea (NRF) grant funded by the Korea government (Ministry of Science and ICT) (2020R1A2C2100012).

Author information

Authors and Affiliations

  1. Department of Robotics Engineering, DGIST, 333, Techno jungang-daero, Hyeonpung-eup, Dalseong-gun, Daegu, 42988, Republic of Korea

    Kyunghwa Jung, Ho-Gun Ha & Jaesung Hong

  2. Department of Orthopaedic Surgery, University of Ulsan, Asan Medical Center, 88, Olympic-ro 43-gil, Songpa-gu, Seoul, 05505, Republic of Korea

    In-Ho Jeon

Authors
  1. Kyunghwa Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ho-Gun Ha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. In-Ho Jeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jaesung Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jaesung Hong.

Ethics declarations

Conflicts of interest/competing interests

Not applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jung, K., Ha, HG., Jeon, IH. et al. Object panorama construction using large-parallax images. Multimed Tools Appl 81, 39059–39075 (2022). https://doi.org/10.1007/s11042-022-13134-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13134-1

Keywords

Search

Navigation