Abstract
Implicit representations such as Neural Radiance Fields (NeRFs) have become a de facto standard in the field of novel view synthesis for 3D scenes. However, their stunning results typically imply the use of dozens of training images, with their corresponding cameras well localized in the scene. This paper studies new, total variation-based regularization approaches to train NeRFs in the context of very few (less than 10) training images. It leverages the NeRF back-propagation algorithm to evaluate first-order and second-order derivatives terms on the inferred depth map to enforce smoothness on the scene underlying surfaces. Through state-of-the-art performance on standard real-images benchmarks, we show that the proposed methods, coined as TV-NeRF and TGV-NeRF, make strong baselines in novel view synthesis with few training views.
This work received financial support from Conahcyt under grant \(\#\)319584.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
This number of iterations varies with the scenario, please refer to [11] for details.
References
Barron, J.T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., Srinivasan, P.P.: MIP-NERF: a multiscale representation for anti-aliasing neural radiance fields. In: Proceedings of the International Conference on Computer Vision (ICCV) (2021)
Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., Hedman, P.: MIP-NERF 360: unbounded anti-aliased neural radiance fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imag. Sci. 3(3), 492–526 (2010)
Deng, K., Liu, A., Zhu, J.Y., Ramanan, D.: Depth-supervised nerf: fewer views and faster training for free. ar**v preprint ar**v:2107.02791 (2021)
Kirsch, A.: An Introduction to the Mathematical Theory of Inverse Problems, vol. 120. Springer, Cham (2011). https://doi.org/10.1007/978-1-4419-8474-6
Li, R., Gao, H., Tancik, M., Kanazawa, A.: NERFACC: efficient sampling accelerates nerfs. ar**v preprint ar**v:2305.04966 (2023)
Martin-Brualla, R., Radwan, N., Sajjadi, M.S.M., Barron, J., Dosovitskiy, A., Duckworth, D.: Nerf in the wild: Neural radiance fields for unconstrained photo collections. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021). https://nerf-w.github.io/
Mildenhall, B., et al.: Local light field fusion: practical view synthesis with prescriptive sampling guidelines. ACM Trans. Graphics (TOG) 38, 1–14 (2019)
Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NERF: representing scenes as neural radiance fields for view synthesis. In: Proceedings of the European Conference on Computer Vision (ECCV) (2020)
Müller, T., Evans, A., Schied, C., Keller, A.: Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph. 41(4), 102:1-102:15 (2022)
Niemeyer, M., Barron, J.T., Mildenhall, B., Sajjadi, M.S.M., Geiger, A., Radwan, N.: Regnerf: regularizing neural radiance fields for view synthesis from sparse inputs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
Ortiz, J., et al.: ISDF: real-time neural signed distance fields for robot perception. In: Robotics: Science and Systems (2022)
Rabby, A.S.A., Zhang, C.: Beyondpixels: a comprehensive review of the evolution of neural radiance fields. ar**v e-prints pp. ar**v-2306 (2023). https://arxiv.org/abs/2306.03000
Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Schönberger, J.L., Zheng, E., Pollefeys, M., Frahm, J.M.: Pixelwise view selection for unstructured multi-view stereo. In: European Conference on Computer Vision (ECCV) (2016)
Yang, J., Pavone, M., Wang, Y.: FREENERF: improving few-shot neural rendering with free frequency regularization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zúniga, E., Batard, T., Hayet, JB. (2024). T(G)V-NeRF: A Strong Baseline in Regularized Neural Radiance Fields with Few Training Views. In: Calvo, H., Martínez-Villaseñor, L., Ponce, H. (eds) Advances in Computational Intelligence. MICAI 2023. Lecture Notes in Computer Science(), vol 14391. Springer, Cham. https://doi.org/10.1007/978-3-031-47765-2_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-47765-2_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47764-5
Online ISBN: 978-3-031-47765-2
eBook Packages: Computer ScienceComputer Science (R0)