Abstract
Lensless image reconstruction is an ill-posed inverse problem in computational imaging, having several applications in machine vision. Existing approaches rely on large datasets for learning to perform deconvolution and are often specific to the point spread function of a particular lensless imager. Generating pairs of lensless images and their corresponding ground truths requires a specialized laboratory setup, thus making the dataset collection procedure challenging. We propose a reconstruction method using untrained neural networks that relies on the underlying physics of lensless image generation. We use an encoder-decoder network for reconstructing the lensless image for a known PSF. The same network can predict the PSF when supplied with a single example of input and ground-truth pair, thus acting as a one-time calibration step for any lensless imager. We used a physics-guided consistency loss function to optimize our model to perform reconstruction and PSF estimation. Our model generates accurate non-blind reconstructions with a PSNR of 24.55 dB.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Antipa, N., et al.: Diffusercam: lensless single-exposure 3d imaging. Optica 5(1), 1–9 (2018)
Salman Asif, M., Ayremlou, A., Sankaranarayanan, S.C., Veeraraghavan, A., Baraniuk, R.: FlatCam: Thin, lensless cameras using coded aperture and computation. IEEE Trans. Comput. Imag. 3(3), 384–397 (2017)
Tanida, J., et al.: Thin observation module by bound optics (TOMBO): concept and experimental verification. Appl. Opt. 40(11), 1806–1813 (2001)
Gill, P.R., Lee, C., Lee, D.-G., Wang, A., Molnar, A.: A microscale camera using direct Fourier-domain scene capture. Opt. Lett. 36(15), 2949–2951 (2011)
Boominathan, V., et al.: Lensless imaging: a computational renaissance. IEEE Signal Process. Mag. 33(5), 23–35 (2016)
Kuo, G., Antipa, N., Ng, R., Waller, L.: 3D fluorescence microscopy with diffusercam. In: Imaging and Applied Optics 2018 (3D, AO, AIO, COSI, DH, IS, LACSEA, LS &C, MATH, pcAOP), page CM3E.3. Optica Publishing Group (2018)
Adams, J.K., et al.: Single-frame 3D fluorescence microscopy with ultraminiature lensless flatscope. Sci. Adv. 3(12) (2017)
Zheng, Y., Salman Asif, M.: Joint image and depth estimation with mask-based lensless cameras. In: 2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP) (2019)
Wang, Z.W., Vineet, V., Pittaluga, F., Sinha, S.N., Cossairt, O., Kang, S.B.: Privacy-preserving action recognition using coded aperture videos. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–10 (2019)
Canh, T.N., Nagahara, H.: Deep compressive sensing for visual privacy protection in flatcam imaging. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3978–3986 (2019)
Hegde, C.: Algorithmic Aspects of Inverse Problems Using Generative Models. IEEE Press (2018)
Ongie, G., Jalal, A., Metzler, C.A., Baraniuk, R.G., Dimakis, A.G., Willett, R.: Deep learning techniques for inverse problems in imaging. IEEE J. Select. Areas Inf. Theory 1(1), 39–56 (2020)
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phy. D Nonlinear Phenomena 60, 259–268 (1992)
Sinha, A., Lee, J., Li, S., Barbastathis, G.: Lensless computational imaging through deep learning. Optica 4(9), 1117–1125 (2017)
Li, Y., Xue, Y., Tian, L.: Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica 5(10), 1181–1190 (2018)
Li, S., Deng, M., Lee, J., Sinha, A., Barbastathis, G.: Imaging through glass diffusers using densely connected convolutional networks. Optica 5(7), 803–813 (2018)
Tan, J., et al.: Face detection and verification using lensless cameras. IEEE Trans. Comput. Imaging 5(2), 180–194 (2018)
Khan, S.S., Adarsh, V.R., Boominathan, V., Tan, J., Veeraraghavan, A., Mitra, K.: Towards photorealistic reconstruction of highly multiplexed lensless images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7860–7869 (2019)
Monakhova, K., Yurtsever, J., Kuo, G., Antipa, N., Yanny, K., Waller, L.: Learned reconstructions for practical mask-based lensless imaging. Opt. Express 27(20), 28075–28090 (2019)
Rego, J.D., Kulkarni, K., Jayasuriya, S.: Robust lensless image reconstruction via PSF estimation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 403–412 (2021)
Nelson, S., Menon, R.: Bijective-constrained cycle-consistent deep learning for optics-free imaging and classification. Optica 9(1), 26–31 (2022)
George Em Karniadakis: Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Phys. Inf. Mach. Learn. Nat. Rev. Phys. 3(6), 422–440 (2021)
Wang, F., et al.: Phase imaging with an untrained neural network. Light: Sci. Appl. 9(1), 1–7 (2020)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Banerjee, A., Kumar, H., Saurav, S., Singh, S. (2023). Lensless Image Reconstruction with an Untrained Neural Network. In: Yan, W.Q., Nguyen, M., Stommel, M. (eds) Image and Vision Computing. IVCNZ 2022. Lecture Notes in Computer Science, vol 13836. Springer, Cham. https://doi.org/10.1007/978-3-031-25825-1_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-25825-1_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25824-4
Online ISBN: 978-3-031-25825-1
eBook Packages: Computer ScienceComputer Science (R0)