Abstract
We address the problem of single image super-resolution by exploring the manifold properties. Given a set of low resolution image patches and their corresponding high resolution patches, we assume they respectively reside on two non-linear manifolds that have similar locally-linear structure. This manifold correlation can be realized by a three-layer Markov network that connects performing super-resolution with energy minimization. The main advantage of our approach is that by working directly with the network model, there is no need to actually construct the map**s for the underlying manifolds. To achieve such efficiency, we establish an energy minimization model for the network that directly accounts for the expected property entailed by the manifold assumption. The resulting energy function has two nice properties for super-resolution. First, the function is convex so that the optimization can be efficiently done. Second, it can be shown to be an upper bound of the reconstruction error by our algorithm. Thus, minimizing the energy function automatically guarantees a lower reconstruction error— an important characteristic for promising stable super-resolution results.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Baker, S., Kanade, T.: Limits on Super-Resolution and How to Break Them. IEEE Trans. Pattern Analysis and Machine Intelligence 24(9), 1167–1183 (2002)
Cao, F.: Good Continuations in Digital Image Level Lines. In: International Conference on Computer Vision, pp. 440–447 (2003)
Chang, H., Yeung, D.Y., **ong, Y.: Super-Resolution through Neighbor Embedding. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 275–282 (2004)
Desolneux, A., Moisan, L., Morel, J.: Partial Gestalts. Tech. Rep., CMLA (2001)
Elad, M., Feuer, A.: Restoration of a Single Superresolution Image from Several Blurred, Noisy, and Undersampled Measured Images. IEEE Trans. Image Processing 6(12), 1646–1658 (1997)
Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and Robust Multiframe Super Resolution. IEEE Trans. Image Processing 13(10), 1327–1344 (2004)
Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Computer Graphics and Applications 22(2), 56–65 (2002)
Fridman, A.: Mixed Markov Models, Ph.D. thesis, Department of Mathematics, Brown University (2000)
Guo, C.E., Zhu, S.C., Wu, Y.N.: Towards a Mathematical Theory of Primal Sketch and Sketchability. In: International Conference on Computer Vision, pp. 1228–1235 (2003)
Hardie, R.C., Barnard, K.J., Armstrong, E.E.: Joint MAP Registration and High Resolution Image Estimation Using a Sequence of Undersampled Images. IEEE Trans. Image Processing 6(12), 1621–1633 (1997)
Hertzmann, A., Jacobs, C.E., Oliver, N., Curless, B., Salesin, D.H.: Image Analogies. In: Proc. of ACM SIGGRAPH 2001, pp. 327–340 (2001)
Huang, T.S., Tsai, R.Y.: Multi-Frame Image Restoration and Registration. Advances in Computer Vision and Image Processing 1, 317–339 (1984)
Irani, M., Peleg, S.: Improving Resolution by Image Registration. GMIP 53, 231–239 (1991)
Lee, A.B., Pedersen, K.S., Mumford, D.: The Nonlinear Statistics of High-Contrast Patches in Natural Images. Int’l J. Computer Vision 54(1-3), 83–103 (2003)
Pickup, L.C., Roberts, S.J., Zisserman, A.: A Sampled Texture Prior for Image Super-Resolution. In: Advances in Neural Information Processing Systems 16 (2004)
Roweis, S.T., Saul, L.K.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290, 2323–2326 (2000)
Schultz, R.R., Stevenson, R.L.: Extraction of High-Resolution Frames from Video Sequences. IEEE Trans. Image Processing 5(6), 996–1011 (1996)
Stark, H., Oskoui, P.: High Resolution Image Recovery form Image-Plane Arrays, Using Convex Projections. J. Opt. Soc. Am. A 6, 1715–1726 (1989)
Sun, J., Zheng, N.N., Tao, H., Shum, H.Y.: Image Hallucination with Primal Sketch Priors. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 729–736 (2003)
Youla, D.C.: Generalized Image Restoration by the Method of Alternating Projections. IEEE Trans. Circuits Syst. CAS-25, 694–702 (1978)
Zomet, A., Peleg, S.: Efficient Super-Resolution and Applications to Mosaics. In: International Conference on Pattern Recognition, vol. I, pp. 579–583 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chang, TL., Liu, TL., Chuang, JH. (2006). Direct Energy Minimization for Super-Resolution on Nonlinear Manifolds. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3954. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744085_22
Download citation
DOI: https://doi.org/10.1007/11744085_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33838-3
Online ISBN: 978-3-540-33839-0
eBook Packages: Computer ScienceComputer Science (R0)