Abstract
In this paper we study the convergence of the relaxed primal-dual algorithm with critical preconditioners for solving composite monotone inclusions in real Hilbert spaces. We prove that this algorithm define Krasnosel’skiĭ-Mann (KM) iterations in the range of a particular monotone self-adjoint linear operator with non-trivial kernel. Our convergence result generalizes (Condat in J Optim Theory Appl 158: 460–479, 2013, Theorem 3.3) and follows from that of KM iterations defined in the range of linear operators, which is a real Hilbert subspace under suitable conditions. The Douglas–Rachford splitting (DRS) with a non-standard metric is written as a particular instance of the primal-dual algorithm with critical preconditioners and we recover classical results from this new perspective. We implement the algorithm in total variation reconstruction, verifying the advantages of using critical preconditioners and relaxation steps.
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
Data Availibility
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Notes
The computational saving time percentage of algorithm B with respect to algorithm A is given by \(100(\text {time} (A)- \text {time} (B))/\text {time} (A)\).
References
Aubin, J.P., Frankowska, H.: Set-valued Analysis. Modern Birkhäuser Classics, Birkhäuser Boston, Inc., Boston, MA (2009) https://doi.org/10.1007/978-0-8176-4848-0
Bauschke, H.H.: New demiclosedness principles for (firmly) nonexpansive operators. In: Bailey, D.H., Bauschke, H.H., Borwein, P., Garvan, F., Théra, M., Vanderwerff, J.D., Wolkowicz, H. (eds.) Computational and Analytical Mathematics. Springer Proceedings in Mathematics & Statistics 50, pp. 19–28. Springer, New York (2013) https://doi.org/10.1007/978-1-4614-7621-4_2
Bauschke, H.H., Combettes, P.L.: Convex analysis and monotone operator theory in Hilbert spaces. Springer, New York (2017)
Bauschke, H.H., Moursi, W.M.: On the Douglas-Rachford algorithm. Math. Program. 164, 263–284 (2017). https://doi.org/10.1007/s10107-016-1086-3
Boţ, R.I., Csetnek, E.R., Heinrich, A.: A primal-dual splitting algorithm for finding zeros of sums of maximal monotone operators. SIAM J. Optim. 23, 2011–2036 (2013). https://doi.org/10.1137/12088255X
Boţ, R.I., Hendrich, C.: A Douglas-Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM J. Optim. 23, 2541–2565 (2013). https://doi.org/10.1137/120901106
Briceño, L., Cominetti, R., Cortés, C.E., Martínez, F.: An integrated behavioral model of land use and transport system: a hyper-network equilibrium approach. Netw. Spat. Econ. 8, 201–224 (2008). https://doi.org/10.1007/s11067-007-9052-5
Briceño-Arias, L.M., Combettes, P.L.: A monotone + skew splitting model for composite monotone inclusions in duality. SIAM J. Optim. 21, 1230–1250 (2011). https://doi.org/10.1137/10081602X
Briceño-Arias, L.M., Combettes, P.L.: Monotone operator methods for Nash equilibria in non-potential games. In: Bailey, D.H., Bauschke, H. H., Borwein, P., Garvan, F., Théra, M., Vanderwerff, J., and Wolkowicz, H. (eds.) Computational and Analytical Mathematics. Springer Proceedings in Mathematics & Statistics 50, pp. 143–159. Springer, New York (2013) https://doi.org/10.1007/978-1-4614-7621-4_9
Briceño-Arias, L.M., Davis, D.: Forward-backward-half forward algorithm for solving monotone inclusions. SIAM J. Optim. 28, 2839–2871 (2018). https://doi.org/10.1137/17M1120099
Briceño-Arias, L.M., Deride, J., Vega, C.: Random activations in primal-dual splittings for monotone inclusions with a priori information. J. Optim. Theory Appl. 192, 56–81 (2022). https://doi.org/10.1007/s10957-021-01944-6
Briceño-Arias, L.M., López Rivera, S.: A projected primal-dual method for solving constrained monotone inclusions. J. Optim. Theory Appl 180, 907–924 (2019). https://doi.org/10.1007/s10957-018-1430-2
Briceño-Arias, L.M., Roldán, F.: Split-Douglas-Rachford algorithm for composite monotone inclusions and split-ADMM. SIAM J. Optim. 31, 2987–3013 (2021). https://doi.org/10.1137/21M1395144
Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vision 20, 89–97 (2004). https://doi.org/10.1023/B:JMIV.0000011320.81911.38
Chambolle, A., Caselles, V., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. In: Fornasier, M. (ed.) Theoretical Foundations and Numerical Methods for Sparse Recovery. Radon Ser. Comput. Appl. Math. 9., pp. 263–340. Walter de Gruyter, Berlin (2010) https://doi.org/10.1515/9783110226157.263
Chambolle, A., Lions, P.L.: Image recovery via total variation minimization and related problems. Numer. Math. 76, 167–188 (1997). https://doi.org/10.1007/s002110050258
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vision 40, 120–145 (2011). https://doi.org/10.1007/s10851-010-0251-1
Colas, J., Pustelnik, N., Oliver, C., Abry, P., Géminard, J.-C., Vidal, V.: Nonlinear denoising for characterization of solid friction under low confinement pressure. Phys. Rev. E 100, 032803 (2019). https://doi.org/10.1103/PhysRevE.100.032803
Combettes, P.L.: Quasi-Fejérian analysis of some optimization algorithms. In: Butnariu, D., Censor, Y., Reich, S. (eds.) Inherently parallel algorithms in feasibility and optimization and their applications. Stud. Comput. Math. 8, pp. 115–152. North-Holland, Amsterdam (2001) https://doi.org/10.1016/S1570-579X(01)80010-0
Combettes, P.L.: Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization 53, 475–504 (2004). https://doi.org/10.1080/02331930412331327157
Combettes, P.L., Pesquet, J.-C.: Primal-dual splitting algorithm for solving inclusions with mixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued Var. Anal. 20, 307–330 (2012). https://doi.org/10.1007/s11228-011-0191-y
Combettes, P.L., Vũ, B.C.: Variable metric quasi-Fejér monotonicity. Nonlinear Anal. 78, 17–31 (2013). https://doi.org/10.1016/j.na.2012.09.008
Combettes, P.L., Vũ, B.C.: Variable metric forward-backward splitting with applications to monotone inclusions in duality. Optimization 63, 1289–1318 (2014). https://doi.org/10.1080/02331934.2012.733883
Condat, L.: A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158, 460–479 (2013). https://doi.org/10.1007/s10957-012-0245-9
Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure Appl. Math. 57, 1413–1457 (2004). https://doi.org/10.1002/cpa.20042
Davis, D.: Convergence rate analysis of primal-dual splitting schemes. SIAM J. Optim. 25, 1912–1943 (2015). https://doi.org/10.1137/151003076
Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992). https://doi.org/10.1007/BF01581204
Fukushima, M.: The primal Douglas-Rachford splitting algorithm for a class of monotone map**s with application to the traffic equilibrium problem. Math. Program. 72, 1–15 (1996). https://doi.org/10.1016/0025-5610(95)00012-7
Gabay, D.: Chapter IX applications of the method of multipliers to variational inequalities. In: Fortin, M. and Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. Studies in Mathematics and Its Applications 15, pp. 299–331. Elsevier (1983) https://doi.org/10.1016/S0168-2024(08)70034-1
Gafni, E.M., Bertsekas, D.P.: Two-metric projection methods for constrained optimization. SIAM J. Control Optim. 22, 936–964 (1984). https://doi.org/10.1137/0322061
Glowinski, R., Marrocco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité, d’une classe de problèmes de Dirichlet non linéaires. Rev. Française Automat. Informat. Recherche Opérationnelle Sér. Rouge Anal. Numér. 9, 41–76 (1975)
Goldstein, A.A.: Convex programming in Hilbert space. Bull. Amer. Math. Soc. 70, 709–710 (1964). https://doi.org/10.1090/S0002-9904-1964-11178-2
Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra, and Filtering. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (2006) https://doi.org/10.1137/1.9780898718874
He, B., Yuan, X.: Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imaging Sci. 5, 119–149 (2012). https://doi.org/10.1137/100814494
Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979). https://doi.org/10.1137/0716071
Mallat, S.: A wavelet tour of signal processing: the sparse way. Elsevier/Academic Press, Amsterdam (2009)
Martinet, B.: Brève communication. régularisation d’inéquations variationnelles par approximations successives. ESAIM: Mathematical Modelling and Numerical Analysis - Modélisation Mathématique et Analyse Numérique 4, 154–158 (1970) http://www.numdam.org/item/M2AN_1970__4_3_154_0
Mises, R.V., Pollaczek-Geiringer, H.: Praktische verfahren der gleichungsauflösung. Z. Angew. Math. Mech. (1929). https://doi.org/10.1002/zamm.19290090206
Molinari, C., Peypouquet, J., Roldan, F.: Alternating forward-backward splitting for linearly constrained optimization problems. Optim. Lett. 14, 1071–1088 (2020). https://doi.org/10.1007/s11590-019-01388-y
Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: 2011 international conference on computer vision, pp. 1762–1769 (2011) https://doi.org/10.1109/ICCV.2011.6126441
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976). https://doi.org/10.1137/0314056
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D. 60, 259–268 (1992). https://doi.org/10.1016/0167-2789(92)90242-F
Showalter, R.E.: Monotone operators in banach space and nonlinear partial differential equations. Mathematical surveys and monographs 49, American mathematical society, Providence, RI (1997) https://doi.org/10.1090/surv/049
Svaiter, B.F.: On weak convergence of the Douglas-Rachford method. SIAM J. Control Optim. 49, 280–287 (2011). https://doi.org/10.1137/100788100
Vũ, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators. Adv. Comput. Math. 38, 667–681 (2013). https://doi.org/10.1007/s10444-011-9254-8
Yang, Y., Tang, Y., Wen, M., Zeng, T.: Preconditioned Douglas-Rachford type primal-dual method for solving composite monotone inclusion problems with applications. Inverse Probl. Imaging 15, 787–825 (2021). https://doi.org/10.3934/ipi.2021014
Acknowledgements
The first author thanks the support of ANID under grants FONDECYT 1190871, Redes 180032, and Centro de Modelamiento Matemático (CMM), ACE210010 and FB210005, BASAL funds for centers of excellence. The second author thanks the support of ANID-Subdirección de Capital Humano/Doctorado Nacional/2018-21181024 and by the Dirección de Postgrado y Programas from UTFSM through Programa de Incentivos a la Iniciación Científica (PIIC).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
6 Appendix
6 Appendix
Proof of Lemma 2.4:
Let \( \varvec{x} \in \mathrm{Fix}\,{\varvec{S}}.\) Since \({\varvec{S}}={\varvec{S}}\circ \varvec{Q}\) we have
which yields \(\varvec{Q}\varvec{x} \in \mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}})\). Thus, \(\varvec{x}={\varvec{S}}(\varvec{Q} \varvec{x}) \in {\varvec{S}}(\mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}}))\) and we conclude \(\mathrm{Fix}\,{\varvec{S}}\subset {\varvec{S}}( \mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}}))\). Conversely, let \(\varvec{x} \in \mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}})\). Since \({\varvec{S}}={\varvec{S}}\circ \varvec{Q}\), we have
Thus \( {\varvec{S}}( \mathrm{Fix}\,(\varvec{Q}\circ {\varvec{S}})) \subset \mathrm{Fix}\,{\varvec{S}}\) and the result follows. \(\square \)
Proof of Proposition 3.1:
1: It is a direct consequence of [13, Proposition 2.1]. 2.: (2a \(\Rightarrow \) 2b). Let \((v_n)_{n \in {\mathbb {N}}}\) be sequence in \(\mathrm{ran }(\varSigma ^{-1} - L \varUpsilon L^*)\) such that \(v_n \rightarrow v.\) Therefore, for each \(n \in {\mathbb {N}},\) there exists \(u_n \in {\mathcal {G}}\) such that \(v_n=\varSigma ^{-1} u_n - L \varUpsilon L^* u_n\). Note that \({\varvec{V}}(\varUpsilon L^* u_n , u_n)=(0, v_n)\rightarrow (0, v)\). Since \(\mathrm{ran }\, {\varvec{V}}\) is closed, there exists some \((x,u) \in {\mathcal {H}}\times {\mathcal {G}}\) such that \({\varvec{V}}(x,u)=(0, v)\), i.e.,
Then \( v \in \mathrm{ran }(\varSigma ^{-1} - L \varUpsilon L^*)\), and therefore \(\mathrm{ran }(\varSigma ^{-1}- L \varUpsilon L^*)\) is closed.
(2b \(\Rightarrow \) 2a). Let \(\big ((y_n,v_n)\big )_{n \in {\mathbb {N}}}\) be a sequence in \(\mathrm{ran }\, {\varvec{V}}\) such that \((y_n,v_n) \rightarrow (y,v).\) Then, for every \(n \in {\mathbb {N}}\), there exists \((x_n,u_n)\) such that \((y_n,u_n)={\varvec{V}}(x_n,u_n)\), or equivalently,
By applying \(L \varUpsilon \) to the first equation in (6.1) and adding it to the second equation, by the continuity of \(\varUpsilon \) and L, we obtain
Hence, since \(\mathrm{ran }(\varSigma ^{-1} - L \varUpsilon L^*)\) is closed, there exists \(u \in {\mathcal {G}}\) such that \(L \varUpsilon y+ v =(\varSigma ^{-1} - L \varUpsilon L^*)u\). We deduce \({\varvec{V}}\left( \varUpsilon (L^*u+y),u \right) =(y,v)\), and therefore \(\mathrm{ran }\, {\varvec{V}}\) is closed.
(2a \(\Leftrightarrow \) 2c). Define \({\tilde{{\varvec{V}}}} : {\mathcal {G}}\oplus {\mathcal {H}}\rightarrow {\mathcal {G}}\oplus {\mathcal {H}}: (u,x) \mapsto (\varSigma ^{-1} u-Lx,\varUpsilon ^{-1} x-L^*u)\). By the equivalence 2a \(\Leftrightarrow \) 2b \(\mathrm{ran }{\tilde{{\varvec{V}}}}\) is closed if and only if \(\mathrm{ran }(\varUpsilon ^{-1} - L^* \varSigma L)\) is closed. Consider the isometric map \(\varvec{\varLambda } : {\mathcal {H}}\oplus {\mathcal {G}}\rightarrow {\mathcal {G}}\oplus {\mathcal {H}}: (x,u) \mapsto (u,x)\). Since \(\varvec{\varLambda }\circ {\varvec{V}}={\tilde{{\varvec{V}}}}\), \({\mathrm{ran }\, {\varvec{V}}}\) is closed if and only if \(\mathrm{ran }{\tilde{{\varvec{V}}}}\) is closed and the result follows. \(\square \)
Proposition 6.1
In the context of Problem 1.1, set \(L=\mathrm{Id}\), let \(\varUpsilon :{\mathcal {H}}\rightarrow {\mathcal {H}}\) be a strongly monotone self adjoint linear bounded operator, set \(\varLambda :{\mathcal {H}}\times {\mathcal {H}}\rightarrow {\mathcal {H}}:(x,u)\mapsto x-\varUpsilon u\), let \({\varvec{V}}\), \({\varvec{W}}\), and \(G_{\varUpsilon ,B,A}\) be the operators defined in (1.8), (3.2), and (3.15), respectively. Then, \(\varLambda (\mathrm{Fix}\,(P_{{\mathrm{ran }\, {\varvec{V}}}}\circ J_{{\varvec{W}}}))=\mathrm{Fix}\,G_{\varUpsilon ,B,A}\).
Proof
The inclusion \(\subset \) is proved in (3.23). Conversely, since \(\varLambda ^*:z\mapsto (z,-\varUpsilon z)\), we have \(\varLambda \circ \varLambda ^*=\mathrm{Id}+\varUpsilon ^2\) and [3, Proposition 3.30 & Example 3.29] yields \(P_{{\mathrm{ran }\, {\varvec{V}}}}=P_{\mathrm{ran }\varLambda ^*}=\varLambda ^*(\mathrm{Id}+\varUpsilon ^2)^{-1} \varLambda \). Therefore, if \({\hat{z}}\in \mathrm{Fix}\,G_{\varUpsilon ,B,A}\), by setting \(({\hat{x}},{\hat{u}}):=\varLambda ^*(\mathrm{Id}+\varUpsilon ^2)^{-1}{\hat{z}}\), we have \({\hat{z}}=\varLambda ({\hat{x}},{\hat{u}})\) and we deduce from (3.22) that
Consequently, \(({\hat{x}},{\hat{u}})\in \mathrm{Fix}\,(P_{{\mathrm{ran }\, {\varvec{V}}}}\circ J_{{\varvec{W}}})\) and \({\hat{z}}=\varLambda ({\hat{x}},{\hat{u}})\in \varLambda (\mathrm{Fix}\,(P_{{\mathrm{ran }\, {\varvec{V}}}}\circ J_{{\varvec{W}}}))\). \(\square \)
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Briceño-Arias, L., Roldán, F. Primal-dual splittings as fixed point iterations in the range of linear operators. J Glob Optim 85, 847–866 (2023). https://doi.org/10.1007/s10898-022-01237-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-022-01237-w
Keywords
- Convex optimization
- Douglas–Rachford splitting
- Krasnosel’skiĭ-Mann iterations
- Monotone operator theory
- Primal-dual algorithm
- Quasinonexpansive operators