Theoretical and numerical comparison of first-order algorithms for cocoercive equations and smooth convex optimization - ENS de Lyon - École normale supérieure de Lyon Accéder directement au contenu
Article Dans Une Revue Signal Processing Année : 2023

Theoretical and numerical comparison of first-order algorithms for cocoercive equations and smooth convex optimization

Luis M Briceño-Arias
  • Fonction : Auteur
Nelly Pustelnik

Résumé

This paper provides a theoretical and numerical comparison of classical first order splitting methods for solving smooth convex optimization problems and cocoercive equations. In a theoretical point of view, we compare convergence rates of gradient descent, forward-backward, Peaceman-Rachford, and Douglas-Rachford algorithms for minimizing the sum of two smooth convex functions when one of them is strongly convex. A similar comparison is given in the more general cocoercive setting under the presence of strong monotonicity and we observe that the convergence rates in optimization are strictly better than the corresponding rates for cocoercive equations for some algorithms. We obtain improved rates with respect to the literature in several instances exploiting the structure of our problems. In a numerical point of view, we verify our theoretical results by implementing and comparing previous algorithms in well established signal and image inverse problems involving sparsity. We replace the widely used 1 norm by the Huber loss and we observe that fully proximal-based strategies have numerical and theoretical advantages with respect to methods using gradient steps. In particular, Peaceman-Rachford is the more performant algorithm in our examples.
Fichier principal
Vignette du fichier
2101.06152.pdf (2.63 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03405740 , version 1 (27-10-2021)

Identifiants

  • HAL Id : hal-03405740 , version 1

Citer

Luis M Briceño-Arias, Nelly Pustelnik. Theoretical and numerical comparison of first-order algorithms for cocoercive equations and smooth convex optimization. Signal Processing, 2023, 206 (C). ⟨hal-03405740⟩
20 Consultations
185 Téléchargements

Partager

Gmail Facebook X LinkedIn More