Prochains exposés

SPOT 60 – Lundi 1 Avril 2019 –  Amphi des thèses ENSEEIHT
14h – Hedy Attouch (Univ. Montpellier) – Fast optimization through inertial dynamics with Hessian driven damping
In a Hilbert space H, we first study the convergence properties, when t→+∞, of the trajectories t → x(t) ∈ H generated by the second-order evolution equation:
edoThen we analyze the fast minimization properties of the algorithms obtained as discrete versions in time. The function f: H → R to be minimized is assumed to be convex, twice continuously differentiable, with non-empty solution set. Extensions to structured optimization problems involving non-smooth functions will be examined next. The parameters α and β are supposed to be non-negative. The above system combines two types of damping with specific properties:
  1. The isotropic viscous damping with vanishing coefficient α/t is related to the Nesterov accelerated gradient method and to the FISTA method, as recently shown by Su-Boyd-Candès. We will review recent results regarding the fast convergence properties of these methods. They depend on the value of α relative to the critical value α =3. For α>3, any trajectory converges weakly to a minimizer of f, and f(x(t))- min(f) = o(1/t2). We will also show the natural link with the Ravine method of Gelfand-Tsetlin (1961).
  2. The Hessian driven damping takes into account the geometry of f. It is linked to the Newton and the Levenberg-Marquardt methods. It was introduced in the context of optimization by Alvarez-Attouch-Bolte-Redont (J. Math. Pures et Appl. 2002). Since then, several variants and associated algorithms have been considered. The above formulation has been studied by Attouch-Peypouquet-Redont (JDE 2016). The introduction of the Hessian induces multiple favorable effects on convergence properties. While maintaining the convergence rate of values of the Nesterov-type methods, it provides the fast convergence to zero of the gradients ∇f (x(t)).  For poorly conditioned minimization problems, it neutralizes wild transverse oscillations.

Surprisingly, the presence of the Hessian driven damping makes the system well-posed for a general proper lower-semicontinuous convex function f : H → R∪{+∞}. This is based on the crucial property that the inertial dynamic with Hessian driven damping can be written equivalently as a first-order system in time and space, allowing it to be extended simply by replacing the gradient with the sub-differential. The extension to the case of the sum of two potential functions « non-smooth + smooth » was exploited by Attouch-Maingé-Redont (DEA 2012) to model non-elastic shocks in mechanics and PDE’s. The study of the associated algorithms for structured convex optimization is an ongoing research topic. We will present a new proximal-based inertial algorithm with fast convergence properties combining Nesterov acceleration with Hessian damping. Based on the dynamical interpretation of the algorithms, we show the effect of temporal scaling on the convergence rates. The study of the dynamic system above has natural connection with control theory. Passing from open-loop control (as in Nesterov’s method) to closed-loop control is an active research topic. For Newton’s regularized method (without inertia), it was examined by Attouch-Redont-Svaiter (JOTA 2013). This study in the context above, as well as the restarting method, is an interesting direction of research.

15h – Emmanuel Soubiès (IRIT-CNRS Toulouse)

Comité local d’organisation

Cf un glossaire expliquant ces sigles et affiliations du système universitaire toulousain.

 Fréquence et structure

Une séance par mois environ, avec deux conférenciers chaque fois (deux conférences de type différent : une orientée fondements et une orientée applications, un conférencier de l’environnement toulousain et un conférencier extérieur, un conférencier du milieu académique et un conférencier du milieu de l’industrie et des services, etc.).

Horaire habituel : le lundi après-midi de 14h à 16h.


Sauf indication contraire, à l’amphithéâtre des (SPOT-) thèses à l’ENSEEIHT (N7), 2 rue Charles Camichel, 31000 Toulouse  (métro B, François Verdier). Attention, présentez-vous au poste de garde afin d’accéder au site.