Teaching

Optimization for large scale Machine Learning, M2 ENS 2022-2023

The goal of the class is to cover theoretical aspects and practical Python implementations of popular optimization algorithms in machine learning.

At the end of the class, the students will:

• be familiar with essential optimization notions

• be able to derive standard convergence proofs and convergence rates

• know which algorithm to chose given the problem to solve

• understand the implementation of these algorithms in Python

Notes for the class are here and exercises are here

Jupyter notebook for Lab 1 (first order) (right click, save link as)

Jupyter notebook for Lab 2 (2nd order) (right click, save link as)

Jupyter notebook for Lab 3 (stochastic methods) (right click, save link as)

Schedule: Thursday 14 h 30, Friday 10 h 15 (room B1).

Validation: theoretical homeworks, labs and paper presentation at the end of the class.

Ressources:

• Introductory lectures on convex optimization: a basic course, Y. Nesterov, 2004. A reference book in optimization, updated in 2018: Lectures on Convex Optimization.

• First order methods in optimization, A. Beck, 2019.

• Convex optimization: algorithms and complexity, S. Bubeck, 2015. A short monograph (100 pages) covering basic topics.

Convex optimization @Computation and Modelling summer school, WUST 2022

• intro slides

• exercises (updated every day)

• notebook 1 (Introduction to gradient descent on least squares)

• notebook 2 (Gradient descent on generic functions)

• notebook 3 (Non smooth optimization: the proximal gradient algorithm)

• notebook 4 (Alternatives to gradient descent)

Resources

My colleague Pierre Ablin and I have created a repository with some Python advice for our students: https://github.com/pierreablin/python-sessions.

Classes taught

Since 2019, I teach the Python for datascience class (42 h per year) in the X/HEC “Datascience for business” Master, using live coding inspired by the Software Carpentry workshops. I designed the course from scratch, collaborating with Joan Massich in 2019, Quentin Bertrand in 2020, and Hicham Janati in 2021.

Since 2020 I teach and handle practical sessions and data camps in Ecole Polytechnique's Executive education (70 h). Topics involved dimension reduction, clustering, scaling computations, visualization and datacamp. I designed 2 full python labs with Erwan Le Pennec on these topics.

From 2017 to 2019, my main teaching activity was the Optimization for datascience class of the Datascience Master, totalling 2*40 h including 4 h as lecturer. Amongst others, this involved refactoring of the practical sessions, tutoring of students during office hours, and partaking in the design of the final exam.

In 2016-2017, I was a TA at Télécom Paris, for

• Analysis and Probabilities (MDI 113/114, Bachelor, 10 h)

• Machine Learning and Data Mining (MDI 343, Executive Master, 20 h)

• Linear Models (SD 204, Master, 10 h)

• Practical Machine Learning (SD 207, Master, 10 h)

• Tools and applications for signals and images (SI 101, Bachelor, 6 h)