Abstract: Many (supervised) learning problems lead to optimisation problems. Often, the quantities required for finding an optimal solution (e.g., the gradient or Hessian) cannot be computed exactly. In this talk I will give an overview of recent approaches that allow one to use such approximated quantities in a rigorous way.