home   mini-CV   news   research  publications   teaching   PhD students   student projects   talks/tutorials   code   memberships

Michael Biehl: research highlights

Theory and Modelling of Machine Learning    Algorithm Development    Interdisciplinary Applications

Theory and Modelling of Machine Learning

The Statistical Physics of of machine learning has re-gained considerable attention in the community.
Tools and numerical methods for the systematic analysis of training processes provide, for instance, typical
learning curves in clear-cut model situations.
Besides analysing equilibrium situations, the framework allows for the mathematical description of the
learning dynamics in multi-layered neural network architectures and other systems.
Our current efforts concern the modelling of learning in the presence of different forms of concept drift.
Furthermore we extend earlier studies of multi-layered neural networks can be extended to techniques
that have recently re-gained popularity in the context of so-called Deep Learning approaches.

Phase transitions in the learning curves of layered neural networks with
ReLU activation     Plateau states in online gradient descent of neural networks
under concept drift

  Most recent example publications:

Algorithm development

Deep theoretical analysis provides valuable insights, helps to optimize existing training schemes and
facilitates the development of entirely new approaches. Very early, the statistical physics approach has
led to, for instance, the formulation of the so-called AdaTron algorithm which has gained significant popularity
in the context of Support Vector Machines:
Optimal stability in the perceptron classifier aka linear kernel SVM          2D-visualization of multiclass Generalized Matrix Learning Vector Quantization, LVQ, GMLVQ

More recently, the use of adaptive distance measures and relevance learning in prototype-based classifiers
has been in the center of my interest. The method of Matrix Relevance Learning is based on the use of
adaptive metrics in Learning Vector Quantization (LVQ). It has been shown to improve classification
performance significantly in many cases and to provide, at the same time, deep insight into the problem
at hand. The basic formalism was communicated in:

Interdisciplinary Applications

The application of newly developed methods and improved algorithms constitutes an integral part of our
research. Many relevant theoretical questions and methodological challenges can only be identified
in the context of real world data and practical applications. To this end, it is vital to establish and
maintain intense collaborations with the corresponding domain experts.
In the following, only a few selected examples of on-going application oriented projects are highlighted in order
to illustrate the diversity of the considered problems.