Open Research Positions

Different kind of positions are available. Feel free to contact me if you are interested in or if you have any questions. Moreover, feel free to also get in touch if you want to work with on nice other topics related to my research interests.

1 Research engineer (3 month open-position)

We invite applications for a Research engineer position (3 months). This position is funded by the ANR project SPEED (started in fall 2021). The whole project is a collaboration between the LAMSADE and the new LISN Lab in Orsay where many researchers and PhD students work on the same subject. The position will start as soon as possible in January 2022. Contact me if you are interested in.

1.1 Aims and scope of the project

The interaction between machine learning and Physics has recently emerged as a new and important research area. Some illustrations are simulations of complex physical systems with machine learning models, or at the opposite, the introduction of numerical methods in machine learning.

At the interfaces of artificial and Physics, different tracks described below can be explored depending on the skills of the candidate.

1.2 Stability and robustness of deep learning architectures in light of dynamical systems

1.2.1 Context

Within the LAMSADE, we have started to work on the study of stability and robustness of neural network architectures. Deep learning algorithms have shown their vulnerability to adversarial attacks: small and imperceptible perturbations of the inputs that maliciously fools the prediction. Since this discovery, building adversarial attacks and defenses against them have been an active and hot research topic. However, the understanding of this phenomenon remains in the early stages and their is still a gap to come up with to better understand adversarial attacks.

Recent papers have explored the interaction between machine learning and numerical methods, opening new perspectives for many questions. For instance, this paper proposes an interpretation of deep learning as a parameter estimation problem of nonlinear dynamical systems. This formulation allows us to analyze the stability and well-posedness of deep neural network architectures like ResNet.

1.2.2 Goals

The goal of this 3 months is to develop a new kind of architecture, based on either the recurrent models or the transformers. The goal is to design a more robust and stable architecture. The development relies on pytorch and the experiments will use standard datasets (i.e NLP or image classification) and datasets from Physics.

1.3 Application

Requirements:

  • Outstanding master's degree (or an equivalent university degree) in computer science or another related disciplines (as e.g. mathematics, information sciences, computer engineering, etc.).
  • Proficiency in machine learning, computer vision, or signal processing.
  • Fluency in spoken and written English is required.
  • Strong background in deep-learning with pytorch.

Application: To apply, please email alexandre.allauzen [at] dauphine.psl.eu with:

  • a curriculum vitae
  • a cover letter
  • a research outcome (e.g. master thesis and/or published papers) of the candidate
  • a transcript of grades

2 PhD: Machine Learning and Physics (CLOSED)

We invite applications for a fully-funded PhD position on the topic of "Deep Learning for physical systems modelling". This is a 3-year position funded by the ANR project SPEED and it will start next fall (as soon as possible). The whole project is a collaboration between the IJLRA, the LAMSADE and the new LISN Lab in Orsay where many other students work on the same subject. Frequent scientific discussions and meetings are planed. The position will start as soon as possible in fall 2021. Contact me if you are interested in.

2.1 Aims and scope

The interaction between machine learning and Physics has recently emerged as a new and important research area. Some illustrations are simulations of complex physical systems with machine learning models, or at the opposite, the introduction of numerical methods in machine learning.

At the interfaces of artificial and Physics, different tracks described below can be explored depending on the skills of the candidate.

2.1.1 Noisy, scarce and partial observation

In modern machine learning, the cornerstone is to let the model learn its own representation of the process from data observation. While, for many applications, data are readily available (computer vision, natural language processing, . . . ), some requirements are not met in the case of complex physical systems. Without loss of generality, let us consider the example of a turbulent flow field or the prediction of the sea surface temperature. The corresponding dataset is really small and scarce compared with usual machine learning applications. More importantly, the state cannot be fully observed in many situations and the data acquisition step often introduces noise.

The issues raised by noisy and scarce dataset are not new in the machine learning domain and there is, for instance, a long history of research in the field of generative models and how to represent high dimensional datasets in a compressed mathematical model. However, in the context of Physics, we can leverage some important properties like symmetries and invariances to address these challenges.

2.1.2 Training algorithm to enforce physical properties

In some cases, a mathematical model for the system at hand is available, for instance: dynamical systems such as the Lorentz (63 and 93) attractors, Kuramoto-Sivashinsky and Kardar-Parisi-Zhang. With these case studies, this step includes the important definitions of the physical properties we want to introduce in the machine learning models.

Two approaches can be considered:

  • Physical regularization: the loss function optimized during the training process can be augmented with tailored regularization terms. As an example, optimal transport-based (OT) loss definitions are often more relevant for physical systems featuring significant structure. This will be made computationaly tractable with the convolutional Wasserstein flavor of OT, e.g., see this paper.
  • Adversarial training: the second approach relies on the recent adversarial learning trend to guide the model during the training process toward solutions that exhibit the desired properties. Early efforts are reported in this paper where the solution and the test functions in the weak formulation of high-dimensional linear and nonlinear PDE problems are parameterized as a primal and adversarial networks respectively.

2.1.3 Neural Ordinary Differential Equations

The relationship between neural networks and differential equations has been studied in several recent works Lu et al. (2018); Chen et al. (2018). In particular, the very efficient neural architecture ResNet (or Residual Network) can be interpreted as discretized ordinary differential equations. This kind of architectures leads to a very large number of parameters. Hence, while the idea is really appealing in our context, architectures like ResNet are suitable for applications beyond our scope, where the data availability is not an issue. Pushing the discretization step towards its limit of zero, along with parameters tying, have given rise to a new family of models called Neural Ordinary Differential Equations (or Neural ODEs). In these recent papers and their extension, Dupont et al. (2019), the experimental setup mainly relies on conventional datasets used in image classification (MNIST or CIFAR10). Preliminary work on this new type of neural networks has demonstrated its parameter efficiency for supervised learning task which can be of a great importance in our case.

2.2 Application and contacts

Applications can be sent electronically and should include a cover letter, full CV, and eventually references. A first round of interviews will start in first week of June 2021 so please submit as soon as possible. Feel free to contact us if you have questions on the topic and the position.

Alexandre Allauzen: alexandre.allauzen@dauphine.psl.eu Sergio Chibbaro: sergio.chibbaro@sorbonne-universite.fr

2.3 Some References

"When deep learning meets ergodic theory", M.A Bucci, O.S emeraro, S. Chibbaro, A. Allauzen, L. Mathelin, in https://hal.archives-ouvertes.fr/LIMSI/hal-03101431v1

"Control of chaotic systems by deep reinforcement learning", M.A Bucci, O.S emeraro, S. Chibbaro, A. Allauzen, et al. https://hal.archives-ouvertes.fr/LIMSI/hal-02406677v1

"Hamiltonian Neural Networks", Samuel Greydanus, Misko Dzamba, Jason Yosinski, NeurIPS 2019 proceedings. https://papers.nips.cc/paper/2019/hash/26cd8ecadce0d4efd6cc8a8725cbd1f8-Abstract.html

3 PhD: Stability and robustness of vision Transformers (CLOSED)

This is a 3-year PhD position, funded by Foxstream, a software company (since 2004), specialized in real-time automated processing of video content analysis. The PhD thesis is a collaboration with Dauphine Université (the MILES team of the LAMSADE) with a join supervision (Quentin Barthélemy from Foxstream and Alexandre Allauzen from MILES). The PhD student will be located at Paris-Dauphine University in close relationships with Foxstream.

For a couple of decades, Deep Learning (DL) added a huge boost to the already rapidly developing field of computer vision. While for some kind of data and tasks, DL is the most successful approach, this is not the case for all applications. For instance, the analysis of video streams generated by thermal cameras is still a research challenge because of the long range perimeter, the depth of focus and the associated geometrical issues, along with the frequent calibration change. Therefore, the stability and robustness of DL models must be better characterized and improved.

Very recently, Transformer architectures have achieved state of the art performances in many domains: from natural language processing to computer vision. In this thesis we will explore the use of Tranformers for videos generated by thermal cameras and their properties.

From a theoritical and application perspectives, the goals are to explore the stability of such architectures, the robustness against adversarial examples, and what kind of invariances and symetries can be captured.

Requirements:

  • Outstanding master's degree (or an equivalent university degree) in computer science or another related disciplines (as e.g. mathematics, information sciences, computer engineering, etc.).
  • Proficiency in machine learning, computer vision, or signal processing.
  • Fluency in spoken and written English is required.

Application: To apply, please email alexandre.allauzen [at] dauphine.psl.eu with:

  • a curriculum vitae, with contact of 2 or more referees
  • a cover letter
  • a research outcome (e.g. master thesis and/or published papers) of the candidate
  • a transcript of grades

References:

Caron et al, Emerging Properties in Self-Supervised Vision Transformers, arXiv, 2021 https://arxiv.org/abs/2104.14294

Dosovitskiy et al, An image is worth 16x16 words Transformers for image recognition at scale, arXiv, 2020 https://arxiv.org/abs/2010.11929

Le, Vial, …, Allauzen et al, FlauBERT: Unsupervised Language Model Pre-training for French, LREC http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.302.pdf

4 Master internship 2021 (Closed for this spring and summer)

The following topics are proposed for funded internship positions. Most of them can also be extended by a funded PhD position.