Table of Contents
Different kind of positions are available. Feel free to contact me if you are interested in or if you have any questions. Moreover, feel free to also get in touch if you want to work with on nice other topics related to my research interests.
1 PhD: Machine Learning and Physics
We invite applications for a fullu-funded PhD position on the topic of "Deep Learning for physical systems modelling". This is a 3-year position funded by the ANR project SPEED and it will start next fall (as soon as possible). The whole project is a collaboration between the IJLRA, the LAMSADE and the new LISN Lab in Orsay where many other students work on the same subject. Frequent scientific discussions and meetings are planed. The position will start as soon as possible in fall 2021. Contact me if you are interested in.
1.1 Aims and scope
The interaction between machine learning and Physics has recently emerged as a new and important research area. Some illustrations are simulations of complex physical systems with machine learning models, or at the opposite, the introduction of numerical methods in machine learning.
At the interfaces of artificial and Physics, different tracks described below can be explored depending on the skills of the candidate.
1.1.1 Noisy, scarce and partial observation
In modern machine learning, the cornerstone is to let the model learn its own representation of the process from data observation. While, for many applications, data are readily available (computer vision, natural language processing, . . . ), some requirements are not met in the case of complex physical systems. Without loss of generality, let us consider the example of a turbulent flow field or the prediction of the sea surface temperature. The corresponding dataset is really small and scarce compared with usual machine learning applications. More importantly, the state cannot be fully observed in many situations and the data acquisition step often introduces noise.
The issues raised by noisy and scarce dataset are not new in the machine learning domain and there is, for instance, a long history of research in the field of generative models and how to represent high dimensional datasets in a compressed mathematical model. However, in the context of Physics, we can leverage some important properties like symmetries and invariances to address these challenges.
1.1.2 Training algorithm to enforce physical properties
In some cases, a mathematical model for the system at hand is available, for instance: dynamical systems such as the Lorentz (63 and 93) attractors, Kuramoto-Sivashinsky and Kardar-Parisi-Zhang. With these case studies, this step includes the important definitions of the physical properties we want to introduce in the machine learning models.
Two approaches can be considered:
- Physical regularization: the loss function optimized during the training process can be augmented with tailored regularization terms. As an example, optimal transport-based (OT) loss definitions are often more relevant for physical systems featuring significant structure. This will be made computationaly tractable with the convolutional Wasserstein flavor of OT, e.g., see this paper.
- Adversarial training: the second approach relies on the recent adversarial learning trend to guide the model during the training process toward solutions that exhibit the desired properties. Early efforts are reported in this paper where the solution and the test functions in the weak formulation of high-dimensional linear and nonlinear PDE problems are parameterized as a primal and adversarial networks respectively.
1.1.3 Neural Ordinary Differential Equations
The relationship between neural networks and differential equations has been studied in several recent works Lu et al. (2018); Chen et al. (2018). In particular, the very efficient neural architecture ResNet (or Residual Network) can be interpreted as discretized ordinary differential equations. This kind of architectures leads to a very large number of parameters. Hence, while the idea is really appealing in our context, architectures like ResNet are suitable for applications beyond our scope, where the data availability is not an issue. Pushing the discretization step towards its limit of zero, along with parameters tying, have given rise to a new family of models called Neural Ordinary Differential Equations (or Neural ODEs). In these recent papers and their extension, Dupont et al. (2019), the experimental setup mainly relies on conventional datasets used in image classification (MNIST or CIFAR10). Preliminary work on this new type of neural networks has demonstrated its parameter efficiency for supervised learning task which can be of a great importance in our case.
1.2 Application and contacts
Applications can be sent electronically and should include a cover letter, full CV, and eventually references. A first round of interviews will start in first week of June 2021 so please submit as soon as possible. Feel free to contact us if you have questions on the topic and the position.
Alexandre Allauzen: firstname.lastname@example.org Sergio Chibbaro: email@example.com
1.3 Some References
"When deep learning meets ergodic theory", M.A Bucci, O.S emeraro, S. Chibbaro, A. Allauzen, L. Mathelin, in https://hal.archives-ouvertes.fr/LIMSI/hal-03101431v1
"Control of chaotic systems by deep reinforcement learning", M.A Bucci, O.S emeraro, S. Chibbaro, A. Allauzen, et al. https://hal.archives-ouvertes.fr/LIMSI/hal-02406677v1
"Hamiltonian Neural Networks", Samuel Greydanus, Misko Dzamba, Jason Yosinski, NeurIPS 2019 proceedings. https://papers.nips.cc/paper/2019/hash/26cd8ecadce0d4efd6cc8a8725cbd1f8-Abstract.html
2 PhD: Stability and robustness of vision Transformers
This is a 3-year PhD position, funded by Foxstream, a software company (since 2004), specialized in real-time automated processing of video content analysis. The PhD thesis is a collaboration with Dauphine Université (the MILES team of the LAMSADE) with a join supervision (Quentin Barthélemy from Foxstream and Alexandre Allauzen from MILES). The PhD student will be located at Paris-Dauphine University in close relationships with Foxstream.
For a couple of decades, Deep Learning (DL) added a huge boost to the already rapidly developing field of computer vision. While for some kind of data and tasks, DL is the most successful approach, this is not the case for all applications. For instance, the analysis of video streams generated by thermal cameras is still a research challenge because of the long range perimeter, the depth of focus and the associated geometrical issues, along with the frequent calibration change. Therefore, the stability and robustness of DL models must be better characterized and improved.
Very recently, Transformer architectures have achieved state of the art performances in many domains: from natural language processing to computer vision. In this thesis we will explore the use of Tranformers for videos generated by thermal cameras and their properties.
From a theoritical and application perspectives, the goals are to explore the stability of such architectures, the robustness against adversarial examples, and what kind of invariances and symetries can be captured.
- Outstanding master's degree (or an equivalent university degree) in computer science or another related disciplines (as e.g. mathematics, information sciences, computer engineering, etc.).
- Proficiency in machine learning, computer vision, or signal processing.
- Fluency in spoken and written English is required.
Application: To apply, please email alexandre.allauzen [at] dauphine.psl.eu with:
- a curriculum vitae, with contact of 2 or more referees
- a cover letter
- a research outcome (e.g. master thesis and/or published papers) of the candidate
- a transcript of grades
Caron et al, Emerging Properties in Self-Supervised Vision Transformers, arXiv, 2021 https://arxiv.org/abs/2104.14294
Dosovitskiy et al, An image is worth 16x16 words Transformers for image recognition at scale, arXiv, 2020 https://arxiv.org/abs/2010.11929
Le, Vial, …, Allauzen et al, FlauBERT: Unsupervised Language Model Pre-training for French, LREC http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.302.pdf
3 Master internship 2021 (Closed for this spring and summer)
The following topics are proposed for funded internship positions. Most of them can also be extended by a funded PhD position.
- Prediction of genetic traits with deep neural networks: how to consider epistasis ?, in collaboration with Philippe Nghe of ESPCI
- Adversarial attacks in the light of stability of ResNets in collaboration with Laurent Meunier
- Neural ODE
- Stability and robustness of Deep Learning models to process video from thermal cameras in collaboration with foxstream
- Dynamic topic models: from text to physics