Table of Contents
Cette page est associée à la formation CNRS (voir le catalogue du CFE). Les cours auront lieu en français, les supports de cours sont rédigés (présentation) en anglais. La formation dure 3 jours, du 16 au 18 octobre. Les parties de travaux pratiques utiliseront pytorch.
Les ressources nécessaires à la formation sont disponible sur ce drive.
1 Recommandation avant la formation
Le plus simple pour avoir une installation de pytorch qui fonctionne est de passer par l'installation de python3, via anaconda 3.7 (et non la 2.7 qui utilise python 2.7).
Sur le drive, il y a un script de test, labs-pytorch-test.py, à lancer afin de s'assurer que votre configuration fonctionne correctement. Il existe aussi en version notebook.
Si vous avez besoin de vous mettre en jambe en python voir la section plus bas.
2 Expected road-map
2.1 Day 1, The basics
- Machine Learning basics (logistic regression, gradient descent)
- From logistic regression to neural networks, from linear to non linear models
- Labs : introduction to pytorch, tensors and training a logistic regression model
2.2 Day 2, Text classification with NNet
- Text classification, introducing word embeddings
- Convolution network
- Labs: Movie reviews classification
2.3 Day 3, Sequence processing
- Sequence models (ngram, recurrent and LSTM)
- Case study: Neural Machine Translation
- Recent trends: ELMO, BERT and co
- Labs: POS tagging with recurrent networks
3 Python refresher
PyTorch is python module. If you need a python refresher:
- https://learnpythonthehardway.org/python3/
- A good free book Dive in python that covers the basics and essentials.
- Otherwise (in French) the wiki site Apprendre à programmer en python.
- And of course don't forget to take a look to the standard documentation in French or in in English
- More practicing: https://github.com/gregmalcolm/python_koans
Numpy is one of the core library for scientific computing in Python. It provides a high-performance multidimensional array object, and tools for working with these arrays.
- A quick introduction to python and numpy and matplotlib: http://cs231n.github.io/python-numpy-tutorial/
- With Numpy exercises: https://github.com/rougier/numpy-100
- If you are already familiar with MATLAB: http://scipy.github.io/old-wiki/pages/NumPy_for_Matlab_Users
- And for R users : http://mathesaurus.sourceforge.net/r-numpy.html
Using python can be easier with ipython, look at this tutorial: http://cs231n.github.io/ipython-tutorial/. If you like more standard IDE : https://www.jetbrains.com/pycharm-edu/
4 NNet basics
To introduce neural networks, start with the videos from Hugo Larochelle. The roadmap is
- Capsules 1.1 to 1.6 : the artificial neuron and the feed-forward architecture (definition)
- Capsules 2.1 to 2.11: training basics