NNet starter

By Alexandre Allauzen

This page is dedicated to a 2-days course on Neural network with application to natural language processing (NNet for NLP). The drive with all the ressources is here.

1 Expected road-map

1.1 First day : 01/07, from 9h to 17h (with breaks)

Courses on:

  • From logistic regression to NNet
  • Machine learning basics
  • NNet for NLP

And lab sessions on:

  • Pytorch basics,
  • Logistic regression
  • Text classification

1.2 Second day : 02/07, from 9h to 17h (with breaks)

Courses on:

  • Sequence models
  • Neural Machine translation
  • ELMo and BERT

And lab sessions on:

  • Convolution for text classification
  • Sequence tagging

2 NNet basics

To introduce neural networks, start with the videos from Hugo Larochelle. The roadmap is

  • Capsules 1.1 to 1.6 : the artificial neuron and the feed-forward architecture (definition)
  • Capsules 2.1 to 2.11: training basics

3 Pytorch basics

Download this notebook and run it with jupyter. To get the tools you can install anaconda3 and then pytorch.

PyTorch is python module. If you need a python refresher:

Numpy is one of the core library for scientific computing in Python. It provides a high-performance multidimensional array object, and tools for working with these arrays.

Using python can be easier with ipython, look at this tutorial: http://cs231n.github.io/ipython-tutorial/. If you like more standard IDE : https://www.jetbrains.com/pycharm-edu/