During the morning there will be lectures focusing on the main areas of ML and their application to NLP. These areas include but are not restricted to: Classification, Structured Prediction (sequences, trees, graphs), Deep Learning and Reinforcement Learning.
For each topic introduced in the morning there will be a practical session in the afternoon, where students will have the opportunity to test the concepts in practice. The practical sessions will consist in implementation exercises in Python of the methods learned during the morning, testing them on real examples.
At the end of the afternoon there will be special talks of concrete applications of the these techniques being currently used in production.
All Morning Sessions and Evening Talks will be held at the Centro de Congressos. All Afternoon Labs will be held at Pavilhão de Informática (see FAQ).
The tentative schedule is shown below.
WEDNESDAY, JULY 10TH
18:00 – 20:00 Early Bird Registration and Meet-up at Centro de Congressos
NOTE: If you can come to the early registration, do so.
THURSDAY, JULY 11TH
08:00 – 09:00 Registration at Centro de Congressos
09:00 – 10:30 Morning Session 1
BASIC TUTORIALS ON PROBABILITY THEORY AND LINEAR ALGEBRA (MARIO FIGUEIREDO)
10:30 – 11:00 Coffee Break
11:00 – 12:30 Morning Session 2
INTRODUCTION TO PYTHON (RAMON ASTUDILLO)
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon session: Introduction to the Labs and Python
17:00 – Welcome reception
FRIDAY, JULY 12TH
09:00 – 12:30 Morning Lecture (30 min coffee break at 10:30 with Poster Session)
LECTURE 1: INTRODUCTION TO MACHINE LEARNING: LINEAR LEARNERS (ANDRE MARTINS)
- Feature representations and linear decision boundaries
- Naive Bayes, logistic regression, perceptron, SVMs
- Online learning
- Linear learning of non-linear models
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Linear Classifiers
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: TOWARDS END-TO-END SPEECH RECOGNITION (TARA SAINATH)
SATURDAY, JULY 13TH
09:00 – 12:30 Morning Lecture (30 min coffee break at 10:30 with Poster Session)
LECTURE 2: INTRODUCTION TO NEURAL NETWORKS (BHIKSHA RAJ)
- Multi-layer perceptrons (Feed Forward networks)
- Training with Backpropagation
- Connectionist Computational Models
- Universal Boolean Machines
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Introduction to Deep Learning and Pytorch
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: NATURAL LANGUAGE REPRESENTATIONS AND CHALLENGES (SLAV PETROV)
20:00 Summer School Banquet at Casa do Alentejo
SUNDAY, JULY 14TH
Free Day!
MONDAY, JULY 15TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)
LECTURE 3: SEQUENCE MODELS (NOAH SMITH)
- Markov models and hidden Markov models (HMMs)
- Dynamic programming algorithms (Viterbi and sum-product)
- Parameter learning (MLE and Baum-Welch/EM)
- Finite state machines and finite state transducers
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Sequence Models
17:00 – 20:00 LxMLS Demo Day
TUESDAY, JULY 16TH
09:00 – 12:30 Morning Lecture (30 min coffee break at 10:30 with Poster Session)
LECTURE 4: LEARNING STRUCTURED PREDICTORS (XAVIER CARRERAS)
- From HMMs to CRFs: discriminative learning and features
- Structured perceptron, structured SVMs and max-margin Markov networks
- Training and optimization
- Iterative scaling, L-BFGS, perceptron, MIRA, stochastic and batch gradient descent
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Structured Predictors
17:30 – 18:30 Evening Talk
PRACTICAL TALK: INFORMATION EXTRACTION FROM TEXT: FROM OPINIONS TO ARGUMENTS TO PERSUASION (CLAIRE CARDIE)
WEDNESDAY, JULY 17th
09:00 – 12:30 Morning Lecture (30 min coffee break at 10:30 with Poster Session)
LECTURE 5: MODELING SEQUENTIAL DATA WITH RECURRENT NETWORKS (CHRIS DYER)
- Recurrent Neural Networks
- Learning challenges and solutions
- Conditional sequence models
- Learning with attention
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Sequence models in deep learning
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: A GENERALIZED FRAMEWORK OF SEQUENCE GENERATION (KYUNGHYUN CHO)
THURSDAY, JULY 18th
09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)
LECTURE 6: REINFORCEMENT LEARNING (STEFAN RIEZLER)
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Reinforcement Learning
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: WHAT DO OUR MODELS LEARN? TRYING TO UNDERSTAND NEURAL MODELS (YOAV GOLDBERG)
18:30 – 19:00 Closing Remarks