During the morning there will be lectures focusing on the main areas of ML and their application to NLP. These areas include but are not restricted to: Classification, Structured Prediction (sequences, trees, graphs), Parsing and Deep Learning.
For each topic introduced in the morning there will be a practical session in the afternoon, where students will have the opportunity to test the concepts in practice. The practical sessions will consist in implementation exercises in Python of the methods learned during the morning, testing them on real examples.
At the end of the afternoon there will be special talks of concrete applications of the these techniques being currently used in production.
All Morning Sessions and Evening Talks will be held at the Centro de Congressos. All Afternoon Labs will be held at Pavilhão de Informática. Here you can find the detailed location.
The tentative schedule is shown below.
WEDNESDAY, JULY 20TH
18:00 – 20:00 Early Bird Registration and Meet-up at Centro de Congressos
NOTE: If you can come to the early registration, do so.
THURSDAY, JULY 21TH
08:00 – 09:00 Registration at Centro de Congressos
09:00 – 10:30 Morning Session 1
BASIC TUTORIALS ON PROBABILITY THEORY AND LINEAR ALGEBRA (MARIO FIGUEIREDO)
10:30 – 11:00 Coffee Break
11:00 – 12:30 Morning Session 2
INTRODUCTION TO PYTHON (LUIS PEDRO COELHO)
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon session: Introduction to the Labs and Python
17:00 Welcome reception
FRIDAY, JULY 22TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)
LECTURE 1: INTRODUCTION TO MACHINE LEARNING: LINEAR LEARNERS (STEFAN RIEZLER)
- Feature representations and linear decision boundaries
- Naive Bayes, logistic regression, perceptron, SVMs
- Online learning
- Linear learning of non-linear models
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Classification
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: STRUCTURED PREDICTION IN NATURAL LANGUAGE PROCESSING WITH IMITATION LEARNING (ANDREAS VLACHOS)
SATURDAY, JULY 23TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)
LECTURE 2: SEQUENCE MODELS (NOAH SMITH)
- Markov models and hidden Markov models (HMMs)
- Dynamic programming algorithms (Viterbi and sum-product)
- Parameter learning (MLE and Baum-Welch/EM)
- Finite state machines and finite state transducers
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Sequence Models
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: MACHINE TRANSLATION AS SEQUENCE MODELLING (PHILIPP KOEHN)
20:00 Summer School Banquet, at Casa do Alentejo
SUNDAY, JULY 24TH
Free Day!
MONDAY, JULY 25TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)
LECTURE 3: LEARNING STRUCTURED PREDICTORS (XAVIER CARRERAS)
- From HMMs to CRFs: discriminative learning and features
- Structured perceptron, structured SVMs and max-margin Markov networks
- Training and optimization
- Iterative scaling, L-BFGS, perceptron, MIRA, stochastic and batch gradient descent
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Structured Predictors
17:00 – 20:00 LxMLS Demo Day
TUESDAY, JULY 26TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)
LECTURE 4: SYNTAX AND PARSING (I, II) (SLAV PETROV)
- Context-free grammars (CFGs) and phrase-based parsing
- Dynamic programming and CKY algorithm
- Probabilistic CFGs, parent annotation and lexicalization
- Dependency parsing (projective and non-projective)
- Transition and graph-based parsers
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Parsing
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: TURBO PARSER REDUX: FROM DEPENDENCIES TO CONSTITUENTS (ANDRÉ MARTINS)
WEDNESDAY, JULY 27th
09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)
LECTURE 5: DEEP NEURAL NETWORKS ARE OUR FRIENDS (WANG LING)
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Introduction to Deep Learning and Theano
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: (ORIOL VINYALS)
THURSDAY, JULY 28th
09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)
LECTURE 6: MODELING SEQUENTIAL DATA WITH RECURRENT NETWORKS (CHRIS DYER)
12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Sequence models in deep learning
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk
PRACTICAL TALK: MEMORY NETWORKS FOR LANGUAGE UNDERSTANDING (pdf version) (ANTOINE BORDES)
18:30 – 19:00 Closing Remarks