During the morning there will be lectures focusing on the main areas of ML and their application to NLP. These areas include but are not restricted to: Classification, Structured Prediction (sequences, trees, graphs), Parsing, Information Retrieval, and their applications to practical language processing on the Web.
For each topic introduced in the morning there will be a practical session in the afternoon, where students will have the opportunity to test the concepts in practice. The practical sessions will consist in implementation exercises (using Python, Numpy, and Matplotlib) of the methods learned during the morning, testing them on real examples. A preliminary version of the lab guide is available here.
At the end of the afternoon there will be special talks of concrete applications of the these techniques being currently used in production.
All Morning Sessions and Evening Talks will be held at IST Congress Center, in the floor 02 of the Civil Engineering building. All Afternoon Labs will be held at Pavilhão de Informática.
The tentative schedule is shown below.
WEDNESDAY, JULY 24TH
09:00 – 10:30 Morning Session 1
BASIC TUTORIALS ON PROBABILITY THEORY AND LINEAR ALGEBRA (MARIO FIGUEIREDO)
10:30 – 11:00 Coffee Break
11:00 – 12:30 Morning Session 2
INTRODUCTION TO PYTHON (LUIS PEDRO COELHO)
[instructions on how to install Python in your machine]
12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon session: Pratical implementation exercises
16:30 Welcome reception
THURSDAY, JULY 25TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break)
LECTURE 1: INTRODUCTION TO MACHINE LEARNING (KOBY CRAMMER)
[VIDEO 2011]
- Decision theory
- Classification
- Generative and discriminative models
- Naive Bayes, logistic regression, support vector machines (SVMs)
- Online learning: perceptron and passive-aggressive algorithms
12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
16:30 – 17:00 Coffee Break
17:00 – 18:00 Evening Talk
PRACTICAL TALK: MODELING MORPHOLOGICALLY RICH LANGUAGES (CHRIS DYER)
[VIDEO]
FRIDAY, JULY 26TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break)
LECTURE 2: SEQUENCE MODELS (NOAH SMITH)
[VIDEO 2011]
- Markov models and hidden Markov models (HMMs)
- Dynamic programming algorithms (Viterbi and sum-product)
- Parameter learning (MLE and Baum-Welch/EM)
- Finite state machines and finite state transducers
12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
17:00 – 20:00 Demo Day
SATURDAY, JULY 27TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break)
LECTURE 3: LEARNING STRUCTURED PREDICTORS (XAVIER CARRERAS)
[VIDEO 2011]
- From HMMs to CRFs: discriminative learning and features
- Structured perceptron, structured SVMs and max-margin Markov networks
- Training and optimization
- Iterative scaling, L-BFGS, perceptron, MIRA, stochastic and batch gradient descent
12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
16:30 – 17:00 Coffee Break
17:00 – 18:00 Evening Talk
PRACTICAL TALK: RELATION EXTRACTION WITH MATRIX FACTORIZATION (SEBASTIAN RIEDEL)
[VIDEO]
20:00 Summer School Banquet:
Restaurante Casa do Alentejo
Rua das Portas Santo Antão 58 1150 Lisbon
phone: (+351) 213 405 140
(Location on Google Maps)
SUNDAY, JULY 28TH
09:00 – 17:00 Free Day
MONDAY, JULY 29TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break)
LECTURE 4: SYNTAX AND PARSING (SLAV PETROV)
[VIDEO 2011]
- Context-free grammars (CFGs) and phrase-based parsing
- Dynamic programming and CKY algorithm
- Probabilistic CFGs, parent annotation and lexicalization
- Dependency parsing (projective and non-projective)
- Transition and graph-based parsers
12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
16:30 – 17:00 Coffee Break
17:00 – 18:00 Evening Talk
PRACTICAL TALK: UNDERSTANDING ALL THE WORLD’S LANGUAGES (SLAV PETROV)
[VIDEO]
TUESDAY, JULY 30TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break)
LECTURE 5: LEARNING WITH BIG DATA I (CHRIS DYER)
[VIDEO (part 1)][VIDEO (part 2)]
12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
16:30 – 17:00 Coffee Break
17:00 – 18:00 Evening Talk
PRACTICAL TALK: LANGUAGE TRANSLATION AND CODE-BREAKING (KEVIN KNIGHT)
[VIDEO]
WEDNESDAY, JULY 31TH
09:00 – 12:30 Morning Lecture (with 30 min coffee break)
LECTURE 6: LEARNING WITH BIG DATA II (MILES OSBORNE)
[VIDEO (part 1)][VIDEO (part 2)]
12:30 – 13:30 Lunch
13:30 – 16:00 Afternoon Labs
16:00 – 16:30 Coffee Break
16:30 – 17:30 Evening Talk
PRACTICAL TALK: MULTI-TASK LEARNING FROM LARGE-SCALE HIGH-DIMENSIONAL DATA (STEFAN RIEZLER)
[VIDEO]
17:30 – 18:00 Closing Remarks