During the morning there will be lectures focusing on the main areas of ML and their application to NLP. These areas include but are not restricted to: Classification, Structured Prediction (sequences, trees, graphs), Parsing, Semi-Supervised Learning, and their applications to practical language processing on the Web.

For each topic introduced in the morning there will be a practical session in the afternoon, where students will have the opportunity to test the concepts in practice. The practical sessions will consist in implementation exercises (using Python, Numpy, and Matplotlib) of the methods learned during the morning, testing them on real examples.

At the end of the afternoon there will be special talks of concrete applications of the these techniques being currently used in production.

All Morning Sessions and Evening Talks will be held at Complexo Interdisciplinar. All Afternoon Labs will be held at Pavilhão de Informática.

The final schedule is shown below.

Wednesday, July 20th

09:00 - 10:30    Morning Session 1 [SLIDES]

Basic tutorials on probability theory and linear algebra

10:30 - 11:00    Coffee Break

11:00 - 12:30    Morning Session 2 [SLIDES]

Introduction to Python (instructions on how to install Python in your machine)

12:30 - 14:00    Lunch

14:00 - 17:00    Afternoon session: Pratical implementation exercises [PDF]

17:30                Welcome reception

Thursday, July 21st

09:00 - 12:30    Morning Lecture (with 30 min coffee break) [SLIDES] [VIDEO]

LECTURE 1: INTRODUCTION TO MACHINE LEARNING (KOBY CRAMMER)

1. ‣Decision theory
   

2. ‣Classification
   

3. ‣Generative and discriminative models
   

4. ‣Naive Bayes, logistic regression, support vector machines (SVMs)
   

5. ‣Online learning: perceptron and passive-aggressive algorithms
   

12:30 - 14:00    Lunch

14:00 - 17:00    Afternoon Labs [PDF]

17:00 - 17:30    Coffee Break

18:00 - 19:30    Evening Talk

PRACTICAL TALK: SOCIAL MEDIA (MILES OSBORNE)

Friday, July 22ND

09:00 - 12:30    Morning Lecture (with 30 min coffee break) [SLIDES] [VIDEO]

LECTURE 2: SEQUENCE MODELS (NOAH SMITH)

1. ‣Markov models and hidden Markov models (HMMs)
   

2. ‣Dynamic programming algorithms (Viterbi and sum-product)
   

3. ‣Parameter learning (MLE and Baum-Welch/EM)
   

4. ‣Finite state machines and finite state transducers
   

12:30 - 14:00    Lunch

14:00 - 17:00    Afternoon Labs [PDF]

17:00 - 17:30    Coffee Break

18:00 - 19:30    Evening Talk

PRACTICAL TALK: NETWORK INFERENCE FROM CO-OCCURENCES (MÁRIO FIGUEIREDO)

Saturday, July 23RD

09:00 - 12:30    Morning Lecture (with 30 min coffee break) [SLIDES] [VIDEO]

LECTURE 3: LEARNING STRUCTURED PREDICTORS (XAVIER CARRERAS)

1. ‣From HMMs to CRFs: discriminative learning and features
   

2. ‣Structured perceptron, structured SVMs and max-margin Markov networks
   

3. ‣Training and optimization
   

4. ‣Iterative scaling, L-BFGS, perceptron, MIRA, stochastic and batch gradient descent
   

12:30 - 14:00    Lunch

14:00 - 17:00    Afternoon Labs [PDF]

17:00 - 17:30    Coffee Break

18:00 - 19:30    Evening Talk

PRACTICAL TALK: LANGUAGE IN THE WILD: LEARNING FROM THE WEB TO UNDERSTAND THE WEB (SLAV PETROV)

20:30                Summer School Banquet (at Mercado da Ribeira)

Sunday, July 24TH

09:00 - 12:30    Morning Lecture (with 30 min coffee break) [SLIDES] [VIDEO]

LECTURE 4: SYNTAX AND PARSING (SLAV PETROV)

1. ‣Context-free grammars (CFGs) and phrase-based parsing
   

2. ‣Dynamic programming and CKY algorithm
   

3. ‣Probabilistic CFGs, parent annotation and lexicalization
   

4. ‣Dependency parsing (projective and non-projective)
   

5. ‣Transition and graph-based parsers
   

12:30 - 14:00    Lunch

14:00 - 17:00    Afternoon Labs [PDF]

17:00 - 17:30    Coffee Break

18:00 - 19:30    Evening Talk

PRACTICAL TALK: STATISTICAL MACHINE TRANSLATION (PHIL BLUNSOM)

Monday, July 25TH

09:00 - 12:30    Morning Lecture (with 30 min coffee break) [SLIDES] [VIDEO]

LECTURE 5: SEMI-SUPERVISED/UNSUPERVISED/TOPIC MODELING (JASON EISNER)

1. ‣Exponential family and conjugate priors
   

2. ‣The Bayesian perspective on finite state and context free models
   

3. ‣Missing data and unsupervised learning
   

4. ‣Topic models and nonparametric Bayes
   

5. ‣Approximate inference: MCMC, variational methods, contrastive divergence
   

12:30 - 14:00    Lunch

14:00 - 17:00    Afternoon Labs [PDF]

17:00 - 17:30    Coffee Break

18:00 - 19:30    Evening Talk

PRACTICAL TALK: RICH PRIOR KNOWLEDGE IN LEARNING FOR NLP (JOÃO GRAÇA)

19:30 - 20:00    Closing Remarks