Schedule

During the morning there will be lectures focusing on the main areas of ML and their application to NLP. These areas include but are not restricted to: Classification, Structured Prediction (sequences, trees, graphs), Parsing, Information Retrieval, and their applications to practical language processing on the Web.

For each topic introduced in the morning there will be a practical session in the afternoon, where students will have the opportunity to test the concepts in practice. The practical sessions will consist in implementation exercises (using Python, Numpy, and Matplotlib) of the methods learned during the morning, testing them on real examples. A preliminary version of the lab guide is available here.

At the end of the afternoon there will be special talks of concrete applications of the these techniques being currently used in production.

All Morning Sessions and Evening Talks will be held at IST Congress Center, in the floor 02 of the Civil Engineering building. All Afternoon Labs will be held at Pavilhão de Informática.

The tentative schedule is shown below.

WEDNESDAY, JULY 24TH

09:00 – 10:30 Morning Session 1

BASIC TUTORIALS ON PROBABILITY THEORY AND LINEAR ALGEBRA (MARIO FIGUEIREDO)

10:30 – 11:00 Coffee Break
11:00 – 12:30 Morning Session 2

INTRODUCTION TO PYTHON (LUIS PEDRO COELHO)

[instructions on how to install Python in your machine]

12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon session: Pratical implementation exercises

16:30 Welcome reception

THURSDAY, JULY 25TH

09:00 – 12:30 Morning Lecture (with 30 min coffee break)

LECTURE 1: INTRODUCTION TO MACHINE LEARNING (KOBY CRAMMER)
[VIDEO 2011]

  • Decision theory
  • Classification
  • Generative and discriminative models
  • Naive Bayes, logistic regression, support vector machines (SVMs)
  • Online learning: perceptron and passive-aggressive algorithms

12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
16:30 – 17:00 Coffee Break
17:00 – 18:00 Evening Talk

PRACTICAL TALK: MODELING MORPHOLOGICALLY RICH LANGUAGES (CHRIS DYER)
[VIDEO]

FRIDAY, JULY 26TH

09:00 – 12:30 Morning Lecture (with 30 min coffee break)

LECTURE 2: SEQUENCE MODELS (NOAH SMITH)
[VIDEO 2011]

  • Markov models and hidden Markov models (HMMs)
  • Dynamic programming algorithms (Viterbi and sum-product)
  • Parameter learning (MLE and Baum-Welch/EM)
  • Finite state machines and finite state transducers

12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
17:00 – 20:00 Demo Day

LXMLS DEMO DAY
 

SATURDAY, JULY 27TH

09:00 – 12:30 Morning Lecture (with 30 min coffee break)

LECTURE 3: LEARNING STRUCTURED PREDICTORS (XAVIER CARRERAS)
[VIDEO 2011]

  • From HMMs to CRFs: discriminative learning and features
  • Structured perceptron, structured SVMs and max-margin Markov networks
  • Training and optimization
  • Iterative scaling, L-BFGS, perceptron, MIRA, stochastic and batch gradient descent

12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
16:30 – 17:00 Coffee Break
17:00 – 18:00 Evening Talk

PRACTICAL TALK: RELATION EXTRACTION WITH MATRIX FACTORIZATION (SEBASTIAN RIEDEL)
[VIDEO]

20:00 Summer School Banquet:

Restaurante Casa do Alentejo
Rua das Portas Santo Antão 58 1150 Lisbon
phone: (+351) 213 405 140
(Location on Google Maps)

SUNDAY, JULY 28TH

09:00 – 17:00 Free Day

MONDAY, JULY 29TH

09:00 – 12:30 Morning Lecture (with 30 min coffee break)

LECTURE 4: SYNTAX AND PARSING (SLAV PETROV)
[VIDEO 2011]

  • Context-free grammars (CFGs) and phrase-based parsing
  • Dynamic programming and CKY algorithm
  • Probabilistic CFGs, parent annotation and lexicalization
  • Dependency parsing (projective and non-projective)
  • Transition and graph-based parsers

12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
16:30 – 17:00 Coffee Break
17:00 – 18:00 Evening Talk

PRACTICAL TALK: UNDERSTANDING ALL THE WORLD’S LANGUAGES (SLAV PETROV)
[VIDEO]

TUESDAY, JULY 30TH

09:00 – 12:30 Morning Lecture (with 30 min coffee break)

LECTURE 5: LEARNING WITH BIG DATA I (CHRIS DYER)
[VIDEO (part 1)][VIDEO (part 2)]

12:30 – 13:30 Lunch
13:30 – 16:30 Afternoon Labs
16:30 – 17:00 Coffee Break
17:00 – 18:00 Evening Talk

PRACTICAL TALK: LANGUAGE TRANSLATION AND CODE-BREAKING (KEVIN KNIGHT)
[VIDEO]

WEDNESDAY, JULY 31TH

09:00 – 12:30 Morning Lecture (with 30 min coffee break)

LECTURE 6: LEARNING WITH BIG DATA II (MILES OSBORNE)
[VIDEO (part 1)][VIDEO (part 2)]

12:30 – 13:30 Lunch
13:30 – 16:00 Afternoon Labs
16:00 – 16:30 Coffee Break
16:30 – 17:30 Evening Talk

PRACTICAL TALK: MULTI-TASK LEARNING FROM LARGE-SCALE HIGH-DIMENSIONAL DATA (STEFAN RIEZLER)
[VIDEO]

17:30 – 18:00 Closing Remarks