Schedule

During the morning there will be lectures focusing on the main areas of ML and their application to NLP. These areas include but are not restricted to: Classification, Structured Prediction (sequences, trees, graphs), Deep Learning and Reinforcement Learning.

For each topic introduced in the morning there will be a practical session in the afternoon, where students will have the opportunity to test the concepts in practice. The practical sessions will consist in implementation exercises in Python of the methods learned during the morning, testing them on real examples.

At the end of the afternoon there will be special talks of concrete applications of the these techniques being currently used in production.

All Morning Sessions and Evening Talks will be held at the Centro de Congressos. All Afternoon Labs will be held at Pavilhão de Informática (see FAQ).

The tentative schedule is shown below.

WEDNESDAY, JULY 10TH

18:00 – 20:00 Early Bird Registration and Meet-up at Centro de Congressos
NOTE: If you can come to the early registration, do so.

THURSDAY, JULY 11TH

08:00 – 09:00 Registration at Centro de Congressos

09:00 – 10:30 Morning Session 1

BASIC TUTORIALS ON PROBABILITY THEORY AND LINEAR ALGEBRA (MARIO FIGUEIREDO)

10:30 – 11:00 Coffee Break
11:00 – 12:30 Morning Session 2

INTRODUCTION TO PYTHON (RAMON ASTUDILLO)

12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon session: Introduction to the Labs and Python

17:00 – Welcome reception

FRIDAY, JULY 12TH

09:00 – 12:30 Morning Lecture (30 min coffee break at 10:30 with Poster Session)

LECTURE 1: INTRODUCTION TO MACHINE LEARNING: LINEAR LEARNERS (ANDRE MARTINS)

  • Feature representations and linear decision boundaries
  • Naive Bayes, logistic regression, perceptron, SVMs
  • Online learning
  • Linear learning of non-linear models

12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Linear Classifiers
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk

PRACTICAL TALK: TOWARDS END-TO-END SPEECH RECOGNITION (TARA SAINATH)

SATURDAY, JULY 13TH

09:00 – 12:30 Morning Lecture (30 min coffee break at 10:30 with Poster Session)

LECTURE 2: INTRODUCTION TO NEURAL NETWORKS (BHIKSHA RAJ)

  • Multi-layer perceptrons (Feed Forward networks)
  • Training with Backpropagation
  • Connectionist Computational Models
  • Universal Boolean Machines

12:30 – 14:00 Lunch

14:00 – 17:00 Afternoon Labs: Introduction to Deep Learning and Pytorch
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk

PRACTICAL TALK: NATURAL LANGUAGE REPRESENTATIONS AND CHALLENGES (SLAV PETROV)

20:00 Summer School Banquet at Casa do Alentejo

SUNDAY, JULY 14TH

Free Day!

MONDAY, JULY 15TH

09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)

LECTURE 3: SEQUENCE MODELS (NOAH SMITH)

  • Markov models and hidden Markov models (HMMs)
  • Dynamic programming algorithms (Viterbi and sum-product)
  • Parameter learning (MLE and Baum-Welch/EM)
  • Finite state machines and finite state transducers

12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Sequence Models
17:00 – 20:00 LxMLS Demo Day

TUESDAY, JULY 16TH

09:00 – 12:30 Morning Lecture (30 min coffee break at 10:30 with Poster Session)

LECTURE 4: LEARNING STRUCTURED PREDICTORS (XAVIER CARRERAS)

  • From HMMs to CRFs: discriminative learning and features
  • Structured perceptron, structured SVMs and max-margin Markov networks
  • Training and optimization
  • Iterative scaling, L-BFGS, perceptron, MIRA, stochastic and batch gradient descent

12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Structured Predictors
17:30 – 18:30 Evening Talk

PRACTICAL TALK: INFORMATION EXTRACTION FROM TEXT: FROM OPINIONS TO ARGUMENTS TO PERSUASION (CLAIRE CARDIE)

WEDNESDAY, JULY 17th

09:00 – 12:30 Morning Lecture (30 min coffee break at 10:30 with Poster Session)

LECTURE 5: MODELING SEQUENTIAL DATA WITH RECURRENT NETWORKS (CHRIS DYER)

  • Recurrent Neural Networks
  • Learning challenges and solutions
  • Conditional sequence models
  • Learning with attention

12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Sequence models in deep learning
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk

PRACTICAL TALK: A GENERALIZED FRAMEWORK OF SEQUENCE GENERATION (KYUNGHYUN CHO)

THURSDAY, JULY 18th

09:00 – 12:30 Morning Lecture (with 30 min coffee break at 10:30)

LECTURE 6: REINFORCEMENT LEARNING (STEFAN RIEZLER)

12:30 – 14:00 Lunch
14:00 – 17:00 Afternoon Labs: Reinforcement Learning
17:00 – 17:30 Coffee Break
17:30 – 18:30 Evening Talk

PRACTICAL TALK: WHAT DO OUR MODELS LEARN? TRYING TO UNDERSTAND NEURAL MODELS (YOAV GOLDBERG)

18:30 – 19:00 Closing Remarks