Instructors: Gijs Wijnholds and Michael Moortgat
Teaching Assistant: Konstantinos Kogkalidis
Venue: ESSLLI 2022, NUI Galway, Ireland
Course Description
Vector-based compositional architectures combine a distributional view of word meanings with a modelling of the syntax-semantics interface as a structure-preserving map relating syntactic categories (types) and derivations to their counterparts in a corresponding meaning algebra. This design is theoretically attractive, but faces challenges when it comes to large-scale practical applications. First there is the curse of dimensionality resulting from the fact that semantic spaces directly reflect the complexity of the types of the syntactic front end. Secondly, modelling of the meaning algebra in terms of finite dimensional vector spaces and linear maps means that vital information encoded in syntactic derivations is lost in translation. The course compares and evaluates methods that are being proposed to face these challenges. Participants gain a thorough understanding of theoretical and practical issues involved, and acquire hands-on experience with a set of user-friendly tools and resources.
Suggested Readings
- Richard Moot & Christian Retoré, 2012. The Logic of Categorial Grammars: A deductive account of natural language syntax and semantics. Available online
- Sheldon Axler, 2015. Linear Algebra Done Right. Available online
Course Schedule
The course runs daily, between 11:00-12:30 UK time, in Week 1 of ESSLLI 2022. The per-day plan is listed below.
| Day | Topic | Links |
|---|---|---|
| Day 1 | What’s in a vector-based model of compositionality? | Slides |
| We discuss the basic motivation behind compositional vector-based models. | ||
| Day 2 | The curse of dimensionality | Slides |
| We discuss the problems with a naive approach to composition between vectors and tensors. | ||
| Day 3 | Refining the type lexicon | Slides |
| We refine the type lexicon to model both function-argument structure and dependency relations, paving the way for practical applications. | ||
| Day 4 | Parsing with Graph Neural Networks | Slides |
| We use supertagging together with neural network machinery to solve the parsing problem. | ||
| Day 5 | Evaluation: Experimenting with semantic tasks | Slides |
| We discuss some experiments that have been done to evaluate the models. |
Exercises!
For the enthousiasts that would like to have a hands-on experience, we attach an exercise sheet here.