Compositional vector-based semantics for Dutch

Compositional Models of Vector-based Semantics: From Theory to Tractable Implementation

Instructors: Gijs Wijnholds and Michael Moortgat

Teaching Assistant: Konstantinos Kogkalidis

Venue: ESSLLI 2022, NUI Galway, Ireland

Course Description

Vector-based compositional architectures combine a distributional view of word meanings with a modelling of the syntax-semantics interface as a structure-preserving map relating syntactic categories (types) and derivations to their counterparts in a corresponding meaning algebra. This design is theoretically attractive, but faces challenges when it comes to large-scale practical applications. First there is the curse of dimensionality resulting from the fact that semantic spaces directly reflect the complexity of the types of the syntactic front end. Secondly, modelling of the meaning algebra in terms of finite dimensional vector spaces and linear maps means that vital information encoded in syntactic derivations is lost in translation. The course compares and evaluates methods that are being proposed to face these challenges. Participants gain a thorough understanding of theoretical and practical issues involved, and acquire hands-on experience with a set of user-friendly tools and resources.

Suggested Readings

  • Richard Moot & Christian Retoré, 2012. The Logic of Categorial Grammars: A deductive account of natural language syntax and semantics. Available online
  • Sheldon Axler, 2015. Linear Algebra Done Right. Available online

Course Schedule

The course runs daily, between 11:00-12:30 UK time, in Week 1 of ESSLLI 2022. The per-day plan is listed below.

Day Topic Links
Day 1 What’s in a vector-based model of compositionality? Slides
We discuss the basic motivation behind compositional vector-based models.
Day 2 The curse of dimensionality Slides
We discuss the problems with a naive approach to composition between vectors and tensors.
Day 3 Refining the type lexicon Slides
We refine the type lexicon to model both function-argument structure and dependency relations, paving the way for practical applications.
Day 4 Parsing with Graph Neural Networks Slides
We use supertagging together with neural network machinery to solve the parsing problem.
Day 5 Evaluation: Experimenting with semantic tasks Slides
We discuss some experiments that have been done to evaluate the models.



For the enthousiasts that would like to have a hands-on experience, we attach an exercise sheet here.