Course syllabus for Structured machine learning

Knowledge and understanding

• Define core concepts such as geometric priors, equivariance, and probability flow generative models.
• Identify key structural properties (e.g., symmetries, invariances) in data and their domains.
• Explain how geometric deep learning (GDL) architectures (e.g., graph neural networks [GNN]) encode domain-specific structure.
• Summarize the mathematical principles underlying probability flow generative models, such as measure transport and change of variables.
• Interpret the role of inductive biases (e.g., locality, symmetry) in model design for scientific applications.

Skills and abilities

• Implement geometric deep learning architectures (e.g., GNNs) using frameworks like PyTorch Geometric.
• Implement and simulate probability flow generative models (e.g., diffusion models or continuous normalizing flows) subject to certain domain symmetries.
• Compare the trade-offs between equivariant architectures (e.g., invariant vs. equivariant layers) in terms of expressivity, computational cost, and sample complexity.

Judgment and approach

• Judge recent scientific reports on machine learning research projects using structure in data or data generating process.
• Propose a small-scale research project that builds-on or applies upon themes covered in the course.
• Appraise small-scale research projects within structured machine learning using structure in data or data generating process.

This course will equip students with the theoretical foundations and practical tools to design machine learning systems that exploit structural patterns inherent in data domains, datasets, and generative processes. The course introduces geometric deep learning and probability flow generative models (e.g., continuous normalizing flows and diffusion models), and through practical exercises students will learn to build models that respect domain-specific constraints, such as invariance and equivariance to transformations or conservation laws.

The course bridges rigorous mathematical principles—such as geometric priors, stochastic differential equations, and symmetry-aware architectures—with hands-on implementation for real-world challenges. A key emphasis is placed on applications in natural sciences (e.g., molecular modeling), where structured data and generative processes are central to solving open problems.

The course will integrates theoretical lectures with assignments. The course will cover the following broad themes:

Geometric Deep Learning (GDL):
We will show how many modern Deep learning architectures, including Transformers, convolutional neural networks, and graph neural networks, emerge naturally from the GDL framework.
Key concepts:
- Learning in high-dimensional spaces, sample complexity, geometric priors, scale separation, symmetry, invariance/equivariance, deformation stability.

Data Generating processess and Probability flow generative models:
We discuss specific considerations about the data generation and acquisition/collection. We will give a brief introduction to generative modeling broadly and focus more specifically on diffusion models and continuous normalizing flows.

Key concepts:
- Direct/indirect observation, independent/correlated data, averaging/aliasing, density estimation, latent space, measure transport, push-forward, diffeomorphisms

Applications:

The course will focus on molecular applications as they provide a natural platform to illustrate all the concepts covered in the course.  Three projects will cover supervised  (classification, regression) and unsupervised learning (density estimation). The course ends with an essay assignment where a small-scale research project is proposed based on themes covered in the course.

This is not a model-zoo course: the course focuses on theoretical and conceptual understanding and how it relates practical implications. The course makes extensive use of high-dimensional calculus and linear algebra. The course also includes a crash-course in abstract algebra (group theory and group representation theory)