Course syllabus for Information theory for complex systems

Course syllabus adopted 2025-02-20 by Head of Programme (or corresponding).

Overview

  • Swedish nameInformationsteori för komplexa system
  • CodeTIF150
  • Credits7.5 Credits
  • OwnerMPCAS
  • Education cycleSecond-cycle
  • Main field of studyBioengineering, Chemical Engineering, Engineering Physics
  • DepartmentSPACE, EARTH AND ENVIRONMENT
  • GradingTH - Pass with distinction (5), Pass with credit (4), Pass (3), Fail

Course round 1

  • Teaching language English
  • Application code 11116
  • Open for exchange studentsYes

Credit distribution

0107 Examination 7.5 c
Grading: TH
7.5 c

In programmes

Examiner

Eligibility

General entry requirements for Master's level (second cycle)
Applicants enrolled in a programme at Chalmers where the course is included in the study programme are exempted from fulfilling the requirements above.

Specific entry requirements

English 6 (or by other approved means with the equivalent proficiency level)
Applicants enrolled in a programme at Chalmers where the course is included in the study programme are exempted from fulfilling the requirements above.

Course specific prerequisites

Basic undergraduate mathematics and probability theory.

Aim

The course introduces the students to important concepts in information theory that can be used to describe and characterise complex systems. The concepts are applied to a number of areas in complex systems: cellular automata, fractals, chemical self-organisation, and chaos. The main aim is to give students the knowledge and skills to apply information theory to a wide variety of different systems. The course also gives a presentation of the connections between information theory and physics, primarily statistical mechanics and thermodynamics.

Learning outcomes (after completion of the course the student should be able to)

Define and apply fundamental concepts in information theory at a level where it becomes useful as a tool for problem solving, analysis and understanding.

In the course you will be able to achieve this by, for example:

- Defining and using the basic concepts of information theory: Shannon entropy, relative entropy, complexity measures based on these
- Using information theory to characterise both cellular automata and low-dimensional chaos
- Understanding the connection between information theory and statistical mechanics
- Using geometric information theory to characterise patterns in spatically extended systems like pictures
- Explaining how information is flowing in chemical self-organising systems exhibiting pattern formation

Content

Basic concepts of information theory
- Shannon entropy, relative information, complexity measures

Information theory for symbol sequences and lattice systems
- correlations and randomness in symbol sequences

Information theory and physics
- entropy in physics and its relation to randomness in information theory

Cellular automata
- order and disorder in the time evolution of various cellular automaton rules

Geometric information theory
- presentation of an information-theoretic framework for characterising patterns

Self-organising chemical systems
- flows of information in the process of pattern formation

Chaotic systems and information
- flows of information from micro to macro in chaotic systems

Organisation

The course is based on a series of lectures, in total 30 hours, covering the topics listed above. Every week a set of home assignments are distributed. An optional mini project can also be done.

Literature

K. Lindgren,  Information theory for complex systems - An information perspective on complexity in dynamical systems, physics, and chemistry. Lecture notes, Chalmers, 2014.

Examination including compulsory elements

The examination will be based on a final written exam, with possibility of extra points from homework assignments and the optional mini project.

The course examiner may assess individual students in other ways than what is stated above if there are special reasons for doing so, for example if a student has a decision from Chalmers about disability study support.