Mehrdad Farahani, a PhD student at the DSAI division at Chalmers, will present his work on 'Exploring generative capabilities and capacities for context comprehension and consistency maintenance in dialogue generation models'.
Overview
- Date:Starts 19 June 2023, 14:00Ends 19 June 2023, 14:30
- Location:Analysen, EDIT building
- Language:English
Abstract
Developing open-domain dialogue systems requires thoroughly examining their generative capabilities and capacities for context comprehension, consistency maintenance, and how they draw on diverse memory types to inform their responses. As part of this seminar, two research avenues will be discussed: first, the application of auxiliary tasks to improve autoregressive, decoder-only models such as GPT-2, and second, we will discuss the memory types and their impact on retrieval augmented generation models.
In the first part of the seminar, we examine the under-researched topic of employing auxiliary tasks in GPT-2 models fine-tuned using the PersonaChat and DailyDialog datasets. In this study, four auxiliary tasks were incorporated, which resulted in slightly improved responses that were more consistent with the context or persona. The second part of the article discusses RAG models, which, although they enhance factual consistency and reduce hallucinations, still have several unexplored areas. Research in this study investigates how the RAG model assigns weights and priorities to different memory types, such as parametric, non-parametric, and exchange memory.
About the speaker
Mehrdad Farahani is a second-year PhD student in the DSAI division under the supervision of Richard Johansson. Currently, he is engaged in the research of representations and language models for conversation.
This is a seminar from the DSAI seminars series usually held every Monday at 14:00 by the Data Science and AI division. The seminars are usually hybrid.
