Description
Date depot: 5 mars 2025
Titre: Unsupervised pre-training of flexible multitask agents by learning to achieve a large diversity of goals
Directeur de thèse:
Olivier SIGAUD (ISIR (EDITE))
Encadrant :
Nicolas PERRIN-GILBERT (ISIR (EDITE))
Domaine scientifique: Sciences et technologies de l'information et de la communication
Thématique CNRS : Intelligence artificielle
Resumé: In deep reinforcement learning, pre-training a generic policy can help the agent learn a more specific policy in a second stage with higher sample efficiency and learning performance. This PhD project casts the pre-training problem as an unsupervised reinforcement learning problem in which an initially naive agent generates a very wide diversity of goals and learns to achieve these goals using simple goal-dependent reward functions. Such a pre-trained policy can then be readily used or quickly fine-tuned to achieve more sophisticated user- defined goals. This PhD topic lies at the crossroad between unsupervised learning, multitask transfer, goal-conditioned reinforcement learning (GCRL) and quality-diversity (QD) methods. The PhD advisors, Olivier Sigaud and Nicolas Perrin-Gilbert, have a significant expertise in these domains. This PhD project will contribute to a larger effort at ISIR towards combining recent learning architectures, foundational models and RL to build general purpose policies for robots.