Description
Date depot: 1 janvier 1900
Titre: 3D emotional rendering and animation models
Directrice de thèse:
Catherine PELACHAUD (ISIR (EDITE))
Domaine scientifique: Sciences et technologies de l'information et de la communication
Thématique CNRS : Non defini
Resumé:
Context
The project takes place within the CECIL project, an ANR project that will start in September 2009 and will last for 3 years. The aim of the project CECIL is to contribute to area of interaction systems by placing emotion at the heart of the interaction between the system and the human user. To meet this general objective the project will clearly define (and disambiguate the definition of) some set of emotions, and incorporate them into the reasoning processes of an embodied conversational agent (ECA). In particular the agent will be endowed with the capabilities to express its emotions by means of different modalities including facial expressions, gestures, and language.
The agent system is based on an existing ECA system Greta. The system takes as input a text to be said by the agent. The text has been enriched with information on the manner the text ought to be said (i.e. with which communicative acts it should be said). The behavioral engine computes the synchronized verbal and nonverbal behaviors of the agent. The animation module follows the MPEG-4 and the H-Anim standards.
PhD Topic
The objective of the work is to develop an embodied conversational agent (ECA) which is capable to manage its emotions and their expressions and to display expressive behaviours. That is, the ECA will have to determine which expressions to show in a particular context and be able to display it.
To this aim, we will develop a qualitative model of multimodal expression of emotions. The model will include not only verbal language but also other multimodal signals such as facial expressions, gaze direction (eye and head direction) and gestures. The model will also consider behaviour expressivity.
Emotions are expressed through the whole body and not solely with facial expressions. Gestures, postures, gaze direction are signs of emotions. The multimodal behaviours are synchronized with each others to produce a coherent message. Moreover, the expression of emotion does not correspond to a static expression. Its behaviours are dynamic, they evolve through time. They form a sequence of behaviours whose meaning convey the emotion. The model of emotional behaviours should be extended to the whole body and should be dynamic.
Evaluation of the model will be performed.
A second aspect of the training regards the rendering of the scene. The existing agent system is rendered with standard OpenGL. The goal will be to reach state-of-the-art quality in real-time human rendering systems by using the recent 3D engines such as OGRE to integrate high quality geometric and appearance models. In particular, high resolution surface synthesis using displaced subdivision surfaces should be integrated in OGRE thanks to recent real time tessellation methods. Advanced lighting effects such as soft shadows, subsurface scattering and radiance transfer, as well as extra post-processing effects, such as depth-of-field and motion blur, will also be added to the rendering engine. A particular focus will be put on the the natural shading effect in real time application, such as subsurface scattering for skin models and anisotropic reflections for hair models. Low-cost computational methods reproducing these effects will be targeted.
Work Description
The PhD student will start by establishing the state of the art in the field of screen-space rendering techniques [SSA,ADG,HAG]. Beyond the use of ambient occlusion [SSA] and one-bounce global illumination [ADG], a particular focus will then be set on the development of a new screen-space formulation of subsurface scattering [APM,ERL], adapting the notion of Gaussian diffusion in texture space [ERH] to the a more general case. A first approach, based on depth image anisotropic diffusion will be explored, using recent advances in fast bilateral filter approximation [AGI,RTE,GKD] to propose a scalable realistic materials featuring significant subsurface scattering effects. The work will then focus on how screen-space and object/texture space methods can be balanced [HBO09] to define a proper yet efficient subsurface scattering model. In the second part of the PhD thesis, the student will include the so-defined models within the GRETA system [GRE] and start to develop the qualitative multimodal expression of emotions. The work will go beyond existing works that consider few expressions of emotions [EDE]. It will focus on creating a variety of multimodal expressions [MFE,ADC,ERS,PGA,ECM]. Expressions will be viewed as dynamic behaviours and no more static ones [MEE]. Thus, particular attention will be paid on the temporal evolution of expressions. The combination of dynamic human body animation and realistic face rendering will be evaluated to provide scalable expressive avatars within the GRETA system.
Doctorant.e: Huang Jing