Fiche de YUNUS Fajrian

Titre : Prediction of Gesture Timing and Study About Image Schema for Metaphoric Gestures
Date : 13 décembre 2021
Heure: 14:00
Résumé : We want to predict gestures based on the speech. In one part of this thesis, we develop a model based on a recurrent neural network with attention mechanism to predict the gesture timing based on the speech acoustic. We use a sequence comparison technique to evaluate the model performance. In another part of the thesis, we deal with the generation of metaphoric gestures. The extraction of the relevant semantics from the content of the speech is done by using the concept of image schema. Yet, machine learning models need vectors as their inputs. We investigate how we can transform the image schema into vector by using word embedding techniques. Lastly, we investigate how we can represent hand gesture shapes. The representation has to be compact enough yet it also has to be broad enough such that it can cover sufficient shapes which can represent sufficient range of semantics. We develop this representation based on the gestures employed in our corpus of natural dyadic interaction.
Lieu : Campus de Jussieu, 4 Place Jussieu, 75005 Paris, FRANCE