A multidimensional approach to the study of Gesture: Exploring the M3D labelling system
7th November (6:30 HK time), Online
Register via QR Code in the poster or register at https://polyu.hk/BILtd
International Society of Gesture Studies Hong Kong
Manual gesture annotation is a predominant method used in gesture research to collect data for analysis. However, unlike other fields such as speech prosody, the community has not broadly come together in support of a single annotation system. This results in different research groups creating individual annotation systems with very specific aims to address their own research objectives. This situation poses a challenge for the community, as it becomes difficult to make direct comparisons across studies and thus hinders the progression of the field. The MultiModal MultiDimensional (M3D) annotation system (Rohrer et al., 2023) addresses this issue by offering an annotation system that (1) reconciles several views into a more widely agreed gesture classification system and that (2) is openly accessible, explicitly described, and offers material to facilitate the learning and its application to novel data. Specifically, M3D builds on currently available systems by explicitly proposing a tripartite dimensional system that allows for the assessment of gestural form (i.e., configuration and motion trajectory), gestural prosody (i.e., timing of movement), and gestural meaning (both referential and pragmatic) in a largely independent and non-mutually exclusive manner. Furthermore, it is the first labeling system to offer an interactive online training program for its application to a range of databases.
​
The aim of this talk is to describe the M3D system and demonstrate how such a multidimensional approach to gesture annotation is beneficial and advances the field. Specifically we will explore how the M3D system is flexible and does not force researchers to adopt a completely new theoretical framework, but rather can be adapted according to individual research objectives and theoretical approaches. We will also explore some of the resources available for interested researchers to learn and apply M3D.
​
Rohrer, P. L., Vilà-Giménez, I., Florit-Pons, J., Gurrado, G., Esteve-Gibert, N., Ren-Mitchell, A., Shattuck-Hufnagel, S. & Prieto, P. (2023). The MultiModal MultiDimensional (M3D) labeling system. https://doi.org/10.17605/OSF.IO/ANKDX
Patrick Louis Rohrer
Register at https://polyu.hk/BILtd
Patrick Louis Rohrer is a Postdoctoral Researcher at the Donders Center for Cognition [DCC], part of the Donders Institute for Brain, Cognition, and Behavior at Radboud University in Nijmegen, The Netherlands. He received his PhD in 2022 within the framework of an international joint cotutelle between Universitat Pompeu Fabra (Barcelona, Catalunya) and Nantes Université (Nantes, France). His main research interest lies in the relationship between gesture and prosody, their joint contribution to pragmatic meaning from a crosslinguistic perspective, and their effects on speech perception, cognition, and acquisition.
​
He is currently a member of the Speech Perception in Audiovisual Communication [SPEAC] research group, funded by the ERC Starting Grant ‘HearingHands’ [PI: Hans Rutger Bosker]. His project within this research grant continues the investigation of gesture-speech synchrony in a set of typologically diverse languages. The project also assesses the effects of this synchrony on audiovisual speech perception. Additionally, he is interested in how prosody and gesture is processed by native and non-native speakers, and their effects on comprehension, memorization, and acquisition using both behavioral and electrophysiological (i.e., EEG) techniques.
register for event
After registering, you will receive a confirmation email containing information about joining the zoom meeting.
event chairperson & contact
Renia Lopez