Imagine Internships

(2011-2012)

Advisors

Rémi Ronfard, IMAGINE team
Olivier Palombi, IMAGINE team

Contact : remi.ronfard@inria.fr (04 76 61 53 03)

Context

The thesis seeks to address the difficult problem of controlling and coordinating the movements of virtual actors in 3D animation. Our long term research goal is this area is to create a nervous system for virtual actors that can interpret high level commands such as take a book from the library and sit down to read it and to decompose it into a low level motor control program that can be executed and coordinated in a real-time simulation environment. In this thesis, we propose to approximate the virtual actor with a hierarchy of deformable body parts, each equipped with a small number of low-level motor controllers, and to build a library of higher-level movements by coordinating the movements of their subparts. We will use an existing ontology of anatomic structures and functions as a starting point.

Objectives

Previous work in multimodal interaction with virtual actors such as the interactive movie system [1] and virtual theater system [2] has focused on a "user-as-spectator" or "user-as-actor" metaphor. Recently, Wang and Van de Panne introduced the "user-as-director" metaphor in their "Walk to here" paper [3] which used a voice and mouse-pointing interface for directing actors and cameras. Currently, such approaches are limited to a small, pre-recorded library of motion-capture data.
The goal of the thesis will be to investigate procedural animation systems such as [4,5,6] for directing the virtual actor with high-level commands that can decomposed into a combination of physics and keyframe animations. More specifically, the candidate will use anatomic knowledge to build a small dictionary of primitive gestures, encoded with a combination of physics and keyframe animation at multiple levels of detail. Then he will combine and synchronize those gestures together in response to more complex director's command, by transcribing anatomic knowledge about human motion into a rich temporal specification language such as the situation calculus [7] or the event calculus [8].
The proposed methodology is expected to be useful in at least two ways. First, we will use it to illustrate an existing anatomical ontology of musculo-skeletal functions of the human body with interactive animation. Second, we expect that the methodology can be used to design and implement a high-level, multimodal command language for directing the actions of computer animated characters in short movies, video games and virtual theatre rehearsals.

Keywords

Physically-based animation, body action units, gesture controllers, natural language command of virtual actors.

References

[1] Nakatsu, R etc. Interactive movie system with multi-person participation and anytime interaction capabilities, Proceedings of the sixth ACM international conference on Multi-media: Technologies for interactive movies, Bristol, United Kingdom, Pages: 2 - 10, 1998.

[2] Oliveira, J.C. etc., Virtual Theater for Industrial Training: A Collaborative Virtual Environment, Proceedings of 4th World Multiconference on Circuits, Systems, Communications & Computers (CSCC 2000), Greece, July 2000.

[3] Z. Wang and M. van de Panne. 2006. Walk to here: a voice driven animation system. In Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation (SCA '06).

[4] Mira Dontcheva, Gary Yngve, Zoran Popovic. Layered Acting For Character Animation. ACM Transactions on Graphics (SIGGRAPH 2003).

[5] Stelian Coros, Philippe Beaudoin, Michiel van de Panne. Robust Task-based Control Policies for Physics-based Characters. ACM Transactions on Graphics (Proc. ACM SIGGRAPH ASIA 2009).

[6] Hironori Mitake, Kazuyuki Asano, Takafumi Aoki, Marc Salvati, Makoto Sato, Shoichi Hasegawa : 'Physics-driven Multi Dimensional Keyframe Animation for Artist-directable Interactive Character', Computer Graphics Forum, Vol.28, No.2, pp.279-287, 2009.

[7] Raymond Reiter, Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems. MIT Press, 2001.

[8] Erik Mueller. Automating commonsense reasoning using the event calculus. Communications of the ACM, 52(1), 2009.