Beneath the Skin: Interactive Biomechanical Facial Simulations via Composable Co-Expressions

Fig. 1. Overview of our physically and biomechanically based facial animation framework. Our approach generates visually realistic and anatomically accurate expressions by propagating internal muscle dynamics directly to an epidermal skin layer. a. The system input consists of an arbitrary rest pose and an artist-sculpted target pose. b. By jointly minimizing Euclidean distance and biomechanical energy consumption, we compute the continuous deformation trajectory of the internal musculature. c. A sagittal cross-section revealing the multi-layered internal anatomy. d. A topologically reduced musculature proxy mesh, augmented with fat compartments, utilized for skin registration. e. A generic template skin is physically registered to the personalized internal tissues. This physical registration is a one-time, offline preprocessing step per character model. f. The dynamic sequence of the registered skin mesh, faithfully driven by the force propagation from the underlying active muscle layers, with the heatmap of the pixel change compared to the rest pose. g. The final textured and rendered skin at the target pose; the red dashed box indicates the region of the cross-section shown in Fig. 1d.

Realistic facial animation plays a pivotal role in animating convincing, expressive characters. While kinematic techniques offer great artistic engagement, and motion capture offers explicit human performances, biomechanical simulation promises physical plausibility that respects volume preservation, passive dynamics, momentum, deformation, and gravity. However, physical approaches have often come at the price of reduced artistic engagement due to the mismatch between the complexity and constraints of biomechanical motion synthesis and the expressive flexibility demanded by artists. We provide a partial resolution to this tension through a biomechanical musculoskeletal facial model that is bridged to expressive animation through the generation of physical motor controllers that interpolate facial configurations and action units, including speech. We chose the FACS model [Ekman et al. 1978] that is commonly used by character animators, although motions, i.e., muscle activation sequences, can be synthesized for any set of physically plausible facial poses. These controllers may then be composed into complex, realistic motion sequences. Furthermore, the optimal simulation of these sequences requires muscle activation co-expression in a manner that is consistent with coarticulation in speech synthesis. While our primary goal is to demonstrate ``under the skin'' motion synthesis, we describe a skinning technique that attaches to our musculoskeletal model to demonstrate the potential of fully biomechanical, expressive, artist-directable facial animation.