Beneath the Skin: Interactive Biomechanical Facial Simulations via Composable Co-Expressions

Fig. 1. Overview of our biomechanically based facial animation framework. Our approach generates visually realistic and anatomically accurate expressions by propagating internal muscle dynamics directly to an epidermal skin layer. a. The system takes a rest pose and multiple artist-sculpted target poses, such as the phoneme "N" and a "raise brow" expression, triggered via a keyboard interface. b. Continuous deformation trajectories of the internal musculature are computed by jointly minimizing Euclidean distance and biomechanical energy consumption. c. A sagittal cross-section reveals the multi-layered internal anatomy. d. A topologically reduced musculature proxy mesh, augmented with fat compartments, used for skin registration. e. A generic template skin is physically registered to the personalized internal tissues in a one-time, offline preprocessing step. f. The dynamic sequence of the registered skin mesh, driven by force propagation from the underlying active muscle layers, shown alongside heatmaps of pixel deviations from the rest pose. g. The head model shakes while doing the same phoneme and expression, demonstrating the passive dynamics. h, i. External finger interactions with the soft tissue, including a detailed zoomed-in view. j. A pathological simulation modeling third-degree burn damage to both muscle and skin. k, l. Asymmetric expressions visualized in both the muscle and skin layers, respectively. m. The interactive interface, which allows users to animate the face model via mouse clicks, text prompts, or a piano-style keyboard mode.

Realistic facial animation plays a pivotal role in animating convincing, expressive characters. While kinematic techniques offer great artistic engagement, and motion capture offers explicit human performances, biomechanical simulation promises physical plausibility. However, physical approaches often sacrifice artistic control due to the complexity of biomechanical motion synthesis. We argue that rigorous physics and intuitive control are not mutually exclusive; they can be seamlessly bridged through physical motor controllers that translate high-level artistic intent into underlying muscle activations. This framework significantly resolves the persistent historical trade-off between physical accuracy and animation usability.
Our framework generates muscle activation sequences for standard Facial Action Coding System poses, which artists can then compose into complex, realistic motion sequences. A key finding of our work is that simulating the smooth overlap of these controllers naturally aligns with the energy parsimony of human speech coarticulation, automatically generating biomechanically accurate trajectories. To fully realize this inside-out pipeline, we also introduce a physically-driven skinning technique that seamlessly attaches to our internal musculoskeletal model. Our experiments demonstrate that this optimization approach matches the visual fidelity of kinematic benchmarks while inherently exhibiting critical physical properties—such as volume conservation, gravity effects, and passive dynamics—at interactive speeds. Ultimately, our fully coupled framework proves that complex internal biomechanics can be practically integrated into artist-directable pipelines, granting unprecedented physical realism without sacrificing creative flexibility.