Lip sync: how spoken lines move the face

Maturity: Draft. Consolidated on 27 September 2026 from the installed 2.31 game's English voice archive, one serialised lip-sync map, three serialised lip-sync animation sets and one voiced scene (WolvenKit CLI 9.0.1, in a private scratch folder), the female player head's facial setup solved with the Cyberpunk Blender add-on's facial solver (pinned 7a4ee793), WolvenKit's type definitions (11720772), the decompiled 2.31 scripts, the Modding Docs at be2f44ee, Audioware's documentation, three installed voiced-quest mods and the JALI paper. Nothing on this page has runtime evidence. Grades follow the knowledge rules: source engine, framework or tool source, decompiled scripts or type definitions; resource extracted game or mod resources; wiki Modding Docs; [paper] a published paper; [offline] measured by our own tools, never the game; hypothesis not established. The design built on these facts is the lip-sync design; the proof of concept is experiment 027.

This page answers: where does a spoken line's facial animation live, what is in it, how does the game play it together with an expression, and how could new voiced lines get lip sync of their own? The facial rig itself (414 control tracks, the facial setup and its solver) is described in facial expressions and facial animation; voice lines, voicesets and subtitles in NPC reactions.

1. The short answers

• Every voiced line has a baked facial clip. CD PROJEKT RED generated them with JALI Research's tools from each recording and its transcript, per language (JALI in Cyberpunk 2077). The game ships the result as ordinary facial animation clips, one per line and per V gender variant, in per-language voice archives resource. There is no JALI code in the game to call: the analysis happened offline [hypothesis, strongly supported: no lip-sync generator exists in the scripts or resources we read].
• A clip is found by scene, actor voice tag and line. A per-language lip-sync map (.lipmap) maps each scene to the lip-sync animation set of each speaking voice tag; the scene's dialogue line names its clip (f_<RUID in hex> / m_<RUID in hex>) resource.
• A clip drives the face's dedicated lip-sync channel, not the expression controls: a fade envelope, JALI's jaw and lip strength (jaliJaw, jaliLips), 141 …LipsyncPoseOutput tracks added on top of the expression, and 86 …AnimOverrideWeight tracks that mute the expression where speech needs the mouth resource. The facial setup combines the two layers, so a smiling face can talk and keep much of its smile [source: the add-on's solver; engine behaviour a hypothesis that the solver matches].
• V does have lip sync. The English voice archive holds 328 v.anims sets, and the lip-sync map lists V's voice tag in 456 scenes (ripperdoc mirror scenes among them) resource. The Modding Docs guide says "V has no lipsync" and WolvenKit's generator skips player actors wiki; that is true of the generator, not of the game data.
• Modders reuse, they don't generate. The wiki's WolvenKit guide copies existing vanilla clips into a mod's lip-sync map, Audioware's guide matches existing clips to new audio the way dubbing does, and three installed voiced-quest mods ship lip-sync maps that point at vanilla sets wiki [source: Audioware docs] resource. We found no mod or tool that generates CP2077 lip sync from new audio.
• Generating it ourselves looks feasible. The target format is fully readable, the Studio already runs the solver that plays it, and open tools give phone timing from audio plus transcript. Our proof of concept produced a clip in the lip-sync channel whose solved lip gap, mouth width and jaw rotation fall in the same ranges as a vanilla V clip, with the lips closing to within 0.25 mm on every bilabial [offline].

2. What a lip-sync clip holds

Measured on the 33 clips of two English sets: V's set for the scene q004_04b_after_tutorial (8 clips, female and male V lines) and Judy's set for the same scene (25 clips) resource.

The tracks it animates

The clip adds to the rig's reference tracks (envelopes and override weights rest at 1, everything else at 0) resource.

Left and right values are identical in the sample (JALI's output is symmetric). The setup's lip-sync poses carry a side flag (1 left, 2 right) and the rig has lipSyncLeftEnvelope/lipSyncRightEnvelope (reference 1, not animated in any sample clip); the add-on's solver reads neither resource source. What they do in the engine is open.

3. How the face combines an expression with speech

The facial setup's solver runs these stages on every frame (the add-on's reimplementation, animation/facial/solver.py at 7a4ee793; the stage order matches the setup's data layout, the arithmetic is the community's reading) source:

flowchart TD
  A["Expression layer: context clip, scene facial animation<br/>(main-pose weights)"] --> B["Envelopes and muzzles"]
  B --> C["Speech limits (only while lipSyncEnvelope > 0):<br/>68 mouth and neck poses clamped to a ceiling<br/>set by jaliJaw or jaliLips, weighted by muzzleLips"]
  C --> D["Influences, upper/lower face scaling"]
  D --> E["Lip-sync overrides: each of 86 poses<br/>times lerp(1, override, envelope)"]
  E --> F["Lip-sync poses: the 141 LipsyncPoseOutput<br/>values added, clamped to 0..1"]
  F --> G["Influences again, inbetweens, correctives"]
  G --> H["Joint rotations and translations"]

• The speech limits are data in the facial setup. In the female basehead setup the jaw open and close, purse, funnel, lips together, suck, puff, tighten and tongue poses have ceilings of 0 (speech owns them), while corner up, wide, stretch and lower raise keep 0.35 to 0.5 and upper raise and pull 0.1 to 0.3 resource. The ceiling moves from its minimum (JA or LI of 0) through its midpoint (1, the reference) to its maximum (2) source. So a speaking face keeps a moderate smile or frown and loses any expression-driven jaw or lip rounding.
• This is JALI's layering in the game's terms: the expression is one layer, speech another, and JALI's two strengths decide how much room the expression's mouth keeps [paper] source.
• Where it enters the graph. V's face graphs run the context's clip through BlendAdditive, then a FacialMixerSlot, then look-at and eye tracks, then the Sermo node that runs the setup resource. Scene facial animation and lip sync are pushed into the mixer slot natively; no script function plays lip sync (the decompiled 2.31 scripts contain none) resource source. Playing a lip-sync clip is the scene and voice-over systems' job.

4. How a line finds its clip

flowchart LR
  A["Scene .scene: actor or player actor<br/>(voicetagId, lipsyncAnimSet)"] --> B["Dialogue line<br/>(locstringId RUID, speaker,<br/>female/maleLipsyncAnimationName)"]
  C["&lt;lang&gt;.lipmap (animLipsyncMapping):<br/>scene path hash → voice tag → anim set"] --> D["Lip-sync set for this language<br/>localization/&lt;lang&gt;/lipsync/…/actor.anims"]
  A --> C
  B --> E["Clip f_/m_ + RUID hex"]
  D --> E
  B --> F["voiceovermap*.json: RUID →<br/>female/male .wem"]
  E --> G["Face: FacialMixerSlot → Sermo"]
  F --> H["Audio"]

5. How modders add voiced lines today

6. Building blocks for our own generator

7. What the proof of concept showed

Experiment 027: a 6 s synthetic English line (Windows speech synthesis, local only), Rhubarb phones, our own JALI-style generator writing the lip-sync channel, solved on the female basehead setup and compared with two vanilla V clips [offline]:

Ranges match; our open vowels run a little wide, and we have not looked at the motion (the Studio has no clip player for arbitrary tracks yet) or compared it in game.

Open questions

1. Does the engine play a scene line's lip sync on V's third-person face outside mirror scenes, and on the photo-mode V? (V's sets exist for 456 scenes.)
2. Does a line's lip sync need its .wem (Audioware's guide says the clip stops when the .wem ends), and can a silent .wem of the right length carry a clip while Audioware plays the real audio?
3. Does the lipmap accept a mod's own sets for V's voice tag in a mod scene, and does the scene need the editor-time …\versions\gold\lipsync\… reference at all?
4. What do lipSyncLeftEnvelope, lipSyncRightEnvelope and the setup's per-pose side flags do?
5. Does the female V's face solve lip-sync clips with the male player setup the face-rig component names (facial animation, question 1)? The clips are authored against the male player skeleton.
6. Can a photo-mode expression clip carrying the lip-sync channel (envelope, poses, overrides) animate V's mouth in photo mode, given that animated photo-mode faces loop?
7. Does stringidvariantlengthsreport.json gate the clip's length, and how do new lines add rows?

Related pages

Facial expressions · Facial animation · NPC reactions · Photo mode · Runtime access · Mod loading · Lip-sync design · Expression editor design