Add time without breaking the light

Keep the normal perpendicular

A tangent t lies along a surface and its normal n is perpendicular: nᵀt=0. A local linear deformation A turns the tangent into At. We need a new normal whose dot product with At is still zero.

Choose n′=A⁻ᵀn. Transpose it: n′ᵀ=nᵀA⁻¹. Then n′ᵀAt=nᵀA⁻¹At=nᵀt=0. The inverse cancels the deformation. Normalize the result because the BRDF needs unit directions.

Stretch a diagonal surface by hand

In 2D take n=(1,1)/√2 and t=(1,−1). Stretch x by 2: A=diag(2,1), so t′=(2,−1). Correct n′ is proportional to (1/2,1), giving dot product 1−1=0. The tempting An is proportional to (2,1), giving 4−1=3 instead.

For pure rotation R, R⁻ᵀ=R; rotating positions and normals alike works. Nonuniform scale or shear breaks this shortcut. Set time to 0.25 in the lab and toggle the intentionally wrong transform to see the shading error.

Time also changes visibility

For nonlinear motion x′=f(x,t), use local A=∂f/∂x. Transport covariance by AΣAᵀ, normals by A⁻ᵀ, and material identity with the canonical point. Degenerate A needs rejection or a defined fallback. A separately learned normal field needs supervision or consistency constraints.

Moving geometry changes light visibility V(x,l,t); a static occlusion bake cannot follow a moving blocker. BEAM studies volumetric video, BecomingLit specializes in heads, and DR-GS drives reconstructed assets with simulation/animation. These are different dynamic settings, not interchangeable 4DGS loaders.

The formula meets actual code

This excerpt is extracted from the repository during the site build. SH environment lighting and inverse training are future extensions; their chapters point to the current direct-light/data boundary, not an implemented trainer.

fn surfel(n: Vec3, frame: RelightFrame) -> Splat {
    let a = frame.transform();
    let helper = if n.y.abs() < 0.9 { Vec3::Y } else { Vec3::X };
    let u = n.cross(helper).normalize() * 0.065;
    let v = n.cross(u).normalize() * 0.065;
    let normal_matrix = if frame.wrong_normals {
        a
    } else {
        a.inverse().transpose()
    };
    Splat {
        center: (a * n).extend(0.96).to_array(),
        u: (a * u).extend(0.0).to_array(),
        v: (a * v).extend(0.0).to_array(),
        w: (a * n * 0.008).extend(0.0).to_array(),
        normal: (normal_matrix * n).normalize().extend(0.0).to_array(),
        base: BASE.extend(0.0).to_array(),
    }
}
fn scene(frame: RelightFrame, view: Mat4) -> Vec<Splat> {
    let mut data: Vec<_> = (0..COUNT)
        .map(|i| {
            let y = 1.0 - 2.0 * (i as f32 + 0.5) / COUNT as f32;
            let phi = i as f32 * 2.3999632;
            let r = (1.0 - y * y).sqrt();
            surfel(Vec3::new(r * phi.cos(), y, r * phi.sin()), frame)
        })
        .collect();
    data.sort_by(|a, b| {
        let z = |s: &Splat| view.transform_point3(Vec3::from_slice(&s.center[..3])).z;
        z(a).total_cmp(&z(b)) // RH camera looks along -Z: far first
    });
    data
}

Primary references for this chapter

BEAM · ACM MM 2025 ↗

BecomingLit · NeurIPS 2025 ↗

DR-GS · ECCV 2026 ↗