PAJAMA / LEARN
← Course overview

See how pixels are made / 01 OF 12

From photographs to a scene you can move through

After this lessonExplain what is learned, what is rendered, and why the two use the same forward function.

Imagine arranging translucent, colored pieces of cotton until every camera sees the right picture. Each piece has a small set of numbers. Training adjusts those numbers; rendering reads them.

Build the idea, one step at a time

Need a math refresher?

Open only the tool you need; return here to use it in the next step.

Vectors & weighted sums

STEP 1 / 5

Start with the answer: pixels

A photograph is a grid of RGB triples. A camera pose says where it was taken; intrinsics say how rays map to pixels. For a dynamic scene, a timestamp says when it was taken. The photograph does not directly tell us the 3D position of every visible detail.

PUT NUMBERS IN

Follow Gaussian A at t = 0.5. Its center is (1, 2, 10). With f = 100 and principal point (64, 48), its screen center is (74, 68). We inspect pixel (74.5, 68.5), then combine it with Gaussian B. Every stage below is calculated from the existing teaching evaluator.

FORMULA → RUNNING CODE

Find the same operation in the code

These excerpts are read from the current repository when the site is built. Expand one to inspect it; the full snapshot opens in a separate tab.

public/math/trace.mjsL94–109

The forward function consumes records and returns the pixel calculation.

export function forward(records, time = defaults.time, pixel = defaults.pixel, options = {}) {
  const expected = options.target ?? target;
  const splats = records.map((row, id) => ({ id, ...evaluateRecord(row, time, pixel) }))
    .sort((a, b) => a.mean[2] - b.mean[2]);
  let remaining = 1, color = [0, 0, 0];
  for (const splat of splats) {
    splat.transmittance = remaining;
    splat.contribution = scaled(splat.color, remaining * splat.alpha);
    color = plus(color, splat.contribution);
    remaining *= 1 - splat.alpha;
  }
  color = plus(color, scaled(background, remaining));
  const error = color.map((v, i) => v - expected[i]);
  return { splats, remaining, color, target: [...expected], loss: dot(error, error) / 3 };
}
Open the deployed source snapshot ↗

CHECK YOUR UNDERSTANDING

Can you predict the result?

What changes during ordinary playback?

This is your own review mark. You can remove it at any time.

Go deeper with primary sources

3D Gaussian Splatting · paper & implementation ↗Spacetime Gaussian Feature Splatting · paper ↗Full mathematical derivation ↗