PAJAMA / LEARN
← Course overview

Learn from observations / 07 OF 12

Good training starts before the first gradient

After this lessonAssemble an image–camera–time training sample and prevent coordinate, resizing and evaluation mistakes.

If a photograph says “look left” but its pose says “look right,” the optimizer receives contradictory instructions. More iterations do not repair a broken data contract.

Build the idea, one step at a time

Need a math refresher?

Open only the tool you need; return here to use it in the next step.

Matrices & covariance

STEP 1 / 5

A sample is more than an image

Associate each target image with camera intrinsics, an extrinsic convention, timestamp, image dimensions and preprocessing. Masks and exposure settings may also matter. Keep filenames and camera IDs traceable. Training compares pixels produced using this exact camera and time against this exact target.

D={(In,Kn,En,tn)}n=1N\mathcal D=\{(I_n,K_n,E_n,t_n)\}_{n=1}^{N}

PUT NUMBERS IN

An 800 × 600 image with fx = 600 and cx = 400 becomes 400 × 300. With no crop, fx′ = 300 and cx′ = 200. A point at X/Z = 0.1 moves from u = 460 to u′ = 230. Keeping fx = 600 would incorrectly predict 260.

FORMULA → RUNNING CODE

Find the same operation in the code

These excerpts are read from the current repository when the site is built. Expand one to inspect it; the full snapshot opens in a separate tab.

public/learn/models.mjsL22–22

The exact arithmetic behind this lesson’s controls and worked example.

export function resizeSample(r,wrong=false){return {focal:wrong?600:600*r,principal:400*r,expected:460*r,actual:(wrong?600:600*r)*.1+400*r};}
Open the deployed source snapshot ↗
public/math/training.mjsL10–19

The browser experiment constructs fixed targets with explicit times and pixel coordinates.

export function createTrainingData() {
  const truth=initialRecords();
  truth[0][0]=1.12; truth[0][8]=0.8; truth[0][20]=0.2;
  const {width,height,left,top,times}=trainingSpec;
  return times.map(time=>({time,samples:Array.from({length:width*height},(_,i)=>{
    const pixel=[left+i%width+0.5,top+Math.floor(i/width)+0.5];
    return {pixel,target:forward(truth,time,pixel).color};
  })}));
}
Open the deployed source snapshot ↗

CHECK YOUR UNDERSTANDING

Can you predict the result?

Halve image dimensions with no crop. What should happen to focal lengths in pixels?

This is your own review mark. You can remove it at any time.

Go deeper with primary sources

COLMAP · camera and pose conventions ↗3D Gaussian Splatting · paper & implementation ↗Spacetime Gaussian Feature Splatting · paper ↗Full mathematical derivation ↗