New: full rendering source study
The optimization notes below record earlier snapshots. The new source study follows Spark 2.3.0 and PlayCanvas 2.23.0-beta.24 through loading, SH, CPU and GPU sorting, projection and blending; it also compares our current GPU-driven path.
RESEARCH NOTES · 22 SEPTEMBER 2026
Borrow the architecture. Measure the result.
Spark and PlayCanvas solve large-scene delivery and rendering. Our extra constraint is time: changing motion and opacity can invalidate visibility, order, and residency even when the camera stays still.
What we can absorb now
| Source | Technique / 技术 | Our decision / 采用方式 |
|---|---|---|
| Spark 2.0 | RAD chunks, HTTP Range, precomputed LOD | Borrow independent chunks and bounded residency. Our index uses temporal support; the payload remains exact STG-Lite. RAD is not implemented. |
| Spark sort.rs | Bucket/radix sort; f16 and full-f32 paths | Use exact 32-bit depth keys and stable original-ID ties. Our 11-bit pass size is measured locally; do not quantize depth just to claim a speedup. |
| PlayCanvas render on demand | Draw only after invalidation | Implemented in Rust: time, view, projection, viewport, canvas size and source replacement trigger a draw. Playback remains continuously invalidated. |
| PlayCanvas unified rendering | One global transparent order | Merge active chunks by original ID and sort all resident Gaussians together. Drawing one independently sorted chunk after another breaks cross-chunk compositing. |
These are independently implemented design ideas. No Spark or PlayCanvas source was copied into our renderer. Their performance numbers are not our benchmark results.
What needs a separate quality gate
Spark's compact PackedSplats and PlayCanvas SOG reduce bandwidth with compressed attributes. Their static representations do not contain our cubic motion, temporal Gaussian or quaternion velocity. A dynamic codec must preserve those fields and report motion, color and silhouette errors over time.
Spark compact encoding & LOD · PlayCanvas Streamed SOG
Next GPU experiment: evaluate time → project and cull → compact visible IDs → stable GPU radix sort → indirect draw, with no per-frame GPU-to-CPU readback. PlayCanvas provides portable and NVIDIA-specific OneSweep sort implementations. Start with the portable design; a 4090-only result is insufficient for our cross-platform viewer.
Pinned PlayCanvas GPU sort source
Spatial LOD is also a quality trade-off. Merging nearby Gaussians at one timestamp can merge objects that move apart later. Build temporal segments first, bound motion through the whole segment, then evaluate spatial LOD with temporal consistency tests. This release does not silently decimate points.
A verified 200-second training sequence
DeskGames / Cube has 6,000 frames at 30 FPS, 21 synchronized camera videos, and calibrated COLMAP files. We read the original MP4 timing and frame-count atoms through HTTP Range: 200 seconds, not a repeated clip. The selected videos and calibration total about 4.06 GB. It is a tabletop/room capture, not a city-scale scene, and these files are training inputs—not a trained dynamic Gaussian model.
Acquisition is complete: all 24 selected files were downloaded and checksummed. Local metadata checks confirm 200 seconds and 6,000 frames in all 21 videos. Training has not started.
Calibration gate: every video is 2560×1440, while the supplied PINHOLE calibration is 2521×1418. Resolve the original-to-undistorted crop/warp and camera correspondence before training; blindly scaling intrinsics is not verified.
All-camera metadata & calibration check · Detailed training plan / 详细训练方案
Pinned author dataset · Measured source metadata · 4DGV paper & publication status
SelfCap / LongVolcap remains the stronger minute-scale research candidate. Its project demonstrates long captures, but the linked framework does not currently provide a verified downloadable TGH checkpoint. Request dataset access from its project page; data access alone does not provide a trained renderer-ready model.
Training plan, with a measurable first gate
- Acquire the pinned Cube videos and COLMAP calibration using scripts/acquire-long-data.mjs --download. Verify each file size and published LFS SHA-256. Keep raw data out of Git and R2 until redistribution terms are established.
- Validate camera axes, image names, distortion, intrinsics after resizing, and frame synchronization. Adapt the DeskGames layout to the trainer; do not blindly run the Neural3DV preprocessing script on a different dataset.
- Train one 50-frame STG-Lite pilot, holding out cam00. Record actual wall time, peak VRAM, Gaussian count and held-out image metrics. The published training setup needs a 24 GB GPU; our 4090 meets that capacity, but actual scene fit and runtime still require the pilot.
- Then train neighboring clips and inspect motion continuity at their boundary. Only after that estimate the full run: 120 non-overlapping 50-frame clips, or more if overlap is needed. Do not present concatenated independent models as one continuous learned representation without a segment-aware loader and boundary validation.
- For the long release, add per-segment start/end seconds, local time normalization, chunk hashes and calibrated cameras to the manifest. Measure cold start, random seeks, sustained playback, peak resident memory and image error. The current v1 demo validates time-addressed chunks inside one STG-Lite checkpoint; multi-checkpoint segment switching is planned, not claimed implemented.
Official STG training instructions · Run the streaming and performance lab