Metrics Overview

All documented and leaderboard metrics are official. Each metric retains its validity checks, input requirements and N/A rules; the status policy does not change scores or aggregate weights.

This page follows the public benchmark surface declared in config/benchmark.yaml. The public generation surface is image_static and image_dynamic. memory_loop is a separate official track for interactive world models and is not folded into those two totals. video_static and video_dynamic are collected real videos used as the benchmark.reference_corpus for jedi, fvmd, and trajan; they are not a video-prefix generation track. experimental is a supplemental slice and should not be merged into a single headline benchmark total.

Reading Rules

The report contract uses the following fields:

• raw — backend-native value; may be a scalar or a component dictionary
• normalized — [0, 1] score written into aggregation; higher is always better
• official — the single public status for every documented and leaderboard metric with valid measurements
• not_applicable — strong supervision is absent, so no score is computed or aggregated; recorded in not_applicable_summary.json
• normalized = null — runtime failure or missing input; it is not automatically treated as zero

Per-metric normalization, suite coverage, and official eligibility are documented on each metric page.

Suite Vectors

Suite	Metrics	Notes
image_static	43	Image-conditioned static world generation; video predictions are evaluated with the video metric surface when inputs allow
image_dynamic	33	Broader motion-prompt coverage than image_static; does not evaluate camera_error
video_static	37	Collected real videos used only as the jedi / fvmd / trajan reference distribution; not a generation suite
video_dynamic	45	Collected real videos used only as the jedi / fvmd / trajan reference distribution; not a generation suite
memory_loop	6	Still-image conditioning, 60-second closed-loop rollout; interactive world models only; scores roll into long_sequence
experimental	20	Supplemental slice; not part of the formal headline benchmark

All documented metrics are official. Whether a sample yields a valid measurement or not_applicable depends on its inputs and backend success; see each metric page and the Benchmark Overview.

Five Evaluation Pillars

The Metrics navigation is organized by five top-level dimensions. Closed-loop memory is a family inside long_sequence, not a separate headline axis.

Dimension	Index page
Long sequence / Memory	Long Sequence / Memory
Action control	Action Control
3D/4D consistency	3D/4D Consistency
Physics	Physics
Quality	Quality