Skip to content

Prompt Alignment

Prompt Alignment measures semantic agreement between protocol anchor frames of a generated video and the target instruction T=sample.prompt_targetT = \texttt{sample.prompt\_target}. It evaluates whether the prediction realizes the requested scene content at benchmark-sampled time points, not whether the full rollout is temporally coherent or visually realistic.

Let A={fi}i=1M\mathcal{A} = \{f_i\}_{i=1}^{M} denote the anchor-frame set selected by the evaluation protocol from the decoded prediction. With CLIP ViT-B/16 image encoder ϕI\phi_I and text encoder ϕT\phi_T, define L2-normalized embeddings ei=ϕI(fi)\mathbf{e}_i = \phi_I(f_i) and t=ϕT(T)\mathbf{t} = \phi_T(T). The native CLIPScore for frame ii is

si=max⁡ ⁣(100⋅⟨ei,t⟩, 0)(1)s_i = \max\!\left(100 \cdot \langle \mathbf{e}_i, \mathbf{t} \rangle,\, 0\right) \tag{1}

WorldAtlas maps each anchor score to [0,1][0,1] via s~i=clamp(si/100, 0, 1)\tilde{s}_i = \mathrm{clamp}(s_i / 100,\, 0,\, 1) and reports the mean

Sprompt=1M∑i=1Ms~i(2)S_{\mathrm{prompt}} = \frac{1}{M}\sum_{i=1}^{M} \tilde{s}_i \tag{2}

Higher SpromptS_{\mathrm{prompt}} indicates stronger instruction following at protocol anchors. The metric is official when explicit instruction supervision is available and conditioning is not self-referential (conditioning_strategy ≠ self). Image suites may condition on a single reference frame, but the model prediction must still be a video.

The examples below are curated from the complete 2026-07-17 image_static snapshot: 20 models evaluated on the same 3,372 cases. Every side-by-side pair is strictly verified to share the sample ID, conditioning-image SHA-256, and actual generation-prompt SHA-256; both generation records must also identify that image as the conditioning input. “Good” and “poor” refer only to Prompt Alignment—not overall video quality—and each score is the mean over four segment_random protocol anchors.

Audit manifest: docs/assets/metric_examples/curated_same_case_manifest.json.

Input image: image_static_stylized_public_spaces_0029
Input image
Input verified · same benchmark case · gap 0.1526

image_static_stylized_public_spaces_0029

Suite: image_staticScene: stylized / indoor / public_spaces
Benchmark prompt target

Camera orbit left. A vintage car display with wooden floor, framed artworks, and cars.

prompt source: prompt_target

Both first frames passed geometric matching against the displayed input; labels compare this metric only.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.3508

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.1982
Input image: image_static_stylized_workspaces_0016
Input image
Input verified · same benchmark case · gap 0.1645

image_static_stylized_workspaces_0016

Suite: image_staticScene: stylized / indoor / workspaces
Benchmark prompt target

Camera pan right. A cozy workspace with desk, plants, and window.

prompt source: prompt_target

Both first frames passed geometric matching against the displayed input; labels compare this metric only.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.3426

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.1781
Input image: image_static_stylized_workspaces_0044
Input image
Input verified · same benchmark case · gap 0.1533

image_static_stylized_workspaces_0044

Suite: image_staticScene: stylized / indoor / workspaces
Benchmark prompt target

Camera pan right. A modern loft bed with bookshelf, indoor plant, and wall.

prompt source: prompt_target

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.3331

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.1798