Style Consistency

Leaderboard scoring is 100 × mean(clip(1 - raw_i / 0.007, 0, 1)), using each video’s retained Gram distance before averaging. Averaged error_ref values are never inverted. Historical published summaries without matching per-video evidence are identified in score tooltips.

This metric evaluates the stability of visual style—color palette, texture, and coarse appearance statistics—across scenes within a generated rollout. Given protocol-selected anchor frames $\{I_t\}_{t=1}^{T}$ with $T \geq 2$, the first frame $I_1$ serves as the style reference; each later anchor $I_{d+1}$ ($d \in [1, T-1]$) is compared against $I_1$ via multi-layer VGG19 Gram representations.

Each frame $I_t$ is resized ($512 \times 512$ on CUDA, $128 \times 128$ on CPU), converted to a tensor, and normalized with ImageNet mean and standard deviation before a forward pass through VGG19 features. At the outputs of convolutional layers $\mathrm{conv}_1, \ldots, \mathrm{conv}_5$ (before ReLU), feature maps $F_{t,l} \in \mathbb{R}^{C \times H \times W}$ are extracted. The Gram matrix is defined as

$$G(F) = \frac{ \mathrm{vec}(F)\,\mathrm{vec}(F)^{\top} }{ C \cdot H \cdot W } \tag{1}$$

where $\mathrm{vec}(\cdot)$ denotes vectorization. The layer-wise style distance for frame $d+1$ at layer $l$ is

$$\delta_{d,l} = \mathrm{MSE}\!\left( G(F_{d+1,l}),\, G(F_{1,l}) \right) \tag{2}$$

The per-frame distance is the sum over five layers:

$$d_d = \sum_{l \in \{\mathrm{conv}_1, \ldots, \mathrm{conv}_5\}} \delta_{d,l} \tag{3}$$

The raw score is the arithmetic mean over later anchors (lower is better):

$$\mathrm{raw} = \frac{1}{T-1} \sum_{d=1}^{T-1} d_d \tag{4}$$

Normalization follows configured empirical bounds $\varepsilon_{\min}=0$ and $\varepsilon_{\max}=0.007$ (higher_is_better=false). The raw value is first clamped to the empirical range, then mapped to a higher-is-better score in $[0,1]$:

$$\mathrm{raw}' = \mathrm{clip}(\mathrm{raw},\, \varepsilon_{\min},\, \varepsilon_{\max}) \tag{5}$$ $$S_{\mathrm{style}} = 1 - \frac{ \mathrm{raw}' - \varepsilon_{\min} }{ \varepsilon_{\max} - \varepsilon_{\min} } \tag{6}$$

Verified Same-Input Examples

Audit manifest: docs/assets/metric_examples/curated_same_case_manifest.json.

Input image

Input verified · same benchmark case · gap 1.0000

image_static_photorealistic_dining_spaces_0150

Suite: image_staticScene: photorealistic / indoor / dining_spaces

Benchmark prompt target

Camera pan left. A living area with fireplace, chairs, and wall sculpture.

prompt source: prompt_target

Both first frames passed geometric matching against the displayed input; labels compare this metric only.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 1.0000

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0000

Input image

Input verified · same benchmark case · gap 1.0000

image_static_stylized_public_spaces_0146

Suite: image_staticScene: stylized / indoor / public_spaces

Benchmark prompt target

Camera pan left. A futuristic facility with control panels, holographic displays, and neon lights. Camera move left. A futuristic facility with interactive kiosks, virtual reality stations, and holographic displays. Camera pull out. A futuristic facility with security checkpoint, automated doors, and interactive kiosks.

prompt source: prompt_target

Both first frames passed geometric matching against the displayed input; labels compare this metric only.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 1.0000

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0000

Input image

Input verified · same benchmark case · gap 1.0000

image_static_stylized_public_spaces_0059

Suite: image_staticScene: stylized / indoor / public_spaces

Benchmark prompt target

Camera orbit right. A grand library hall with bookshelves, reading table, and chairs.

prompt source: prompt_target

Both first frames passed geometric matching against the displayed input; labels compare this metric only.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 1.0000

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0000

References

Very Deep Convolutional Networks for Large-Scale Image Recognition

Karen Simonyan and Andrew Zisserman. arXiv:1409.1556, 2014. https://arxiv.org/pdf/1409.1556