Skip to content

Memory Rendering Recovery

This metric evaluates whether lighting, chroma, and texture at a revisited viewpoint return to how they first appeared. It discards high-frequency content on purpose, targeting the failure where geometry is still there but the scene is relit or restyled. Spatial CIELAB means ignore object outlines, so this metric does not replace the DINOv2 / SSIM reading in memory_revisit_fidelity.

It uses the same pose-paired frames as revisit fidelity. A failed memory_return_gate does not block scoring; it uses the same nearest-pose fallback as fidelity when strict revisit pairs are absent, and records gate status and pair source. The revisit frame is resized to the reference size and converted to OpenCV 8-bit CIELAB. Lightness is the absolute difference of mean LL restored to [0,100][0,100]; chroma is the Euclidean distance between mean a,ba,b after removing the 128128 offset:

ΔL=∣Lˉa−Lˉb∣⋅100255,ΔC=∥(aˉa,bˉa)−(aˉb,bˉb)∥2(1)\Delta L = \left|\bar{L}_a - \bar{L}_b\right|\cdot\frac{100}{255}, \qquad \Delta C = \left\| (\bar{a}_a, \bar{b}_a) - (\bar{a}_b, \bar{b}_b) \right\|_2 \tag{1}

The two deltas map to exponentially decaying similarities; the texture term is the same transform of the VGG Gram distance dGd_G (the descriptor used by style_consistency):

sL=exp⁡(−ΔL/τL),sC=exp⁡(−ΔC/τC),sG=exp⁡(−dG/τG)(2)s_L = \exp(-\Delta L / \tau_L), \qquad s_C = \exp(-\Delta C / \tau_C), \qquad s_G = \exp(-d_G / \tau_G) \tag{2}

Defaults are τL=6\tau_L = 6, τC=8\tau_C = 8, and τG=0.002\tau_G = 0.002. When all three terms exist they are averaged; a missing VGG checkpoint drops the texture term and averages lightness and chroma only. The video score is the mean over the selected KK pairs:

sk={(sL+sC+sG)/3,Gram available(sL+sC)/2,otherwise,Srend=1K∑k=1Ksk(3)s_k = \begin{cases} (s_L + s_C + s_G)/3, & \text{Gram available} \\[4pt] (s_L + s_C)/2, & \text{otherwise} \end{cases}, \qquad S_{\mathrm{rend}} = \frac{1}{K}\sum_{k=1}^{K} s_k \tag{3}

In WorldAtlas this algorithm is implemented as the official metric memory_rendering_recovery (cielab_gram backend). Under the default normalization config, raw and normalized outputs lie in [0,1][0,1] with higher is better. On memory_loop it is official whenever the selected pairs can be scored, including after a failed gate; the texture term may degrade on its own.

Audit manifest: docs/assets/metric_examples/longseq_memory_60s_examples_20260909.json.

Input image: memory-dolly_loop-memory_pool_002
Input image
60s memory_loop · same benchmark case · gap 0.8937

memory-dolly_loop-memory_pool_002

Suite: memory_loopScene: photorealistic / outdoor / city
Benchmark prompt target

Move the camera forward through the scene, then back to where it started.

prompt source: memory_loop_prompt

Check lighting and style on revisit. The better example keeps the same palette; the weaker one shifts exposure or rendering style.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9392 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0455 · official
Input image: memory-square_loop-memory_pool_396
Input image
60s memory_loop · same benchmark case · gap 0.9288

memory-square_loop-memory_pool_396

Suite: memory_loopScene: photorealistic / outdoor / terrestrial_landscape
Benchmark prompt target

Walk a closed square around the scene and return to the starting view.

prompt source: memory_loop_prompt

Check lighting and style on revisit. The better example keeps the same palette; the weaker one shifts exposure or rendering style.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9438 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0150 · official
Input image: memory-yaw_loop_180-memory_pool_147
Input image
60s memory_loop · same benchmark case · gap 0.8957

memory-yaw_loop_180-memory_pool_147

Suite: memory_loopScene: photorealistic / outdoor / terrestrial_landscape
Benchmark prompt target

Turn the camera to the right, then turn back to the starting view.

prompt source: memory_loop_prompt

Check lighting and style on revisit. The better example keeps the same palette; the weaker one shifts exposure or rendering style.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9021 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0064 · official
Input image: memory-palindrome_loop-memory_pool_255
Input image
60s memory_loop · same benchmark case · gap 0.8686

memory-palindrome_loop-memory_pool_255

Suite: memory_loopScene: photorealistic / indoor / workspaces
Benchmark prompt target

Move forward, turn and continue, then retrace the whole path back.

prompt source: memory_loop_prompt

Check lighting and style on revisit. The better example keeps the same palette; the weaker one shifts exposure or rendering style.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.8940 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0254 · official
Input image: memory-double_yaw_loop-memory_pool_405
Input image
60s memory_loop · same benchmark case · gap 0.8988

memory-double_yaw_loop-memory_pool_405

Suite: memory_loopScene: photorealistic / outdoor / city
Benchmark prompt target

Turn away and back to the start twice in a row.

prompt source: memory_loop_prompt

Check lighting and style on revisit. The better example keeps the same palette; the weaker one shifts exposure or rendering style.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9179 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0191 · official