Keystroke nATEr

A normalized, time-aligned rotational shape error with a rotation activity guard:

$$\mathrm{ATE}_r = \frac{\sum_i w_i \cdot (e_{r,\mathrm{start},i} + e_{r,\mathrm{end},i}) / 2}{\sum_i w_i} \tag{1}$$ $$\mathrm{shape}_r = \mathrm{clamp}\!\left( \frac{\mathrm{ATE}_r}{\max(\theta_{\mathrm{obs}},\, \theta_{\min})}, \,0,\,1 \right) \tag{2}$$ $$\mathrm{activity}_r = \frac{n_{\mathrm{mismatch}}}{n_{\mathrm{active}}} \tag{3}$$ $$\mathrm{nATEr} = \max(\mathrm{shape}_r,\, \mathrm{activity}_r) \tag{4}$$

Here $e_{r,\mathrm{start},i}$ and $e_{r,\mathrm{end},i}$ are angular errors in degrees against the target at the two ends of observed interval $i$, within its constant-action segment; $w_i = 1 / \mathrm{FPS}$. $\mathrm{ATE}_r$ is the trapezoidal time integral divided by total observed duration, a time-weighted mean error rather than an unnormalized integral. No arc-length resampling is used. The default minimum rotation normalizer is $\theta_{\min} = 10$ degrees. $\theta_{\mathrm{obs}}$ is the observed total rotation. $n_{\mathrm{mismatch}}$ counts intervals whose commanded and observed rotation-activity states disagree; $n_{\mathrm{active}}$ counts intervals with commanded or observed rotation.

Observed rotation activity means angular speed exceeds rotation_speed_threshold=6.0 degrees/second by default. Commanded but stalled rotation and uncommanded rotation contribute to activity error; observation gaps are missing data, not freezes.

Lower raw error is better. If rotation is neither commanded nor observed, the metric is not_applicable, not a perfect score. Target angular amplitude is prediction-derived (amplitude_calibrated=false); this does not establish accurate execution of an independently specified angle. Backend: pose_action; official metric.

60-Second Same-Input Examples

Audit manifest: docs/assets/metric_examples/longseq_memory_60s_examples_20260909.json.

Input image

60s memory_loop · same benchmark case · gap 0.9871

memory-double_yaw_loop-memory_pool_405

Suite: memory_loopScene: photorealistic / outdoor / city

Benchmark prompt target

Turn away and back to the start twice in a row.

prompt source: memory_loop_prompt

Focus on heading drift over the 60s loop. The weaker example yaws away from the requested orientation.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9871 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0000 · official

Input image

60s memory_loop · same benchmark case · gap 0.9685

memory-palindrome_loop-memory_pool_003

Suite: memory_loopScene: photorealistic / outdoor / terrestrial_landscape

Benchmark prompt target

Move forward, turn and continue, then retrace the whole path back.

prompt source: memory_loop_prompt

Focus on heading drift over the 60s loop. The weaker example yaws away from the requested orientation.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9951 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0266 · official

Input image

60s memory_loop · same benchmark case · gap 0.9546

memory-yaw_loop_180-memory_pool_439

Suite: memory_loopScene: stylized / outdoor / aquatic_landscape

Benchmark prompt target

Turn the camera to the right, then turn back to the starting view.

prompt source: memory_loop_prompt

Focus on heading drift over the 60s loop. The weaker example yaws away from the requested orientation.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9631 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0085 · official

Input image

60s memory_loop · same benchmark case · gap 0.9312

memory-yaw_loop_360-memory_pool_478

Suite: memory_loopScene: photorealistic / outdoor / terrestrial_landscape

Benchmark prompt target

Rotate the camera a full turn in place and come back to the starting view.

prompt source: memory_loop_prompt

Focus on heading drift over the 60s loop. The weaker example yaws away from the requested orientation.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9312 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0000 · official

Input image

60s memory_loop · same benchmark case · gap 0.5906

memory-square_loop-memory_pool_476

Suite: memory_loopScene: stylized / indoor / workspaces

Benchmark prompt target

Walk a closed square around the scene and return to the starting view.

prompt source: memory_loop_prompt

Focus on heading drift over the 60s loop. The weaker example yaws away from the requested orientation.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9346 · official

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.3440 · official