Measures duration-weighted set F1 between recovered and commanded atomic actions. It penalizes both missing and extra actions. For prediction P i P_i P i G i G_i G i w i w_i w i
F 1 ( P i , G i ) = 2 ∣ P i ∩ G i ∣ ∣ P i ∣ + ∣ G i ∣ (1) \mathrm{F1}(P_i, G_i)
=
\frac{2\, |P_i \cap G_i|}{|P_i| + |G_i|}
\tag{1} F1 ( P i , G i ) = ∣ P i ∣ + ∣ G i ∣ 2 ∣ P i ∩ G i ∣ ( 1 ) p a r t i a l = ∑ i w i ⋅ F 1 ( P i , G i ) ∑ i w i (2) \mathrm{partial}
=
\frac{\sum_i w_i \cdot \mathrm{F1}(P_i, G_i)}{\sum_i w_i}
\tag{2} partial = ∑ i w i ∑ i w i ⋅ F1 ( P i , G i ) ( 2 ) Two empty sets have F 1 = 1 \mathrm{F1} = 1 F1 = 1 F 1 \mathrm{F1} F1 1 / 3 1/3 1/3 1 1 1
Uses the same fixed-rate decoder, native timing and coverage rules as strict accuracy . This official action-control metric measures label agreement; it does not independently measure memory or calibrated motion amplitude. Backend: pose_action.
Audit manifest: docs/assets/metric_examples/longseq_memory_60s_examples_20260909.json.
Input image
60s memory_loop · same benchmark case · gap 0.9781
memory-dolly_loop-memory_pool_238Suite: memory_loop Scene: stylized / outdoor / suburb
Benchmark prompt target
Move the camera forward through the scene, then back to where it started.
prompt source: memory_loop_prompt
Partial credit still counts overlapping keys. The better 60s example follows the requested direction; the weaker one substitutes a different move.
Higher metric score Score 0.9781 · official
Lower metric score Score 0.0000 · official
Input image
60s memory_loop · same benchmark case · gap 0.9174
memory-yaw_loop_180-memory_pool_439Suite: memory_loop Scene: stylized / outdoor / aquatic_landscape
Benchmark prompt target
Turn the camera to the right, then turn back to the starting view.
prompt source: memory_loop_prompt
Partial credit still counts overlapping keys. The better 60s example follows the requested direction; the weaker one substitutes a different move.
Higher metric score Score 0.9441 · official
Lower metric score Score 0.0267 · official
Input image
60s memory_loop · same benchmark case · gap 0.9161
memory-yaw_loop_360-memory_pool_132Suite: memory_loop Scene: stylized / outdoor / terrestrial_landscape
Benchmark prompt target
Rotate the camera a full turn in place and come back to the starting view.
prompt source: memory_loop_prompt
Partial credit still counts overlapping keys. The better 60s example follows the requested direction; the weaker one substitutes a different move.
Higher metric score Score 0.9283 · official
Lower metric score Score 0.0122 · official
Input image
60s memory_loop · same benchmark case · gap 0.4901
memory-double_yaw_loop-memory_pool_260Suite: memory_loop Scene: stylized / indoor / living_spaces
Benchmark prompt target
Turn away and back to the start twice in a row.
prompt source: memory_loop_prompt
Partial credit still counts overlapping keys. The better 60s example follows the requested direction; the weaker one substitutes a different move.
Higher metric score Score 0.7072 · official
Lower metric score Score 0.2171 · official
Input image
60s memory_loop · same benchmark case · gap 0.3753
memory-dolly_loop-memory_pool_463Suite: memory_loop Scene: photorealistic / outdoor / terrestrial_landscape
Benchmark prompt target
Move the camera forward through the scene, then back to where it started.
prompt source: memory_loop_prompt
Partial credit still counts overlapping keys. The better 60s example follows the requested direction; the weaker one substitutes a different move.
Higher metric score Score 0.8285 · official
Lower metric score Score 0.4531 · official
