Reprojection Error

reprojection_error measures whether a prediction video is geometrically self-consistent under the wrapped DROID-SLAM reconstruction path [Teed and Deng, 2021]. It requires only the prediction itself—no reference video and no camera ground truth. The core question is whether camera poses and dense depth recovered from the generated video can reproject scene points close to their observed pixel locations in other frames.

Unlike sparse reconstruction methods such as COLMAP [Schönberger and Frahm, 2016, Schönberger et al., 2016], which rely mainly on a small set of high-quality feature matches, DROID-SLAM follows a dense SLAM path. Its differentiable Dense Bundle Adjustment (DBA) layer jointly refines camera poses and per-pixel depth while keeping them consistent with optical-flow correspondences. Geometric failures in generated video often appear not only near a few matched points, but also as background warping, local object drift, unstable depth layering, or cross-frame structures that fail to close. Dense DBA uses as many valid pixels and co-visible points as possible, making the metric closer to “can the full rollout be explained by one 3D reconstruction?”

After DBA refinement, reprojection error over the valid co-visible set $\mathcal{V}$ is

e_{\mathrm{reproj}} = \frac{1}{|\mathcal{V}|} \sum_{(i,j)\in\mathcal{V}} \left\lVert p^{*}_{ij} - \Pi(P_{ij}) \right\rVert_2 \tag{1}

where $p^{*}_{ij}$ is the observed pixel location in frame $i$ , $P_{ij}$ is the 3D point recovered from pose and depth, and $\Pi(\cdot)$ is the projection operator. If the generated video is 3D-consistent, the same scene point should reproject near its observation in other frames; geometric drift, depth instability, or cross-frame inconsistency increases this distance.

In WorldAtlas, the runtime decodes the full prediction into a frame sequence, runs DROID-SLAM to estimate trajectory and internal geometry, and sets the raw score to the mean of accepted reprojection errors, i.e. $e_{\mathrm{reproj}}$ in Equation (1). WorldAtlas reverse-normalizes on the empirical range $[0, 1.0719]$ to $[0, 1]$ (higher normalized score is better). If SLAM reconstruction fails or no valid score is produced, it returns raw = null and normalized = null. When DROID-SLAM succeeds, the metric is official in all formal suites and enters the consistency_3d_4d aggregate. It is a useful geometry-consistency signal, but it does not replace geometry ground-truth metrics such as camera error, and fixed-intrinsics SLAM assumptions warrant cautious interpretation across heterogeneous data.

Verified Same-Input Examples

Audit manifest: docs/assets/metric_examples/curated_same_case_manifest.json.

Input verified · same benchmark case · gap 0.9325

`image_static_photorealistic_public_spaces_0149`

Suite: image_staticScene: photorealistic / indoor / public_spaces

Benchmark prompt target

Camera push in. Public spaces with dining table, chairs, and stainless steel countertop.

prompt source: prompt_target

Both first frames passed geometric matching against the displayed input; labels compare this metric only.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9504Mean reprojection error 0.053 px

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0179Mean reprojection error 1.053 px

Input verified · same benchmark case · gap 0.8875

`image_static_photorealistic_public_spaces_0154`

Suite: image_staticScene: photorealistic / indoor / public_spaces

Benchmark prompt target

Camera move right. A gym interior with cardio machines, water cooler, and exercise mats.

prompt source: prompt_target

Both first frames passed geometric matching against the displayed input; labels compare this metric only.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9021Mean reprojection error 0.105 px

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0146Mean reprojection error 1.056 px

Input verified · same benchmark case · gap 0.9211

`image_static_stylized_workspaces_0105`

Suite: image_staticScene: stylized / indoor / workspaces

Benchmark prompt target

Camera pull out. A modern workspace with desk, office chair, and large windows. Camera pan right. A modern workspace with wall shelves, potted plant, and large windows. Camera move right. A modern workspace with meeting table, whiteboard, and large windows.

prompt source: prompt_target

Both first frames passed geometric matching against the displayed input; labels compare this metric only.

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Higher metric scoreScore 0.9308Mean reprojection error 0.074 px

Model result videos will be available with the ModelScope release. / 模型推理视频将在 ModelScope 方案上线后提供。

Lower metric scoreScore 0.0096Mean reprojection error 1.062 px

References

BibTeX

@article{teed2021droid,
    title = {Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras},
    author = {Teed, Zachary and Deng, Jia},
    journal = {Advances in neural information processing systems},
    volume = {34},
    pages = {16558--16569},
    year = {2021},
}