跳转到内容

Evaluation 环境配置

开源基线只需要一个 Conda 环境,环境名约定为 worldatlas。不要使用旧的 new 环境。

Terminal window
# 已在 CUDA 12.1 上验证
export CUDA_HOME=/path/to/cuda-12.1/
conda create -n worldatlas python=3.10 -y
conda activate worldatlas
Terminal window
python -m pip install \
torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 \
--index-url https://download.pytorch.org/whl/cu121
python -m pip install -e .
conda install -c conda-forge decord=0.6.0 -y
python -m pip install \
transformers==4.46.3 diffusers accelerate pyiqa open3d iopath pycocotools \
pytorch-lightning torchmetrics timm==0.4.12 xformers==0.0.28.post3 \
easydict roma jaxtyping lpips "pyglet<2" e3nn moviepy==1.0.3 \
plyfile pillow-heif pycolmap imageio-ffmpeg OpenEXR kornia gsplat trimesh
python -m pip install \
-f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py310_cu121_pyt251/download.html \
pytorch3d
python -m pip install \
-f https://data.pyg.org/whl/torch-2.5.1+cu121.html \
torch-scatter
python -m pip install causal-conv1d==1.5.0.post8 mamba-ssm==2.2.4
ln -sfn "$(python -c 'import imageio_ffmpeg; print(imageio_ffmpeg.get_ffmpeg_exe())')" \
"$CONDA_PREFIX/bin/ffmpeg"

如果已装好的环境里 conda-forge solver 不能收敛,再使用 PyPI fallback:

Terminal window
python -m pip install decord==0.6.0

多个 headline 指标(camera_error、reprojection_error、motion_accuracy 及相关 flow 指标)会通过 in-tree adapter 复用上游实现。需要完整 official benchmark 路径时,请初始化指标源码 submodule 并编译 DROID-SLAM:

WorldAtlas 只在 worldatlas/benchmark/official_backends/ 中保留指标 adapter。DROID-SLAM、SEA-RAFT、GroundingDINO、SAM2 和 VFIMamba 都会 直接复用 thirdparty/ 下的 submodule checkout。

Terminal window
mkdir -p thirdparty
git submodule update --init --recursive thirdparty/DROID-SLAM
cd thirdparty/DROID-SLAM/
python setup.py install
cd ../..
git submodule update --init thirdparty/GroundingDINO
python -m pip install --no-build-isolation -e thirdparty/GroundingDINO
git submodule update --init thirdparty/sam2
cd thirdparty/sam2/
python -m pip install -e .
cd ../..
git submodule update --init thirdparty/SEA-RAFT thirdparty/VFIMamba

安装其余 native metric runtime(已注册子模块用 submodule update,否则手动 clone 一次):

Terminal window
git submodule update --init \
thirdparty/AMT \
thirdparty/RAFT \
thirdparty/Pulse-of-Motion \
thirdparty/Depth-Anything-3 \
thirdparty/met3r \
thirdparty/vipe \
thirdparty/UniDepth \
thirdparty/Prior-Depth-Anything
# 尚未注册为子模块 — 按需手动 clone
git clone --depth 1 https://github.com/soCzech/TransNetV2 thirdparty/TransNetV2
git -C thirdparty/TransNetV2 lfs pull

官方分布与轨迹 metric:

Terminal window
git clone --depth 1 https://github.com/DSL-Lab/FVMD-frechet-video-motion-distance thirdparty/FVMD-frechet-video-motion-distance
git clone --depth 1 https://github.com/oooolga/JEDi thirdparty/JEDi
git clone --depth 1 https://github.com/google-deepmind/tapnet thirdparty/tapnet
# 将 JAX/CUDA 栈与共享 WorldAtlas 环境隔离。
python -m venv --system-site-packages ../envs/worldatlas-metrics-cu121
METRIC_PYTHON=../envs/worldatlas-metrics-cu121/bin/python
"$METRIC_PYTHON" -m pip install \
--constraint requirements/metrics-cu121.txt \
"jax[cuda12]==0.4.35" flax==0.10.2 prettytable==3.9.0 tensorboardX==2.6.2.2
"$METRIC_PYTHON" -m pip install \
--constraint requirements/metrics-cu121.txt \
-e thirdparty/tapnet
"$METRIC_PYTHON" -m pip install --no-deps vjepa==0.1.2

FVMD 和 JEDi backend 会直接从配置的 repo_root 导入,无需安装它们的上游包。 尤其不要把 FVMD 的完整上游 requirements 装进共享环境,它的旧版严格 pin 会与当前 WorldAtlas 依赖冲突。评测时使用 "$METRIC_PYTHON" scripts/run_eval_shards.py ... 启动。以上约束已在 H100、driver 535 上完成真实计算验证;driver 535 仍可能输出不影响结果的 cuFuncGetName 警告。

fvmd 和 jedi 是分布 metric:分片评测时 worker 只收集样本,完整 merge 后各计算一次。 trajan 是论文中的单视频重建 Average Jaccard,同时用于两个 dynamic suite。

hpsv3_norm 需在 worldatlas 环境中安装 inferencer:

Terminal window
python -m pip install hpsv3

可选 MegaSAM camera_error backend(使用独立 conda 环境):

Terminal window
export MEGASAM_PYTHON=/path/to/mega_sam/bin/python
Terminal window
python -m pip check
python scripts/check_installation.py

当前已配置并验证过的 evaluation 环境使用 decord==0.6.0、pytorch3d==0.7.8、pyiqa==0.1.15.post2、open3d==0.19.0、xformers==0.0.28.post3、torch-scatter==2.1.2+pt25cu121、causal-conv1d==1.5.0.post8、mamba-ssm==2.2.4。

如果从 PyPI 安装后 pip check 只报 decord 0.6.0 is not supported on this platform,通常是 PyPI wheel 元数据仍标成 CPython 3.6;运行时一般可以正常 import。开源环境优先使用 conda-forge decord,否则单独记录这个 decord 元数据问题,并用 python -c "import decord; print(decord.__version__)" 验证。