Board access: [N2]. Tool reference: [Q1].

1. Lab PC: Meta's evaluation

Needs git-lfs and conda; passes when load_eval.py prints Mean keypoint error 9.3 to 9.5 mm for known skeleton, separate hands [U2].

GIT_LFS_SKIP_SMUDGE=1 git clone --recurse-submodules https://github.com/facebookresearch/UmeTrack
cd UmeTrack/UmeTrack_data && git lfs pull --include "raw_data/real/*/testing/**" && cd ..
conda create --name umetrack python=3.9.12 && conda activate umetrack
pip install av numpy scipy opencv-python "git+https://github.com/facebookresearch/pytorch3d.git@stable"
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
python run_eval_known_skeleton.py
python load_eval.py

2. RB5 CPU: the encoder on the board

Install QAIRT 2.33 (it contains SNPE, Qualcomm's NN SDK; its release notes cover QRB5165.LE) from Qualcomm's package manager, copy its aarch64 runtime libraries and Hexagon DSP skel libraries to the board, and set the library paths the SNPE setup guide names.

# PC: export only the image encoder to encoder.onnx (one 96 x 96 crop in, 72 x 6 x 6 features out)
snpe-onnx-to-dlc --input_network encoder.onnx --output_path encoder.dlc
# PC: at the _image_backbone(...) call in lib/models/umetrack_model.py, dump the crops of 100 frames
#     as float32 .raw files listed in input_list.txt; the board's features go back in at the same call
# board, after copying encoder.dlc, input_list.txt and the .raw files over:
snpe-net-run --container encoder.dlc --input_list input_list.txt --profiling_level basic
snpe-diagview --input_log output/SNPEDiag_0.log
# ms per crop = median over the 100 crops, first (warm-up) run excluded; time PyTorch on the PC the same way
# pass: board and PC errors differ by under 0.1 mm on the same 100 frames

3. RB5 DSP, then GPU

Qualcomm's notes say the DSP's tensor block (HTA) is not offered on QRB5165 Linux, so expect the plain DSP runtime; the validator settles it [Q2].

# board: which of GPU and DSP this Linux image reaches
snpe-platform-validator --runtime all --testRuntime
# PC: int8, with the same 100 crops as calibration
snpe-dlc-quantize --input_dlc encoder.dlc --input_list input_list.txt --output_dlc encoder_int8.dlc
# board: time each as in step 2 and note the error change; the GPU computes in float, so it gets the float file
snpe-net-run --container encoder_int8.dlc --input_list input_list.txt --use_dsp --profiling_level basic
snpe-net-run --container encoder.dlc --input_list input_list.txt --use_gpu --profiling_level basic

Sources