Lots of engineering went into this build, but I finally got a few new things working.

3D scanning:

• LiDAR depth is now filtered and fused with the AR mesh for cleaner, more accurate surfaces.
• Meshes now weld seams, remove bad geometry, preserve corners, and repair small holes.
• Blurry or fast-moving frames are rejected for sharper textures.
• AprilTags are “lock”able now and after a few positions viewed + ray casts on the mesh they cannot move.

UI changes:

• The scanner now shows mapped surfaces, live depth points, coverage, range, tracking quality, and tag-lock progress.
• Photos and 3D scans are now both “Areas”.
• The 3D viewer looks nicer and has highlighting + better camera controls now.
• Photo areas can rerun AprilTag detection without being deleted and recreated.

AprilTag Metal backend: I looked at some NVIDIA shaders and the C++ code and built a Metal backend for AprilTag detection. This runs on the GPU and gets a way higher fps for me. This reduces lag when scanning around as well as 3D scanning (makes the position more accurate to the mesh). Concurrency is also possible. Fallback to CPU.