uqer1244/coreML_lingbot-map

Apple CoreML LingBot-MAP Model Packages

This repository hosts converted Apple CoreML .mlpackage model bundles for LingBot-MAP / Geometric Context Transformer (GCT), optimized for real-time 3D spatial mapping, camera pose estimation, and metric depth prediction on macOS and iOS (Apple Silicon M1/M2/M3/M4 & Apple Neural Engine).

---

🔗 Links & Resources

• GitHub Repository: uqer1244/coreML_lingbot-map
• Hugging Face Repository: uqer1244/coreML_lingbot-map
• MLX Repository: uqer1244/mlx_lingbot-map
• Original Model Weights: robbyant/lingbot-map

---

📦 Hosted CoreML Model Packages

Package Name	Input Spec	Output Spec	Description
Aggregator.mlpackage	images: [1, 2, 3, 518, 378]	t0..t3: [1, 2, 1005, 2048]	Feature trunk backbone for spatio-temporal embeddings
StatefulCameraHead.mlpackage	t0..t3, k_cache_in, v_cache_in, frame_idx_in	pred_pose_enc, k_cache_out, v_cache_out, frame_idx_out	Stateful KV Cache camera pose head (Rank-3 state tensors)
DepthHead.mlpackage	t0..t3, images	depth: [1, 2, 518, 378, 1], confidence	Metric depth map ($518 \times 378$) & confidence estimator
Encoder.mlpackage	images: [1, 2, 3, 518, 378]	tokens: [1, 2, 1005, 1024]	ViT patch embedder (Integrated inside Aggregator)

---

🛠️ Technical Specifications

• Preprocessing: Aspect-preserving portrait center-crop ($518 \times 378$) preserving $1:1$ physical pixel geometry.
• State Management: Rank-3 KV Cache state tensors ($256 \times 64 \times 128$) with dynamic frame index tracking.
• Pose Transformation: Closed-form SE3 matrix inversion ($W2C \rightarrow C2W$) and cumulative camera transformation ($C2W_k = C2W_{k-1} \times \Delta C2W_k$).

---

💻 Download & Usage Guide

1. Download via Hugging Face Hub (Python)

pip install huggingface_hub

from huggingface_hub import snapshot_download

# Download all CoreML model packages to local models/ directory
snapshot_download(repo_id="uqer1244/coreML_lingbot-map", local_dir="./models")

2. Standalone Python MVP Runner

Clone the GitHub repository and run inference:

git clone https://github.com/uqer1244/coreML_lingbot-map.git
cd coreML_lingbot-map/LingBot-CoreML-MVP

pip install coremltools numpy opencv-python matplotlib viser
python3 run_coreml_mvp.py

3. Swift Native Xcode Integration

Add LingBot-CoreML-Runtime as a Swift Package Manager dependency in Xcode:

import LingBotCoreMLRuntime

let runtime = StreamingRuntime(config: .default)
try runtime.loadModels(from: modelDirectoryURL)

---

📄 License

This repository is licensed under the Apache 2.0 License.