--- license: apache-2.0 library_name: pytorch pipeline_tag: image-text-to-text tags: - 3d - point-cloud - multimodal - segmentation - grounding - qwen3-vl base_model: Qwen/Qwen3-VL-4B-Instruct datasets: - amolharsh/Ground3D_Dataset --- # Ground3D-LMM-4B (Joint) Ground3D-LMM is a large multimodal model for **fine-grained 3D point grounding** and **spatial reasoning** in indoor scenes. It produces per-point segmentation masks alongside grounded natural-language answers across 8 sub-tasks on ScanNet and ScanNet++. This checkpoint is the **Joint** variant: 3D point cloud + aligned RGB frames. Main paper result — best performance. - **Backbone**: Qwen3-VL-4B (4.47B params, ~400M LoRA-trainable) - **3D encoder**: SpConvUNet (from UniSeg3D) - **Decoder**: Mask2Former-style with 2000 learned queries - **Training data**: [Ground3D Dataset](https://huggingface.co/datasets/amolharsh/Ground3D_Dataset) (2.47M Q/A pairs) - **Code**: https://github.com/amolharsh/Ground3D-LMM - **File**: `pytorch_model.pth` (full PyTorch state dict, ~11.7 GB) ## Quick start ```python from huggingface_hub import snapshot_download ckpt_dir = snapshot_download(repo_id="amolharsh/Ground3D-LMM-4B-Joint") # checkpoint at: f"{ckpt_dir}/pytorch_model.pth" ``` ```bash CKPT="$(huggingface-cli download amolharsh/Ground3D-LMM-4B-Joint --quiet)/pytorch_model.pth" PYTHONPATH=. python tools/test.py configs/ground3dlmm_eval_ground3d_scannet.py "$CKPT" --work-dir work_dirs/quickstart ``` See the [GitHub README](https://github.com/amolharsh/Ground3D-LMM) for full inference and training instructions. ## License Apache-2.0