amolharsh's picture
Add initial model card
1ab7a72 verified
|
Raw History Blame Contribute Delete
1.61 kB
metadata
license: apache-2.0
library_name: pytorch
pipeline_tag: image-text-to-text
tags:
  - 3d
  - point-cloud
  - multimodal
  - segmentation
  - grounding
  - qwen3-vl
base_model: Qwen/Qwen3-VL-4B-Instruct
datasets:
  - amolharsh/Ground3D_Dataset

Ground3D-LMM-4B (Joint)

Ground3D-LMM is a large multimodal model for fine-grained 3D point grounding and spatial reasoning in indoor scenes. It produces per-point segmentation masks alongside grounded natural-language answers across 8 sub-tasks on ScanNet and ScanNet++.

This checkpoint is the Joint variant: 3D point cloud + aligned RGB frames. Main paper result — best performance.

  • Backbone: Qwen3-VL-4B (4.47B params, ~400M LoRA-trainable)
  • 3D encoder: SpConvUNet (from UniSeg3D)
  • Decoder: Mask2Former-style with 2000 learned queries
  • Training data: Ground3D Dataset (2.47M Q/A pairs)
  • Code: https://github.com/amolharsh/Ground3D-LMM
  • File: pytorch_model.pth (full PyTorch state dict, ~11.7 GB)

Quick start

from huggingface_hub import snapshot_download
ckpt_dir = snapshot_download(repo_id="amolharsh/Ground3D-LMM-4B-Joint")
# checkpoint at: f"{ckpt_dir}/pytorch_model.pth"
CKPT="$(huggingface-cli download amolharsh/Ground3D-LMM-4B-Joint --quiet)/pytorch_model.pth"
PYTHONPATH=. python tools/test.py configs/ground3dlmm_eval_ground3d_scannet.py "$CKPT" --work-dir work_dirs/quickstart

See the GitHub README for full inference and training instructions.

License

Apache-2.0