DESRLab/dynamic-sam

DynamicSAM

Interactive 3D point cloud segmentation with adaptive masking. Given a point cloud and click prompts (foreground/background), DynamicSAM produces a per-point foreground probability mask.

• Input: XYZ-only point cloud (no RGB) + click prompts (point coordinates + foreground/background labels)
• Output: per-point foreground probability, binarized at inference time by a tunable threshold
• Architecture: PointBERT-style point cloud encoder, a prompt encoder that shares its positional-embedding MLP with the point cloud encoder, and a two-way-transformer mask decoder with a PointNet++ upsampling head (adapted from the Segment Anything Model's decoder design, without an IoU-prediction head)

Files

• model.safetensors — inference weights (point cloud encoder, prompt encoder, mask decoder), including group_size/num_group in its metadata.

Training

• Dataset: ScanNet (ScanNet40)
• Config: trans_dim=384, group_size=32, num_group=128, batch size 128, AdamW + multi-step LR decay

Usage

Install dynamic-sam with the inference extra, then serve directly from this repo:

pip install -e ".[inference]"
dynamic-sam serve --hf-repo-id Marali/dynamic-sam

This downloads model.safetensors through the local Hugging Face cache (HF_HOME) and serves it behind a FastAPI HTTP API (/encode_pcd, /predict_mask).

To load the weights directly in Python instead:

from src.serve.checkpoint import load_dynamic_sam

model = load_dynamic_sam("model.safetensors", device=0)