arkodeep/celeb-fbi-weight-resnet50

Research artifact — do not use for medical, fitness, or individual assessment. Trained on celebrity images; under-predicts heavier subjects by 14-15 kg.

Load & predict

git clone https://github.com/shubhodeepcx/weight-from-image && cd weight-from-image
pip install -r requirements.txt
huggingface-cli download arkodeep/celeb-fbi-weight-resnet50 --local-dir ckpts
python -m src.predict --ckpt ckpts/hybrid_resnet50_best.pt --image photo.jpg --height-cm 175

Model Card — Human Weight Prediction from a Full-Body Image

Project: DeLTA26_041 (IIIT Pune research internship) · Task: single-image regression → body weight (kg) · Status: research prototype.

---

1. Model details

• Architecture. ResNet-50 (ImageNet-pretrained) trunk + regression head Linear(2048→256) → ReLU → Dropout(0.3) → Linear(256→1).
• Hybrid variants.
  • Feature fusion (FusionRegressor): CNN 2048-d penultimate features concatenated with a small MLP over a 7-d anthropometric vector → shared head.
  • Late fusion: trained-CNN predictions + an XGBoost regressor on the anthropometric features, combined by a linear meta-learner.
• Anthropometric features (7-d). Six scale-invariant pose features — shoulder-width/torso, hip-width/torso, shoulder/hip ratio, leg-length/torso, arm-length/torso, bbox width/height — plus a pose_valid flag. Extracted with MediaPipe PoseLandmarker (Tasks API, 33-point BlazePose). Normalising by torso length makes them invariant to camera distance (a single image has no absolute scale).
• Training. Huber (Smooth-L1) loss; AdamW with split LRs (head 1e-4, backbone 1e-5, weight-decay 1e-4); cosine schedule; AMP on CUDA; early stopping on validation MAE (patience 7); batch 64; ≤40 epochs; input 224×224 with ImageNet normalisation. Targets standardised (mean 65.8, std 16.0 kg); metrics inverted to kg.
• Frameworks. PyTorch / torchvision, MediaPipe, scikit-learn, XGBoost.

2. Intended use

• Intended. Research and education — studying how far appearance + body-shape cues can estimate weight from one image; coursework / IEEE write-up.
• Out of scope (do not use for). Any medical, clinical, insurance, fitness, employment, or individual assessment. Outputs are point estimates with large per-person error (see §5) and a known directional bias against heavier bodies.

3. Data

• Source. Celeb-FBI (alecccdd/celeb-fbi) — 7,208 full-body celebrity photos.
• Splits. Provided test split (721) held out untouched. Training pool split 90/10 into train 5,032 / val 559, BMI-stratified and seeded (seed 42); the four ablation models share this identical split.
• Labels. weight (kg, target), height (cm, for BMI), gender, age; missing values use a -1 sentinel and are dropped (units confirmed kg/cm in EDA).
• Pose coverage. Landmarks detected on 6,480 / 6,487 train (99.9%) and 721 / 721 test images, so the anthropometric branch is real signal, not imputation.
• Leakage. Per-image id checked across splits — no id shared. Residual risk remains: a celebrity may appear under multiple ids, so subject-level leakage cannot be fully excluded (documented limitation).

4. Metrics (test set, 721 images)

Model	MAE (kg)	RMSE (kg)	Pearson r	R²	BMI-band acc
A — CNN only	8.11	12.51	0.635	0.402	0.626
B — Anthropometric only (XGBoost)	9.73	13.84	0.520	0.267	0.558
A+B — late fusion	8.35	12.22	0.657	0.429	0.614
A+B — feature fusion	8.01	12.49	0.641	0.403	0.624

Primary metric is MAE. Run-to-run variance is ≈ ±0.2 kg (a separate baseline run measured 7.9 kg), so the fusion gains are modest. Late fusion most improves the variance-sensitive metrics (RMSE, r, R²) by trimming large errors; feature fusion gives the best MAE.

5. Error analysis & limitations

Error by BMI band (feature-fusion):

Band	MAE (kg)	n
underweight	6.3	107
normal	6.1	381
overweight	14.1	66
obese	15.4	66

• Dominant failure: heavy subjects are under-predicted. The worst-20 cases are large/muscular men (true 102–109 kg) predicted 70 kg — 30+ kg under. The model is accurate (6 kg) on the well-represented normal/underweight bands and collapses on the rare overweight/obese bands.
• Root cause is dataset bias, not the features. Celebrities skew lean and tall, so the training distribution has few heavy exemplars; the model regresses toward the mean (65.8 kg) for out-of-distribution bodies. Adding anthropometric features helps only marginally because the limitation is coverage, not signal.
• No absolute scale. Weight fundamentally needs size, which a single image lacks; scale-invariant ratios supply shape but not size — hence anthropometric- only is the weakest model (9.7 kg).
• Other risks. Heavy clothing, unusual poses, occlusion, and extreme framing degrade both the CNN and pose features. Gender/age fairness beyond the gender/BMI breakdowns here was not exhaustively audited.

6. Ethical considerations

Inferring body weight from images is sensitive. Predictions could enable body-shaming, discrimination, or unwanted profiling. The model's directional bias means errors are not evenly distributed — it is least reliable exactly for the heavier individuals most vulnerable to such harms. This artifact must not be deployed against real people or used for any individual judgement. Any downstream use should obtain consent, disclose uncertainty, and re-validate on a representative, weight-balanced dataset.

7. Reproduce the evaluation

1. pip install -r requirements.txt
2. Obtain the best checkpoint (*_best.pt, written to the Drive cache during training) and the cached pose-feature parquets.
3. Run notebooks/97_colab_eval.ipynb (or the equivalent src.eval calls) — it regenerates the ablation table, BMI confusion matrices, error-by-group charts, and the worst-k grid into reports/.