README.md

---
title: Prometheus Prototype
emoji: 🔥
colorFrom: yellow
colorTo: gray
sdk: docker
app_port: 8080
pinned: false
short_description: launching my prometheus, aerial wildlife image detection pro
---
# Prometheus — Aerial Wildlife Intelligence

Automated wildlife detection and population counting from aerial drone footage, built for the **Malilangwe Wildlife Trust** — a conservation organisation managing one of Zimbabwe's most biodiverse reserves (~98,000 acres, southeastern Zimbabwe).

**Prometheus** processes drone and helicopter survey footage using YOLOv11 object detection and BoT-SORT multi-object tracking, surfacing species counts, herd locations, and anomaly flags through an interactive web dashboard.

---

## The Problem

Wildlife population monitoring across large terrain is costly, slow, and error-prone when done manually. Rangers walk transects or scan footage by hand — a process that takes days and introduces counting errors. Malilangwe monitors 19 priority species across terrain that is difficult to survey on foot.

## The Solution

An end-to-end ML pipeline that processes aerial imagery to **detect**, **classify**, and **count** wildlife in real time.

| | Pretrained (COCO) | Phase A (WAID) | Phase A+ (Merged) |
|---|---|---|---|
| Aerial elephants | "sheep", "bird" | not in schema | ✓ elephant |
| Aerial buffalo | "horse", "cow" | not in schema | ✓ buffalo |
| Aerial zebra | "horse" | ✓ zebra | ✓ zebra |
| mAP50 | ~0.05 | **0.956** (yolo11s — up from 0.918 with yolo11n; see comparison below) | 0.317 and climbing |

---

## Project Structure

```
wildlife-detector-malilangwe/
├── app.py                        # Prometheus Streamlit dashboard (working app — run this)
├── dashboard_mockup.html         # Static UI design prototype (Leaflet.js, not connected to model)
├── api/                          # FastAPI service backing the React dashboard (reuses src/detection)
│   ├── main.py                   # App, CORS, /api/health
│   ├── routes/                   # detection + catalog (models/classes/capabilities/metrics)
│   ├── services/                 # detection_service — thin layer over Detector
│   └── data/                     # capabilities + training-run history (JSON)
├── dashboard/                    # React + TS + Tailwind console (Vite) — see dashboard/README.md
├── config/
│   ├── default.yaml              # Master config (paths, model, training, tracking)
│   └── merged_classes.yaml       # Unified 7-class schema + per-dataset remappings
├── data/
│   ├── waid.yaml                 # Ultralytics dataset YAML (WAID only)
│   └── merged.yaml               # Ultralytics dataset YAML (multi-dataset, generated)
├── src/
│   ├── config.py                 # YAML config loader with dot-access & deep merge
│   ├── detection/
│   │   └── detector.py           # YOLOv11 wrapper with config-driven inference
│   ├── tracking/
│   │   └── tracker.py            # BoT-SORT multi-object tracking interface
│   ├── data/
│   │   ├── dataset.py            # WAID validation & YAML generation
│   │   ├── merge.py              # Class remapping & multi-dataset merge utilities
│   │   └── convert_aed.py        # AED point-annotation CSV → YOLO format converter
│   └── utils/
│       ├── logging_setup.py      # Structured logging (file + stdout)
│       └── visualization.py      # Bounding box & summary overlay drawing
├── scripts/
│   ├── detect.py                 # CLI detection runner
│   ├── train.py                  # Fine-tuning script (WAID or merged dataset)
│   ├── evaluate.py               # Evaluation — mAP, precision, recall
│   ├── merge_datasets.py         # Merge multiple datasets into unified format
│   └── prepare_datasets.py       # Download + structure guide for all datasets
├── weights/                      # Trained model weights (not tracked — too large)
├── WAID/                         # WAID labels (images not tracked)
├── requirements.txt
├── pyproject.toml
├── STATUS.md                     # Current training state and roadmap
└── BACKLOG.md                    # Future scope outside current MVP
```

---

## Quick Start

### Prerequisites

- Python 3.10+
- GPU recommended for training (Google Colab T4 works well)

### Installation

```bash
git clone https://github.com/Tadiwa-M/wildlife-detector-malilangwe.git
cd wildlife-detector-malilangwe
pip install -r requirements.txt
```

### Launch Dashboard

```bash
streamlit run app.py
```

Opens at `http://localhost:8501`. Upload an aerial image or video clip, select weights, and run detection. Supports drag-and-drop, side-by-side original vs annotated view, density heatmap, species count cards, and download of annotated results.

### Run Detection (CLI)

```bash
# Single image
python scripts/detect.py --source path/to/image.jpg --show

# Video file
yolo predict model=weights/best.pt source=path/to/clip.mp4 conf=0.20 save=True

# Save annotated output
python scripts/detect.py --source path/to/image.jpg --save --conf 0.20
```

### Multi-Dataset Training (Phase A+)

```bash
# 1. See download instructions for each dataset
python scripts/prepare_datasets.py

# 2. Merge datasets into unified format
python scripts/merge_datasets.py \
    --waid WAID/WAID \
    --aed path/to/AED \
    --liege path/to/liege_yolo

# 3. Train on merged dataset
python scripts/train.py \
    --dataset data/merged.yaml \
    --base-weights weights/best.pt
```

### Train on WAID Only (Phase A)

```bash
git clone https://github.com/xiaohuicui/WAID.git

python scripts/train.py                              # fresh run
python scripts/train.py --resume runs/train/weights/last.pt  # resume
python scripts/train.py --validate-only              # check dataset
```

---

## Datasets

### Unified Class Schema (7 classes)

| ID | Class | Sources |
|----|-------|---------|
| 0 | elephant | AED, Liege |
| 1 | zebra | WAID, Liege |
| 2 | buffalo | Liege |
| 3 | antelope | Liege (kob, topi, waterbuck, alcelaphinae) |
| 4 | cattle | WAID |
| 5 | giraffe | Liege |
| 6 | other | Liege (warthog) |

### Dataset Sources

| Dataset | Species | Images | Format | Status |
|---------|---------|--------|--------|--------|
| [WAID](https://github.com/xiaohuicui/WAID) | sheep, cattle, seal, camelus, kiang, zebra | ~14,000 | YOLO | ✓ merged |
| [AED](https://zenodo.org/record/3234780) | elephant | ~2,100 | CSV point annotations → auto-converted | ✓ merged |
| [Liege African Mammals](https://dataverse.uliege.be) | elephant, zebra, buffalo, kob, topi, warthog, waterbuck | ~1,300 | COCO JSON → auto-converted | ✓ merged |
| [WildlifeMapper](https://data.4tu.nl/articles/dataset/12713903/1) | 20 species incl. lion, giraffe | TBC | YOLO | optional |
| [MMLA](https://arxiv.org/pdf/2504.07744) | 6 species, 37 aerial videos | 811K annotations | YOLO | optional |

> The merge pipeline handles all format conversions automatically — no manual pre-processing needed.

---

## Architecture

- **Detection:** YOLOv11n (2.6M params, 6.4 GFLOPs) — real-time, CPU-deployable
- **Tracking:** BoT-SORT — multi-object tracking for counting individuals across frames
- **Merge pipeline:** Unified class remapping across 5 datasets with different annotation formats
- **Dashboard:** Streamlit — dark earth-tone UI, drag-and-drop upload, density heatmap, species cards
- **Config:** YAML-driven — no hardcoded paths or hyperparameters

---

## Training

Training runs on Google Colab T4 GPU. The one-shot Colab cell handles the full pipeline: extract → convert → merge → train → save to Drive.

### Phase A: yolo11n → yolo11s

Re-running Phase A on a larger model with multi-GPU DDP produced a clear jump in both detection quality and training speed:

| | yolo11n (Colab, 1x T4) | yolo11s (Kaggle, 2x T4 DDP) |
|---|---|---|
| **mAP50** | 0.918 | **0.956** |
| **mAP50-95** | 0.552 | **0.578** |
| Precision / Recall | — | 0.946 / 0.912 |
| Image size | — | 1024 |
| Epochs | 120 | 120 (~45s/epoch — full run in ~90 min) |

> Two changes compounded here: moving to the larger **yolo11s** backbone, and training on **2x T4 with `device=[0,1]`** (Ultralytics defaults to a single GPU otherwise — DDP splits the batch automatically). Together they delivered both a stronger model *and* a dramatically shorter wall-clock time versus the single-GPU Colab runs — worth digging into further for the discussion section of a potential write-up (effect of multi-GPU DDP, and Kaggle vs. Colab as a training platform).
>
> Note: the yolo11s run evaluated on a different validation split (a self-carved 15% holdout from a broader 10-class dataset, 292 images) than the yolo11n run (the canonical WAID val set, 2,873 images) — see the per-run breakdowns and reproducibility note below for the full picture.

**Phase A (WAID baseline):** 120 epochs, yolo11n. Evaluated on WAID validation set (2,873 images, 46,703 instances): **mAP50 = 0.918, mAP50-95 = 0.552**. Per-class mAP50: seal 0.983, cattle 0.964, sheep 0.972, zebra 0.878, kiang 0.881, camelus 0.832.

**Phase A revised (yolo11s, Kaggle T4 x2):** Re-ran the WAID fine-tune on a larger, 10-class aerial-wildlife variant (sheep, cattle, seal, camelus, kiang, zebra, crocodile, elephant, deer, horse), 120 epochs at imgsz=1024, with a self-carved 15% validation holdout (292 images, 3,035 instances — not the original WAID val split, so not a direct apples-to-apples comparison with the yolo11n run above, but a stronger result on its own terms). Final: **mAP50 = 0.956, mAP50-95 = 0.578** (P = 0.946, R = 0.912). Per-class mAP50: sheep 0.978, cattle 0.968, seal 0.935, camelus 0.972, kiang 0.947, zebra 0.956, crocodile 0.928, elephant 0.952, deer 0.971, horse 0.955. This checkpoint (`waid_yolo11s_best.pt`) becomes the starting point for the merged Phase A+ run.

> **Note for reproducibility:** the "10-class aerial-wildlife variant" used for the revised run is a *different* Kaggle dataset from the canonical 6-class WAID (sheep, cattle, seal, camelus, kiang, zebra) referenced in `config/merged_classes.yaml` and used for the original yolo11n baseline above — it happens to share the WAID name and most of its classes, plus four extra ones (crocodile, elephant, deer, horse). Of the canonical WAID's 6 classes, only `cattle` and `zebra` survive into the unified Prometheus schema (the other four — sheep, seal, camelus, kiang — are dropped as not African/not Malilangwe-relevant; see `dataset_mappings.waid` in `config/merged_classes.yaml`). The revised run's checkpoint is used purely as a *warm-start* for Phase A+, not as a merge input, so this naming overlap doesn't affect the merge pipeline — but it's worth knowing which "WAID" produced which numbers when comparing results.

**Phase A+ (multi-dataset):** Transfer learning from Phase A weights on merged 3,300-image dataset (WAID + AED elephant + Liege African mammals). 120 epochs, patience=20, full network fine-tuning (freeze=0). Currently training — best mAP50 reached **0.180** at epoch 22, with losses still declining.

---

## Roadmap

**Phase A — WAID Baseline**
- [x] Project structure, config system, detection module
- [x] CLI scripts: detect, train, evaluate
- [x] Colab training notebook with Drive backup
- [x] 120-epoch WAID training complete

**Phase A+ — Multi-Dataset (current)**
- [x] Unified 7-class schema
- [x] AED point-annotation CSV → YOLO converter
- [x] Multi-dataset merge pipeline (WAID + AED + Liege + WildlifeMapper + MMLA)
- [x] Prometheus Streamlit dashboard with video support
- [x] Interactive map mockup (Leaflet.js, real Malilangwe coordinates)
- [ ] Complete multi-dataset training run
- [ ] SAHI tiled inference for dense herds at altitude

**Phase 1 — Geolocation & Population Estimation**
- [ ] Geolocation layer: project pixel detections to ground coordinates using drone GPS, altitude, and gimbal/attitude data (camera calibration + terrain awareness)
- [ ] Distance-sampling estimation: per-detection perpendicular distance → detection-function fit (half-normal / hazard-rate) → population density estimates with confidence intervals (faithful implementation of Buckland et al.)
- [ ] Population-density mapping across a park over time (built on the geolocation layer)

**Phase 2 — FastAPI Backend**
- [ ] Async video processing (Celery + Redis)
- [ ] Postgres detection result storage
- [ ] REST API: upload, poll, retrieve

**Phase 3 — React Dashboard**
- [ ] React + Tailwind + shadcn/ui
- [ ] Real-time processing status
- [ ] Annotated video playback, population charts, map view

**Phase B — Malilangwe Fine-Tune**
- [ ] Blocked on labelled imagery from Malilangwe Trust (contact made)
- [ ] 19 priority species: elephant, black/white rhino, lion, leopard, buffalo, giraffe, and more

---

## About Malilangwe

The [Malilangwe Trust](https://www.malilangwe.org/) manages the Malilangwe Wildlife Reserve in southeastern Zimbabwe (20°58′–21°15′S, 31°47′–32°01′E) — roughly 98,000 acres (≈396 km²) of protected savanna, woodland, and wetland. The reserve hosts one of Africa's most intact wildlife communities, including black and white rhino, lion, leopard, and large elephant and buffalo herds.

---

## References

- [WAID: A Large-Scale Dataset for Wildlife Detection with Drones](https://www.mdpi.com/2076-3417/13/18/10397) — Cui et al., 2023
- [Aerial Elephant Dataset](https://zenodo.org/record/3234780) — Zenodo, 2019
- [WildlifeMapper: Aerial Image Analysis for Multi-Species Detection](https://arxiv.org/abs/2311.00880) — CVPR 2024
- [Ultralytics YOLOv11](https://docs.ultralytics.com/)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://arxiv.org/abs/2206.14651)

## License

MIT