1simo/recaptcha-classification-57k-224

reCAPTCHA tile classifier, 224 px (ONNX)

DannyLuna/recaptcha-classification-57k (Ultralytics YOLO11x-cls, trained at 640 px) fine-tuned at a 224 px input, for the Go solver in mmhanda/VisionAIRecaptchaSolver (go-solver/), which downloads this file on first use and checks its SHA-256.

reCAPTCHA tiles are about 100 px, so 640 px inputs upscale each one 6x. At 224 px the model does about 28x the throughput of the 640 px one on the same GPU (RTX 3060 Laptop: 134-141 3x3 grids/s on TensorRT fp16, against 4.8 for the 640 px model on CUDA; about 70 complete solves/s with 1,000 in flight), with click decisions within noise of it on held-out data.

Model

• File: recaptcha_classification_57k_224.onnx, SHA-256 9003763cc591c24882830a472aa31e703f1cbf5cda7c56309c791a55217cd1e5
• Input images: float32 [batch, 3, 224, 224], RGB in [0, 1]; the shortest edge resized to 224 (bilinear), then a 224x224 centre crop, as Ultralytics classification does. Dynamic batch.
• Output output0: softmax over 14 classes, in this order: Bicycle, Bridge, Bus, Car, Chimney, Crosswalk, Hydrant, Motorcycle, Mountain, Other, Palm, Stair, Tractor, Traffic Light.
• The input size is also in the metadata (imgsz); run the model at 224 px. At any other size its accuracy collapses without an error.

Training

From the 640 px weights, on the dataset's full train split at 224 px: AdamW, lr0 3e-4 with a cosine schedule over a fixed 18 epochs, 1 warmup epoch, label smoothing 0.1, batch 64 (1.3 h on an RTX 3060); the last epoch is the model, no checkpoint was picked. The recipe was chosen against the previous one on a leak-free split carved out of the training set (8% of each class's distinct images, whole clusters of near-copies held out, 4,517 images neither recipe saw): static grids answered exactly 69.6% vs 66.2%, tile top-1 94.3% vs 92.8%, background false clicks 1.9% vs 2.4%, targets clicked level. go-solver/train/train.py (devsplit, train, export) reproduces it.

Evaluation

The dataset's validation split leaks: 632 of its 1,474 images are byte-for-byte copies of training images and 89 more are re-encoded copies. These numbers come from the 748 that are not (train.py heldout), and from their 191 tile-sized images in particular, which are what a solver classifies. 95% intervals come from resampling the tiles.

	this model (224 px)	base model (640 px)
top-1, all 748 images	85.7% (641)	86.4% (646)
top-1, tile-sized, background excluded	90.4% [86-95]	88.8% [84-94]
target tiles over 0.7 (a dynamic click)	86.4% [81-92]	86.4% [80-92]
background tiles over 0.7 for a target	5.6% [4.8-6.4]	5.6% [4.7-6.4]
simulated static grids answered exactly	61.6% [55-71]	62.4% [55-72]

Per class the two differ by one to four held-out images, in both directions, which is below what 748 images can resolve. Both share the dataset's weakness: they almost never predict "Other" (background), and about 30% of held-out background tiles score at least 0.7 as Car. The training set's "Other" class has 128 distinct images, and "Other" means "not this challenge's target", so it is noisy. Details in the solver's README, under Accuracy.

License

AGPL-3.0, as Ultralytics licenses models built with its framework (and as the file's own metadata states). The base model and the dataset are published by DannyLuna under MIT.