LocHeal Verdict 2B (kinasr/locheal-verdict-2b) 🩹🎯

LocHeal Verdict 2B is a specialized System 1 Typed-Decision Vision-Language Model fine-tuned for Automated Self-Healing Web Locators & Multimodal DOM Element Grounding in Playwright/Selenium test suites.

Built on top of LiquidAI/d1-2B (powered by the LFM3-VL-2B tri-hybrid architecture), LocHeal Verdict 2B takes a Set-of-Marks (SoM) annotated screenshot alongside broken locator telemetry (broken_locator, variable_name, intended_action, and optional description) and scores live DOM candidate elements in a single forward pass—returning temperature-calibrated probabilities or explicitly abstaining (Unknown) when the target element has been genuinely removed or blocked from the UI.

---

✨ Why LocHeal Verdict 2B?

Traditional heuristic self-healing tools rely solely on brittle DOM attribute similarity (failing when a frontend refactor changes tags, CSS classes, and DOM hierarchy simultaneously) and suffer from dangerous False Heals (clicking the wrong button when the target element was actually deleted). Conversely, generative LLMs hallucinate invalid XPath/CSS selectors and add multi-second latency.

LocHeal Verdict 2B solves both problems through Candidate Selection + Visual Grounding:

• ⚡ Zero Selector Hallucination (Candidate Selection Paradigm): Instead of generating fragile XPath/CSS strings token-by-token, Playwright extracts the live interactive DOM elements [0..N], overlays numeric Set-of-Marks badges on the screenshot, and locheal-verdict-2b selects the exact candidate index in a single forward pass via SlotReader. Your automation framework then synthesizes a 100% valid locator directly from the chosen live DOM node.
• 🖼️ Visual + Semantic Fusion: Even if a button's HTML changes from <button id="checkout-btn"> to <div role="button" class="tw-cta-92">Pay Now</div>, its visual position, surrounding context, and semantic intent remain recognizable in the Set-of-Marks screenshot.
• 🛡️ Explicit Abstention (Unknown Gate to Prevent False Heals): Trained on genuine element-deletion and modal/loading blockage scenarios so that if a developer removes a feature or breaks a page, the model selects Unknown (Do NOT heal) with 0.0% False-Heal Rate rather than clicking a nearby distractor button.
• 🔧 Works With or Without Human Descriptions: Trained with a 50/50 curriculum split—operating at 99.4%–100.0% accuracy whether given an explicit description or inferring intent purely from broken_locator, variable_name, and intended_action.

---

📊 Benchmark & Evaluation Results

Evaluated on the held-out Out-of-Distribution (locheal-ood-v1-benchmark, 360 samples) benchmark across 6 categories (300 healable frontend refactorings + 60 unhealable Unknown states) under matched evaluation settings (4 cyclic option rotations, MPS float16, native SlotReader temperature calibration, dataset SHA-256 a418556dcb4ab86f...):

1. Overall Performance, Safety & Calibration

Model	Overall Top-1 (360)	Heal Top-1 (300)	Heal Top-2 (300)	Unknown Recall (60)	False-Heal Rate ↓	Safe Action ↑	ECE ↓	Brier ↓
kinasr/locheal-verdict-2b (Ours)	99.7% (359/360)	99.7%	100.0%	100.0%	0.0%	100.0%	0.4%	0.0068
heuristic-dom-matcher (Rule-Based)	93.3%	98.7%	99.7%	66.7%	33.3%	93.3%	5.4%	0.0959
LiquidAI/d1-2B (Base Zero-Shot)	92.2%	91.7%	98.7%	95.0%	5.0%	98.9%	17.0%	0.1703
LiquidAI/LFM3-VL-2B (Zero-Shot)	76.1%	79.0%	89.0%	61.7%	38.3%	84.7%	9.2%	0.3472

Key Takeaways:

• Near-Perfect OOD Accuracy (99.7%): locheal-verdict-2b solved 359 out of 360 unseen OOD samples on its first choice (Top-1 = 99.7%) and 100.0% within Top-2, rescuing 27 samples missed by base LiquidAI/d1-2B with 0 regressions.
• Eliminating the Heuristic "False-Heal" Trap (0.0% vs. 33.3%): Traditional DOM similarity matchers (heuristic-dom-matcher) suffer from a 33.3% False-Heal Rate (scoring only 26.7% when a target button is genuinely deleted, dangerously clicking a wrong distractor button in 73.3% of deleted-element cases). locheal-verdict-2b achieves 100.0% Unknown Recall and 0.0% False-Heal Rate.
• Razor-Sharp Calibration: Post-training temperature calibration reduced Expected Calibration Error (ECE) from 17.0% in base d1-2B down to 0.4% (Brier = 0.0068).

2. Per-Category & Unknown Subtype Accuracy

Category / Subtype	Rows	kinasr/locheal-verdict-2b	LiquidAI/d1-2B	heuristic-dom-matcher	LiquidAI/LFM3-VL-2B
id_testid_refactor	60	100.0%	91.7%	100.0%	78.3%
css_dom_restructure	60	100.0%	93.3%	100.0%	73.3%
copy_and_i18n_drift	60	98.3%	83.3%	93.3%	78.3%
disambiguated_repeated_control	60	100.0%	90.0%	100.0%	81.7%
form_field_relayout	60	100.0%	100.0%	100.0%	83.3%
Unknown (All Unhealable States)	60	100.0%	95.0%	66.7%	61.7%
↳ target_deleted	15	100.0%	93.3%	26.7%	26.7%
↳ loading_skeleton_state	15	100.0%	93.3%	53.3%	53.3%
↳ modal_or_consent_blocked	15	100.0%	100.0%	100.0%	73.3%
↳ wrong_route_or_error_page	15	100.0%	93.3%	86.7%	93.3%

3. Ablation: With vs. Without Semantic description

Model	With description (180 rows)	Without description (180 rows)	Delta
kinasr/locheal-verdict-2b (Ours)	99.4%	100.0%	+0.6% (Zero degradation)
LiquidAI/d1-2B (Base Zero-Shot)	95.0%	89.4%	-5.6%
LiquidAI/LFM3-VL-2B (Zero-Shot)	89.4%	62.8%	-26.6%
heuristic-dom-matcher	92.2%	94.4%	+2.2%

4. CI/CD Confidence Gating & Latency

Model	Threshold $\ge$	Auto-Heals	Coverage (All 360)	Accuracy Among Auto-Heals	4-Rotation Mean Latency*	Peak RAM / VRAM (MiB)
kinasr/locheal-verdict-2b	0.75	298 / 300	82.8%	100.0%	3030.1 ms (~757 ms / pass)	3037 / 5534
kinasr/locheal-verdict-2b	0.90	297 / 300	82.5%	100.0%	3030.1 ms (~757 ms / pass)	3037 / 5534
LiquidAI/d1-2B	0.90	48 / 300	13.3%	100.0%	3031.8 ms	5642 / 5534
heuristic-dom-matcher	0.90	242 / 300	67.2%	99.6%	8.6 ms	288 / —

*Note: Maximum theoretical auto-heal coverage is 83.3% (300/360), as 60/360 rows (16.7%) are genuine Unknown abstentions where healing must not occur. At $\ge 0.90$ confidence, locheal-verdict-2b automatically heals 297 of the 300 healable rows with 100.0% precision.

---

🏗️ Model Architecture & Repository Structure

Property	Specification
Base Architecture	LiquidAI/d1-2B (LFM3-VL-2B Tri-Hybrid: 13 Gated Short Conv + 5 Sliding Window + 6 Global Attention layers)
Parameters	1.92B
Vision Encoder	164M Native-Resolution Vision Tower (optimized at 896x896 max pixel budget for crisp Set-of-Marks badge legibility)
Fine-Tuning Method	Merged LoRA (r=16, lora_alpha=32) over unfused language projections (q_proj, k_proj, v_proj, out_proj, in_proj, gate_proj, w1, w2, w3)
Calibration	Post-trained LBFGS temperature scaling (calibration.json) across choice:low, choice:mid, and choice:high candidate list buckets
Weights Format	Standalone merged model.safetensors (bfloat16) + original LoRA adapter backup in ./lora_adapter/

---

🚀 Quickstart & Inference

1. Installation

pip install -U "transformers>=5.19.0" accelerate safetensors pillow torch torchvision

2. Running Self-Healing Locator Resolution in Python

import os
import sys
import json
import torch
from PIL import Image
from transformers import AutoModel, AutoProcessor

# Keep Set-of-Marks (SoM) numeric badges sharp
os.environ["D1_VISION_MAX_PIXELS"] = str(896 * 896)

MODEL_ID = "kinasr/locheal-verdict-2b"
DEVICE = "cuda" if torch.cuda.is_available() else ("mps" if torch.backends.mps.is_available() else "cpu")
DTYPE = torch.float16 if DEVICE != "cpu" else torch.float32

# 1. Load processor and standalone fine-tuned model
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModel.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
    dtype=DTYPE,
).to(DEVICE).eval()

# 2. Initialize the d1 SlotReader decision engine
pkg_name = model.__class__.__module__.rsplit(".", 1)[0]
slots_mod = sys.modules[f"{pkg_name}.slots"]
engine = slots_mod.SlotReader(model, processor)

# 3. Load Set-of-Marks annotated screenshot + Broken Locator Telemetry
som_screenshot = Image.open("current_page_som.png").convert("RGB")

broken_locator_state = {
    "route": "/checkout/payment",
    "viewport": "1440x900",
    "broken_locator": "button[data-testid='submit-order-cta']",
    "variable_name": "placeOrderBtn",
    "intended_action": "click",
    "description": "Primary CTA button to confirm and pay for the order in the checkout summary" # Optional
}

# Candidate interactive elements extracted from the live DOM + explicit Unknown option
candidates = [
    "[0] <a role='link' text='Return to Cart' section='Checkout Header' bbox=(64,42,130,32)>",
    "[1] <input type='text' placeholder='Promo code' section='Order Summary' bbox=(920,280,240,40)>",
    "[2] <button role='button' text='Apply' section='Order Summary' bbox=(1170,280,80,40)>",
    "[3] <button role='button' text='Pay & Complete Order' aria='Confirm order' section='Order Summary' bbox=(920,460,330,48)>",
    "[4] <button role='button' text='Cancel Order' section='Order Summary' bbox=(920,520,330,40)>",
    "Unknown / Element removed or missing from the current page (Do NOT heal)"
]

criteria = {f"opt_{i}": text for i, text in enumerate(candidates)}
question = {
    "type": "choice",
    "instructions": "Identify the correct replacement interactive element from the Set-of-Marks screenshot and DOM candidates that matches the broken locator intent, or select Unknown if the target element was removed.",
    "criteria": criteria
}

# 4. Single forward-pass healing decision
result = engine.decide(
    state=broken_locator_state,
    questions={"healed_candidate": question},
    images=[som_screenshot]
)

print(json.dumps(result, indent=2))

---

🛠️ Recommended Production Pipeline (Playwright Self-Healing Hook)

1. Catch TimeoutError on Locator Action: When a Playwright locator fails (click, fill, check), trigger the self-healing interceptor.
2. Extract Filtered DOM Candidates & Render SoM Badges: Query visible interactive elements (button, a, input, select, textarea, [role=button], [role=tab]), draw numbered bounding-box badges [0..N] on a screenshot copy, and build compact one-line DOM candidate descriptors.
3. Single Forward-Pass Verdict: Pass the SoM screenshot, broken_locator_state, and candidate list to kinasr/locheal-verdict-2b.
4. Confidence & Abstention Gate:
   • If the model predicts Unknown or confidence is < 0.75, fail the test immediately and report that the target element is missing or blocked on the UI.
   • If confidence is >= 0.75, resolve the selected candidate index back to its live DOM handle, generate a resilient Playwright locator (getByRole / data-testid / CSS), log a self-healing warning for the PR, and continue the test execution.

---

🔗 Related Models in the Verdict QA Suite

• kinasr/visual-verdict-2b — System 1 Typed-Decision VLM (LiquidAI/d1-2B fine-tune) for Automated Web Visual Regression Testing & 6-Class UI Triage.
• kinasr/imajev-2b-web-visual-testing — Qwen3.5-2B / Imajev-based visual regression decision model.

📜 License

This model inherits the LFM Open License v1.0 from its base model LiquidAI/d1-2B.