Update model/dataset card: arXiv link, updated abstract, unified documentation

README.md

@@ -1,55 +1,84 @@
 ---
 library_name: transformers
 license: cc-by-nc-sa-4.0
-datasets:
-- Aletheia-Bench/Aletheia-DPO
+pipeline_tag: text-generation
 base_model:
 - deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
+datasets:
+- INSAIT-Institute/Aletheia-DPO
+tags:
+- code
+- code-verification
+- rlvr
+- dpo
 ---
 
-<font size=3><div align='center' >  
-[[**🤗 Model & Dataset**](https://huggingface.co/Aletheia-Bench)] 
-[[**📊 Code**](https://github.com/insait-institute/aletheia)] 
-[[**📖 Paper**](https://arxiv.org/)]
+<font size=3><div align='center'>
+[[**📖 Paper**](https://arxiv.org/pdf/2601.12186)]
+[[**💻 Code**](https://github.com/insait-institute/aletheia)]
+[[**🤗 Models & Datasets**](https://huggingface.co/collections/INSAIT-Institute/aletheia-6a7c60fab9eadaef73aa427b)]
 </div></font>
 
 # Aletheia: What Makes RLVR For Code Verifiers Tick?
-Multi-domain thinking verifiers trained via Reinforcement Learning from Verifiable Rewards (RLVR) are a prominent fixture of the Large Language Model (LLM) post-training pipeline, owing to their ability to robustly rate and rerank model outputs. However, the adoption of such verifiers towards code generation has been comparatively sparse, with execution feedback constituting the dominant signal. Nonetheless, code verifiers remain valuable toward judging model outputs in scenarios where execution feedback is hard to obtain and are a potentially powerful addition to the code generation post-training toolbox. To this end, we create and open-source Aletheia, a controlled testbed that enables execution-grounded evaluation of code verifiers' robustness across disparate policy models and covariate shifts. We examine components of the RLVR-based verifier training recipe widely credited for its success: (1) intermediate thinking traces, (2) learning from negative samples, and (3) on-policy training. While experiments show the optimality of RLVR, we uncover important opportunities to simplify the recipe. Particularly, despite code verification exhibiting positive training- and inference-time scaling, on-policy learning stands out as the key component at small verifier sizes, and thinking-based training emerges as the most important component at larger scales.
 
+Multi-domain thinking verifiers trained via Reinforcement Learning with Verifiable Rewards (RLVR) are a cornerstone of modern post-training. However, their adoption in code generation has lagged behind that of execution feedback due to the prohibitive costs of the full RLVR pipeline. In this work, we ablate three primary choices along the performance-cost trade-off in RLVR: intermediate thinking traces, learning from negative samples, and on-policy training. We introduce **Aletheia**, a controlled, execution-grounded testbed to facilitate a contamination-free analysis of code verifier training recipes across disparate model sizes and covariate shifts across two common verifier application scenarios. Our analysis reveals that the optimal training recipe is scale-dependent: on-policy learning is the primary performance driver for small verifiers, whereas the thinking budget becomes the most vital factor at larger scales. While leveraging negative samples has a consistent impact on top-1 selection accuracy across sizes, their contribution to ranking reconstruction increases monotonically with scale and plays a key role in stabilizing training at large sizes. Our Pareto optimality analysis demonstrates that eliminating on-policy training at larger model scales yields a verifier that performs comparably to the full RLVR recipe. Furthermore, we find that eschewing thinking traces serves as a compute-efficient strategy at lower budgets, offering a strong trade-off between training cost and verifier accuracy. Ultimately, our work provides the empirical foundation necessary to efficiently deploy robust code verifiers, thereby enabling their wider adoption in post-training pipelines for large code generation models.
+
+## 🤖 This Model
 
+**`DPO-Think-14B`** is a **DPO-Think** verifier: trained with **offline** preference optimization (DPO) on pre-collected thinking traces (no on-policy sampling). It is fine-tuned from [`deepseek-ai/DeepSeek-R1-Distill-Qwen-14B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B) (14B scale) on [`INSAIT-Institute/Aletheia-DPO`](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-DPO).
+
+Given a competitive-programming problem and a set of candidate solutions, the verifier judges and ranks the candidates. See the [GitHub repository](https://github.com/insait-institute/aletheia) for the exact prompt format, plus training and evaluation scripts.
 
 ## 📦 Model Zoo
-Fine-tuned code verifiers at 1.5B, 7B, and 14B scales using several algorithms
+
+Fine-tuned code verifiers at 1.5B, 7B, and 14B scales using several algorithms:
+
 | Algorithm | Thinking | Negatives | Online | Description |
 | :--- | :---: | :---: | :---: | :--- |
-| **GRPO-Think** | ✅ | ✅ | ✅ | Standard GRPO-style approach to training verifiers. |
-| **GRPO-Instruct** | ❌ | ✅ | ✅ | RLVR training without intermediate thinking traces. |
-| **RAFT** | ✅ | ❌ | ✅ | On-policy rejection sampling fine-tuning using only positive reasoning samples. |
-| **Think-DPO** | ✅ | ✅ | ❌ | Offline preference optimization using pre-collected thinking traces. |
-| **Batch-online GRPO** | ✅ | ✅ | ⚠️ | Semi-online training where the generator policy is synced every 4 steps. |
+| [**GRPO-Think**](https://huggingface.co/INSAIT-Institute/GRPO-Think-7B-16k) | ✅ | ✅ | ✅ | Standard GRPO-style approach to training verifiers. |
+| [**GRPO-Instruct**](https://huggingface.co/INSAIT-Institute/GRPO-Instruct-7B) | ❌ | ✅ | ✅ | RLVR training without intermediate thinking traces. |
+| [**RAFT**](https://huggingface.co/INSAIT-Institute/RAFT-7B) | ✅ | ❌ | ✅ | On-policy rejection sampling fine-tuning using only positive reasoning samples. |
+| [**DPO-Think**](https://huggingface.co/INSAIT-Institute/DPO-Think-7B) | ✅ | ✅ | ❌ | Offline preference optimization using pre-collected thinking traces. |
+| [**BatchOnline-GRPO**](https://huggingface.co/INSAIT-Institute/BatchOnline-GRPO-7B) | ✅ | ✅ | ⚠️ | Semi-online training where the generator policy is synced every 4 steps. |
+
+The `-4k` / `-8k` / `-16k` suffix on GRPO-Think checkpoints denotes the reasoning-token budget (maximum completion length) used during training.
 
 ## 🎁 Datasets
-The Aletheia dataset collection is available on HuggingFace and includes:
 
-* **Aletheia-Train**: A dataset of 2-5 candidate code snippets to solve a coding problem, each with exactly one correct code
-* **Aletheia-DPO**: A companion dataset to Aletheia-Train, containing "chosen" and "rejected" responses for each instance. The chosen response identifies the correct code snippet, while the rejected responses do not.
-* **Aletheia-Heldout**: A completely in-distribution test set
-* **Aletheia-Strong**: An OOD test set where the candidates are generated by stronger models
-* **Aletheia-Hard**: An OOD test set where the comparison between candidates is more difficult
-* **Aletheia-Adv**: An OOD test set where the candidates are adversarially modified to exploit common LLM biases
+The Aletheia dataset collection includes:
+
+* [**Aletheia-Train**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-Train): 50,000 training instances, each pairing a competitive-programming problem with 2–5 candidate solutions — exactly one of which is correct (execution-verified) — generated by a pool of weak and strong policy models across Python, C++, and Java.
+* [**Aletheia-Train-Questions**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-Train-Questions): The 3,574 unique problem statements underlying Aletheia-Train, useful for on-policy sampling.
+* [**Aletheia-DPO**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-DPO): A companion dataset to Aletheia-Train containing "chosen" and "rejected" verification responses for each instance. The chosen response identifies the correct code snippet, while the rejected response does not.
+* [**Aletheia-Mixed**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-Mixed): A variant of Aletheia-Train in which ~25% of instances carry adversarial modifications that exploit common LLM biases (authority references, self-declared correctness, misleading comments, etc.).
+* [**Aletheia-Mixed-DPO**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-Mixed-DPO): The DPO-style companion to Aletheia-Mixed.
+* [**Aletheia-Heldout**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-Heldout): A completely in-distribution test set.
+* [**Aletheia-Strong**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-Strong): An OOD test set where the candidates are generated by stronger models.
+* [**Aletheia-Hard**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-Hard): An OOD test set where the comparison between candidates is more difficult.
+* [**Aletheia-Adv**](https://huggingface.co/datasets/INSAIT-Institute/Aletheia-Adv): An OOD test set where the candidates are adversarially modified to exploit common LLM biases.
+
+## 💡 Intended Uses
 
-## 💡 Intended uses
-* **RLHF / RLAIF**: plug-and-play reward function for code generation policy optimization.  
-* **Automated evaluation**: LLM-as-a-judge for a variety of code-related tasks.  
-* **Research**: study the effects of thinking traces, on-policy learning, and negative samples in training successful code verifiers
+* **RLHF / RLAIF**: plug-and-play reward function for code generation policy optimization.
+* **Automated evaluation**: LLM-as-a-judge for a variety of code-related tasks.
+* **Research**: study the effects of thinking traces, on-policy learning, and negative samples in training successful code verifiers.
 
-## Citations
+## 📚 Citation
+
+If you find this work useful, please cite our paper:
 
 ```bibtex
-@article{aletheia2025,
-  title={Aletheia: What Makes RLVR For Code Verifiers Tick?},
-  author={Venkatkrishna, Vatsal and Paul, Indraneil and Gurevych, Iryna},
-  journal={arXiv preprint},
-  year={2025}
+@misc{venkatkrishna2026aletheiamakesrlvrcode,
+      title={Aletheia: What Makes RLVR For Code Verifiers Tick?},
+      author={Vatsal Venkatkrishna and Indraneil Paul and Iryna Gurevych},
+      year={2026},
+      eprint={2601.12186},
+      archivePrefix={arXiv},
+      primaryClass={cs.SE},
+      url={https://arxiv.org/abs/2601.12186},
 }
-```
+```
+
+## 📄 License
+
+This work is licensed under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).