File size: 4,652 Bytes
277f3fb 64212e5 5b2f985 277f3fb b906812 277f3fb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen3-8B
pipeline_tag: image-text-to-text
tags:
- Bee-8B
- Fully-Open-MLLMs
datasets:
- Open-Bee/Honey-Data-15M
library_name: transformers
---
# Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMs
[[🏠 Homepage](https://open-bee.github.io/)] [[📖 Arxiv Paper](https://arxiv.org/pdf/2510.13795)] [[🤗 Models & Datasets](https://huggingface.co/collections/Open-Bee/bee-8b-68ecbf10417810d90fbd9995)] [[💻 Code](https://github.com/Open-Bee)]
## Introduction
We introduce **Bee-8B**, a new state-of-the-art, fully open 8B Multimodal Large Language Model (MLLM) designed to close the performance gap with proprietary models by focusing on data quality.
Bee-8B is trained on our new **Honey-Data-15M** corpus, a high-quality supervised fine-tuning (SFT) dataset of approximately 15 million samples. This dataset was meticulously created with our transparent, adaptable, and open-source data curation pipeline, **HoneyPipe**, which systematically cleans noisy data and enriches it with a novel dual-level (short and long) Chain-of-Thought (CoT) strategy.
This dataset enables Bee-8B to achieve exceptional performance, particularly in complex reasoning, establishing a new standard for fully open MLLMs.
## Key Features
- **High-Quality, Large-Scale Dataset:** We release **Honey-Data-15M**, a new 15M-sample SFT corpus. It has undergone extensive cleaning to remove widespread noise and has been enriched with dual-level CoT reasoning to enhance advanced problem-solving capabilities.
- **Fully Open-Source Data Curation Suite:** We provide not just the data, but the entire methodology. **HoneyPipe** and its underlying framework **DataStudio** offer the community a transparent and reproducible pipeline, moving beyond static dataset releases.
- **State-of-the-Art Open Model:** Our model, **Bee-8B**, achieves state-of-the-art performance among fully open MLLMs and is highly competitive with recent semi-open models like InternVL3.5-8B, demonstrating the power of high-quality data.
## News
- **[2025.12.17]** 🔥 We have released all data and model weights across different stages. For the final stage (RL data), you can directly merge [ViRL39K](https://huggingface.co/datasets/TIGER-Lab/ViRL39K) and [MMK12](https://huggingface.co/datasets/FanqingM/MMK12) and use the [VeRL](https://github.com/volcengine/verl) framework for training.
- **[2025.11.03]** 📊 **[Honey-Data-15M](https://huggingface.co/datasets/Open-Bee/Honey-Data-15M) & [Honey-Data-1M](https://huggingface.co/datasets/Open-Bee/Honey-Data-1M) is Released\!** You can download the 15M full version and the 1M efficient version from [HuggingFace]((https://huggingface.co/collections/Open-Bee/bee-8b-68ecbf10417810d90fbd9995)).
- **[2025.10.20]** 🚀 **vLLM Support is Here!** Bee-8B now supports high-performance inference with [vLLM](https://github.com/vllm-project/vllm), enabling faster and more efficient deployment for production use cases.
- **[2025.10.13]** 🐝 **Bee-8B is Released\!** Our model is now publicly available. You can download it from [Hugging Face](https://huggingface.co/collections/Open-Bee/bee-8b-68ecbf10417810d90fbd9995).
## Bee-8B-Stage1
> [!IMPORTANT]
> **This is NOT a complete model and cannot be used for inference directly.**
This repository contains the MLP projector weights that bridge the vision encoder ([SigLIP2](https://huggingface.co/google/siglip2-so400m-patch14-384)) and the language model ([Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)).
**Weights:**
| Key | Shape | Description |
|-----|-------|-------------|
| `model.multi_modal_projector.pre_norm.weight` | [1152] | Pre-normalization weight |
| `model.multi_modal_projector.pre_norm.bias` | [1152] | Pre-normalization bias |
| `model.multi_modal_projector.linear_1.weight` | [4096, 1152] | First linear layer |
| `model.multi_modal_projector.linear_1.bias` | [4096] | First linear bias |
| `model.multi_modal_projector.linear_2.weight` | [4096, 4096] | Second linear layer |
| `model.multi_modal_projector.linear_2.bias` | [4096] | Second linear bias |
## Acknowledgements
Bee-8B is developed based on the architectures and codebases of the following projects: [R-4B](https://huggingface.co/YannQi/R-4B), [LLaVA-OneVision](https://github.com/LLaVA-VL/LLaVA-NeXT), [SigLIP2](https://huggingface.co/google/siglip2-so400m-patch14-384), [Qwen3](https://github.com/QwenLM/Qwen3), and evaluated using [VLMEvalKit](https://github.com/open-compass/VLMEvalKit). We sincerely thank these projects for their outstanding contributions to the open-source community. |