Upload README.md with huggingface_hub

README.md

@@ -1,62 +1,359 @@
 ---
+license: other
+license_name: deepseek-license
+license_link: https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/LICENSE-MODEL
 base_model: deepseek-ai/deepseek-coder-6.7b-instruct
-library_name: peft
-model_name: forge-coder-v1.21.11
 tags:
-- base_model:adapter:deepseek-ai/deepseek-coder-6.7b-instruct
-- lora
-- sft
-- transformers
-- trl
-licence: license
+  - minecraft
+  - forge
+  - modding
+  - code-generation
+  - lora
+  - peft
+  - sft
+  - trl
+language:
+  - en
+  - code
+library_name: peft
 pipeline_tag: text-generation
+datasets:
+  - custom
+model-index:
+  - name: forge-coder-v1.21.11
+    results: []
+---
+
+<div align="center">
+  <h1>🔨 Forge Coder v1.21.11</h1>
+  <p><strong>A Specialized Code Generation Model for Minecraft Forge Mod Development</strong></p>
+  <p>
+    <a href="#quickstart">Quick Start</a> •
+    <a href="#capabilities">Capabilities</a> •
+    <a href="#examples">Examples</a> •
+    <a href="#training">Training Details</a>
+  </p>
+</div>
+
 ---
 
-# Model Card for forge-coder-v1.21.11
+## Overview
+
+**Forge Coder** is a fine-tuned large language model specifically designed to assist developers in creating Minecraft Forge mods. Built on top of DeepSeek Coder 6.7B, this model has been trained on extensive Forge mod source code and documentation to provide accurate, idiomatic, and up-to-date code generation for Minecraft modding.
+
+### Key Features
+
+- 🎯 **Specialized Knowledge**: Deep understanding of Forge API, registry systems, and modding patterns
+- 🔄 **Version-Aligned**: Trained specifically for Minecraft 1.21.11 and Forge 1.21.11
+- 💡 **Code Completion**: Generate complete mod components from natural language descriptions
+- 📚 **Best Practices**: Follows modern Forge modding conventions and patterns
+
+## Model Details
 
-This model is a fine-tuned version of [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+| Property | Value |
+|----------|-------|
+| **Base Model** | [deepseek-ai/deepseek-coder-6.7b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct) |
+| **Fine-tuning Method** | QLoRA (4-bit quantization + LoRA) |
+| **LoRA Rank** | 64 |
+| **LoRA Alpha** | 128 |
+| **Trainable Parameters** | 159.9M (2.3% of 6.7B) |
+| **Target Forge Version** | 1.21.11 |
+| **Target Minecraft Version** | 1.21.11 |
+| **MCP Mappings** | 20251209.095502 |
 
-## Quick start
+## Quickstart
+
+### Installation
+
+```bash
+pip install transformers peft accelerate bitsandbytes torch
+```
+
+### Basic Usage
 
 ```python
-from transformers import pipeline
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+from peft import PeftModel
+
+# Quantization config for efficient inference
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype=torch.bfloat16,
+    bnb_4bit_use_double_quant=True,
+)
+
+# Load model
+base_model_id = "deepseek-ai/deepseek-coder-6.7b-instruct"
+adapter_id = "hwding/forge-coder-v1.21.11"
+
+tokenizer = AutoTokenizer.from_pretrained(adapter_id)
+model = AutoModelForCausalLM.from_pretrained(
+    base_model_id,
+    quantization_config=bnb_config,
+    device_map="auto",
+)
+model = PeftModel.from_pretrained(model, adapter_id)
+
+# Generate code
+def generate_forge_code(prompt: str, max_tokens: int = 1024) -> str:
+    messages = f"""### System:
+You are an expert Minecraft Forge mod developer for version 1.21.11. Write clean, efficient, and well-structured Java code.
+
+### User:
+{prompt}
+
+### Assistant:
+"""
+    inputs = tokenizer(messages, return_tensors="pt").to(model.device)
+    outputs = model.generate(
+        **inputs,
+        max_new_tokens=max_tokens,
+        temperature=0.7,
+        top_p=0.95,
+        do_sample=True,
+        pad_token_id=tokenizer.eos_token_id,
+    )
+    return tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+# Example
+code = generate_forge_code("Create a custom ore block that drops experience when mined")
+print(code)
+```
+
+## Capabilities
+
+The model excels at generating code for various Forge mod components:
+
+| Category | Examples |
+|----------|----------|
+| **Blocks** | Custom blocks, ore blocks, machine blocks, multi-blocks |
+| **Items** | Tools, weapons, armor, food items, custom items |
+| **Entities** | Mobs, projectiles, vehicles, tile entities |
+| **GUI/Screens** | Container menus, custom screens, HUD elements |
+| **Networking** | Packet handling, client-server sync |
+| **World Generation** | Ore generation, structures, biomes, features |
+| **Events** | Forge event handlers, lifecycle events |
+| **Registries** | DeferredRegister patterns, registry objects |
+| **Capabilities** | Custom capabilities, capability providers |
+| **Recipes** | Custom recipe types, recipe serializers |
+| **Data Generation** | Datagen for models, blockstates, loot tables, tags |
+| **Rendering** | Block/item/entity renderers, shaders |
+
+## Examples
+
+### Creating a Custom Block
+
+**Prompt:** "Create a copper lamp block that emits light when powered by redstone"
+
+```java
+public class CopperLampBlock extends Block {
+    public static final BooleanProperty LIT = BlockStateProperties.LIT;
+    
+    public CopperLampBlock(Properties properties) {
+        super(properties);
+        this.registerDefaultState(this.stateDefinition.any().setValue(LIT, false));
+    }
+    
+    @Override
+    protected void createBlockStateDefinition(StateDefinition.Builder<Block, BlockState> builder) {
+        builder.add(LIT);
+    }
+    
+    @Override
+    public void neighborChanged(BlockState state, Level level, BlockPos pos, 
+                                Block block, BlockPos fromPos, boolean isMoving) {
+        if (!level.isClientSide) {
+            boolean powered = level.hasNeighborSignal(pos);
+            if (powered != state.getValue(LIT)) {
+                level.setBlock(pos, state.setValue(LIT, powered), 2);
+            }
+        }
+    }
+    
+    @Override
+    public int getLightEmission(BlockState state, BlockGetter level, BlockPos pos) {
+        return state.getValue(LIT) ? 15 : 0;
+    }
+}
+```
+
+### Creating a Custom Item
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
-generator = pipeline("text-generation", model="None", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
-print(output["generated_text"])
+**Prompt:** "Create a magic wand item that shoots fireballs on right click"
+
+```java
+public class MagicWandItem extends Item {
+    public MagicWandItem(Properties properties) {
+        super(properties.stacksTo(1).durability(100));
+    }
+    
+    @Override
+    public InteractionResultHolder<ItemStack> use(Level level, Player player, InteractionHand hand) {
+        ItemStack stack = player.getItemInHand(hand);
+        
+        if (!level.isClientSide) {
+            Vec3 look = player.getLookAngle();
+            SmallFireball fireball = new SmallFireball(level, player, look.x, look.y, look.z);
+            fireball.setPos(player.getX(), player.getEyeY() - 0.1, player.getZ());
+            level.addFreshEntity(fireball);
+            
+            stack.hurtAndBreak(1, player, (p) -> p.broadcastBreakEvent(hand));
+            player.getCooldowns().addCooldown(this, 20);
+        }
+        
+        level.playSound(player, player.blockPosition(), SoundEvents.BLAZE_SHOOT, 
+                        SoundSource.PLAYERS, 1.0F, 1.0F);
+        
+        return InteractionResultHolder.sidedSuccess(stack, level.isClientSide);
+    }
+}
 ```
 
-## Training procedure
+## Training Data
+
+### Data Sources
+
+| Source | Description | Files |
+|--------|-------------|-------|
+| **Forge Source** | MinecraftForge core, ForgeGradle, EventBus, ModLauncher | ~1,200 |
+| **Popular Mods** | 27 well-maintained open-source mod repositories | ~21,700 |
+| **Documentation** | Official Forge docs and tutorials | 74 |
+
+### Featured Mod Repositories
+
+Training data includes code from highly-regarded mods:
+
+- **Applied Energistics 2** - Storage & automation
+- **Mekanism** - Tech & machinery
+- **Create** - Mechanical contraptions
+- **Botania** - Nature magic
+- **Thermal Series** - Energy systems
+- **Tinkers' Construct** - Tool crafting
+- **Immersive Engineering** - Industrial machines
+- **JustEnoughItems (JEI)** - Recipe viewing
+- **TerraFirmaCraft** - Survival overhaul
+- **The Twilight Forest** - Dimension mod
+- **Quark** - Vanilla enhancements
+- **RFTools** - RF-powered utilities
+- And 15 more...
+
+### Dataset Statistics
 
- 
+| Metric | Value |
+|--------|-------|
+| Total Java Files Processed | 22,916 |
+| Training Samples | 13,936 |
+| Validation Samples | 734 |
+| Sample Types | Code completion, explanation, Q&A |
 
+## Training
 
-This model was trained with SFT.
+### Configuration
 
-### Framework versions
+| Parameter | Value |
+|-----------|-------|
+| Epochs | 3 |
+| Batch Size | 2 (per device) |
+| Gradient Accumulation | 8 steps |
+| Effective Batch Size | 128 |
+| Learning Rate | 2e-4 |
+| LR Scheduler | Cosine |
+| Warmup Ratio | 3% |
+| Max Sequence Length | 2,048 tokens |
+| Precision | BF16 |
+| Hardware | 8× NVIDIA H20 (96GB each) |
 
-- PEFT 0.18.0
+### Training Metrics
+
+| Metric | Value |
+|--------|-------|
+| Training Duration | 9h 12m |
+| Total Steps | 1,848 |
+| Final Training Loss | 0.27 |
+| Final Validation Loss | 0.325 |
+| Token Accuracy | 92.5% |
+| Eval Accuracy | 91.2% |
+
+### Loss Curve
+
+```
+Epoch 1: 0.89 → 0.42
+Epoch 2: 0.38 → 0.31
+Epoch 3: 0.29 → 0.27
+```
+
+### Framework Versions
+
+- PEFT: 0.18.0
 - TRL: 0.26.1
 - Transformers: 4.57.3
-- Pytorch: 2.5.1+cu121
+- PyTorch: 2.5.1+cu121
 - Datasets: 4.4.1
 - Tokenizers: 0.22.1
 
-## Citations
+## Limitations
+
+- **Version Specific**: Optimized for Forge 1.21.11; may produce outdated patterns for older versions
+- **Java Only**: Does not generate Kotlin, Gradle scripts, or JSON resources
+- **No Runtime Testing**: Generated code should be tested before use in production
+- **Context Window**: Limited to 2,048 tokens; very large classes may need to be split
+
+## Intended Use
 
+✅ **Recommended Uses:**
+- Learning Forge modding patterns and best practices
+- Rapid prototyping of mod components
+- Code completion and suggestions
+- Understanding Forge API usage
 
+⚠️ **Not Recommended For:**
+- Production code without review
+- Security-critical applications
+- Forge versions significantly different from 1.21.11
+
+## Citation
+
+If you use this model in your research or projects, please cite:
+
+```bibtex
+@misc{forge-coder-2024,
+  author = {hwding},
+  title = {Forge Coder: A Specialized Code Generation Model for Minecraft Forge Mod Development},
+  year = {2024},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/hwding/forge-coder-v1.21.11}
+}
+```
 
 Cite TRL as:
-    
+
 ```bibtex
 @misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
+    title        = {{TRL: Transformer Reinforcement Learning}},
+    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
+    year         = 2020,
+    journal      = {GitHub repository},
+    publisher    = {GitHub},
+    howpublished = {\url{https://github.com/huggingface/trl}}
 }
-```
+```
+
+## License
+
+This model is released under the [DeepSeek License](https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/LICENSE-MODEL), consistent with the base model.
+
+Training data was sourced from open-source repositories under various permissive licenses (MIT, Apache 2.0, LGPL, etc.).
+
+## Acknowledgments
+
+- [DeepSeek](https://github.com/deepseek-ai) for the excellent base model
+- [MinecraftForge](https://github.com/MinecraftForge) team for the modding framework
+- All open-source mod developers whose code made this training possible
+
+---
+
+<div align="center">
+  <p><strong>Happy Modding! 🎮⛏️</strong></p>
+</div>