---
tags:
- ssm
- mamba
- meta-learning
- few-shot
- pytorch
- experimental
license: mit
library_name: transformers
pipeline_tag: text-generation
model_type: ssm
---

#  HyperMambaLM-300M

[![Model type](https://img.shields.io/badge/model-type%3A%20SSM-blueviolet)](https://huggingface.co/models?search=ssm)
[![Framework](https://img.shields.io/badge/framework-PyTorch-red)](https://pytorch.org/)
[![Meta-Learning](https://img.shields.io/badge/meta--learning-enabled-green)](https://arxiv.org/abs/1703.03400)
[![License: MIT](https://img.shields.io/badge/license-MIT-yellow)](./LICENSE)
![Status](https://img.shields.io/badge/status-experimental-orange)

> ⚠️ This is an **architecture-only repository** – no pretrained weights are available yet.  
>
> **HyperMambaLM** is a research prototype combining modern **state-space modeling** with **meta-learning components**.  
> Inspired by [Mamba](https://github.com/state-spaces/mamba), but extended with additional mechanisms for **few-shot adaptation**, **neuro-symbolic reasoning**, and **progressive learning**.

---

## 🧠 Highlights

- 🌀 **Mamba-style SSM**: Parallel scan for efficient sequence modeling  
- 🧬 **Meta-Learning (MAML)**: Learns to adapt with few examples  
- 🧠 **Neuro-Symbolic Layer**: Combines neural networks with logic reasoning  
- 🌱 **Progressive & Continual Learning**: Learns without forgetting  
- 💡 **Adaptive Precision**: Smart compute control  
- 🧩 Built for: NAS, federated learning, knowledge distillation...

---

## 📂 Files included

| File | Description |
|------|-------------|
| `config.json` | Model hyperparameters |
| `modeling_hypermamba.py` | Core model definition |
| `modeling_utils.py` | (Optional) Utility components |
| `demo.py` | Quick usage test |
| `__init__.py` | Python module loader |
| `README.md` | This file |

---

## 🚀 Quickstart (Colab / Local)

📌 This model is **not yet trained**, so only the architecture is available.

```python
# Step 1: Download model code (if not cloned)
!wget https://huggingface.co/hoanghai2110/HyperMambaLM-300M/resolve/main/modeling_hypermamba.py

# Step 2: Import and initialize
from modeling_hypermamba import HyperMambaLM, HyperMambaConfig

config = HyperMambaConfig.from_pretrained("hoanghai2110/HyperMambaLM-300M")
model = HyperMambaLM(config)

# Step 3: Run a dummy forward pass
import torch
input_ids = torch.randint(0, config.vocab_size, (1, 16))
output = model(input_ids)

print("✅ Output shape:", output.logits.shape)  # [1, 16, vocab_size]