---
language:
- en
license: mit
library_name: transformers
tags:
- materials-science
- crystallography
- generative-ai
- inverse-design
- chemistry
datasets:
- c-bone/mattergen_den_ehull
base_model: c-bone/CrystaLLM-pi_base
pipeline_tag: text-generation
---

# Model Card for CrystaLLM-pi_density

## Model Details

### Model Description

**CrystaLLM-pi_density** is a conditional generative model designed for the inverse design of inorganic crystalline materials. It is a fine-tuned version of the `CrystaLLM-pi` framework, based on a GPT-2 decoder-only architecture. This specific variant employs the **Property-Key-Value (PKV)** attention mechanism (referred to as "Prefix attention" in the associated preprint) to condition the generation of Crystallographic Information Files (CIFs) on physical density and thermodynamic stability.

The model generates crystal structures (cell parameters and atomic positions) based on two target scalar properties:
1. **Density** ($g/cm^3$)
2. **Energy above convex hull** ($E_{hull}$, eV/atom) - a proxy for thermodynamic stability.

- **Developed by:** Bone et al. (University College London)
- **Model type:** Autoregressive Transformer with Prefix Attention Conditioning
- **Language(s):** CIF (Crystallographic Information File) syntax
- **License:** MIT
- **Finetuned from model:** `c-bone/CrystaLLM-pi_base`

### Model Sources

- **Repository:** [GitHub: CrystaLLM-pi](https://github.com/C-Bone-UCL/CrystaLLM-pi)
- **Paper:** [Discovery and recovery of crystalline materials with property-conditioned transformers (arXiv:2511.21299)](https://arxiv.org/abs/2511.21299)
- **Dataset:** [HuggingFace: c-bone/mattergen_den_ehull](https://huggingface.co/datasets/c-bone/mattergen_den_ehull)

## Uses

### Direct Use

The model is intended for research in materials science, specifically for generating structures with targeted densities, such as porous materials (low density) or radiation shielding candidates (high density). Users can input a desired density (e.g., 2.0 $g/cm^3$) and a stability criterion to generate candidate crystal structures.

### Out-of-Scope Use

- **Organic Materials:** The model was trained exclusively on inorganic crystal structures.
- **Large Unit Cells:** Due to the context window limit of 1024 tokens, the model cannot reliably generate unit cells containing more than approximately 20 atoms.
- **Disordered Systems:** The model currently generates ordered structures and does not natively handle partial occupancies.
- **Production Deployment:** This is a research artifact. Generated structures must be validated via Density Functional Theory (DFT) or other simulation methods before synthesis attempts.

## Bias, Risks, and Limitations

- **Training Distribution Bias:** The model is fine-tuned on the MatterGen Density dataset. Performance may degrade for density targets that are outliers in the training distribution (e.g., extremely dense or extremely porous materials).
- **Validity:** As an autoregressive language model, it may generate syntactically incorrect CIFs or chemically implausible structures. Post-processing validation is required.

## Getting started

Generation: [`T2_load_and_generate.ipynb`](https://github.com/C-Bone-UCL/CrystaLLM-pi/blob/main/notebooks/T2_load_and_generate.ipynb). The finetune that produced it: [`T1_finetune_density_example.ipynb`](https://github.com/C-Bone-UCL/CrystaLLM-pi/blob/main/notebooks/T1_finetune_density_example.ipynb).

## Citation

```bibtex
@misc{bone2025discoveryrecoverycrystallinematerials,
      title={Discovery and recovery of crystalline materials with property-conditioned transformers},
      author={Cyprien Bone and Matthew Walker and Bradley A. A. Martin and Kuangdai Leng and Luis M. Antunes and Ricardo Grau-Crespo and Amil Aligayev and Javier Dominguez and Keith T. Butler},
      year={2025},
      eprint={2511.21299},
      archivePrefix={arXiv},
      primaryClass={cond-mat.mtrl-sci},
      url={https://arxiv.org/abs/2511.21299},
}