File size: 1,363 Bytes
b67582b 3894d07 b67582b a0281ec b67582b a2dcb92 b67582b 7d45f75 9cd4cf8 b67582b bb9454a b67582b cd342b5 b67582b 9a0e55d bb9454a 9cd4cf8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | ---
license: apache-2.0
datasets:
- Shuu12121/rust-codesearch-dataset-open
- Shuu12121/java-codesearch-dataset-open
- code-search-net/code_search_net
- google/code_x_glue_ct_code_to_text
language:
- en
pipeline_tag: fill-mask
tags:
- code
- code-search
- retrieval
- sentence-similarity
- bert
- transformers
- deep-learning
- machine-learning
- nlp
- programming
- multi-language
- rust
- python
- java
- javascript
- php
- ruby
- go
new_version: Shuu12121/CodeModernBERT-Owl-v3-len2048
---
# **🦉CodeModernBERT-Owl**
旧版のモデルです。[こちらのモデル](https://huggingface.co/Shuu12121/CodeModernBERT-Owl-v3-len2048)など最新版をお使いください！！
## **モデルパラメータ / Model Parameters**
| パラメータ / Parameter | 値 / Value |
|-------------------------|------------|
| **vocab_size** | 50,004 |
| **hidden_size** | 768 |
| **num_hidden_layers** | 12 |
| **num_attention_heads**| 12 |
| **intermediate_size** | 3,072 |
| **max_position_embeddings** | 2,048 |
| **type_vocab_size** | 2 |
| **hidden_dropout_prob**| 0.1 |
| **attention_probs_dropout_prob** | 0.1 |
| **local_attention_window** | 128 |
| **rope_theta** | 160,000 |
| **local_attention_rope_theta** | 10,000 |
---
## **ライセンス / License**
**Apache-2.0** |