| --- |
| license: apache-2.0 |
| datasets: |
| - Shuu12121/rust-codesearch-dataset-open |
| - Shuu12121/java-codesearch-dataset-open |
| - code-search-net/code_search_net |
| - google/code_x_glue_ct_code_to_text |
| language: |
| - en |
| pipeline_tag: fill-mask |
| tags: |
| - code |
| - code-search |
| - retrieval |
| - sentence-similarity |
| - bert |
| - transformers |
| - deep-learning |
| - machine-learning |
| - nlp |
| - programming |
| - multi-language |
| - rust |
| - python |
| - java |
| - javascript |
| - php |
| - ruby |
| - go |
| new_version: Shuu12121/CodeModernBERT-Owl-v3-len2048 |
| --- |
| |
| # **🦉CodeModernBERT-Owl** |
|
|
| 旧版のモデルです。[こちらのモデル](https://huggingface.co/Shuu12121/CodeModernBERT-Owl-v3-len2048)など最新版をお使いください！！ |
|
|
|
|
| ## **モデルパラメータ / Model Parameters** |
| | パラメータ / Parameter | 値 / Value | |
| |-------------------------|------------| |
| | **vocab_size** | 50,004 | |
| | **hidden_size** | 768 | |
| | **num_hidden_layers** | 12 | |
| | **num_attention_heads**| 12 | |
| | **intermediate_size** | 3,072 | |
| | **max_position_embeddings** | 2,048 | |
| | **type_vocab_size** | 2 | |
| | **hidden_dropout_prob**| 0.1 | |
| | **attention_probs_dropout_prob** | 0.1 | |
| | **local_attention_window** | 128 | |
| | **rope_theta** | 160,000 | |
| | **local_attention_rope_theta** | 10,000 | |
|
|
| --- |
|
|
| ## **ライセンス / License** |
| **Apache-2.0** |
