Instructions to use codefuse-ai/CodeFuse-DeepSeek-33B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use codefuse-ai/CodeFuse-DeepSeek-33B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="codefuse-ai/CodeFuse-DeepSeek-33B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("codefuse-ai/CodeFuse-DeepSeek-33B") model = AutoModelForCausalLM.from_pretrained("codefuse-ai/CodeFuse-DeepSeek-33B") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use codefuse-ai/CodeFuse-DeepSeek-33B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "codefuse-ai/CodeFuse-DeepSeek-33B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "codefuse-ai/CodeFuse-DeepSeek-33B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/codefuse-ai/CodeFuse-DeepSeek-33B
- SGLang
How to use codefuse-ai/CodeFuse-DeepSeek-33B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "codefuse-ai/CodeFuse-DeepSeek-33B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "codefuse-ai/CodeFuse-DeepSeek-33B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "codefuse-ai/CodeFuse-DeepSeek-33B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "codefuse-ai/CodeFuse-DeepSeek-33B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use codefuse-ai/CodeFuse-DeepSeek-33B with Docker Model Runner:
docker model run hf.co/codefuse-ai/CodeFuse-DeepSeek-33B
How to use the Multi-Round with System Prompt
I use the Multi-Round prompt template
"""system
System instructionhuman
Human 1st round inputbot
Bot 1st round output<|end▁of▁sentence|>human
Human 2nd round inputbot
Bot 2nd round output<|end▁of▁sentence|>
...
...
...human
Human nth round inputbot
"""
assystem
程序修复任务human
这是一个存在缺陷的Java方法:public TimeSeries createCopy(RegularTimePeriod start, RegularTimePeriod end) throws CloneNotSupportedException {
if (start == null) {
throw new IllegalArgumentException("Null 'start' argument.");
}
if (end == null) {
throw new IllegalArgumentException("Null 'end' argument.");
}
if (start.compareTo(end) > 0) {
throw new IllegalArgumentException("Requires start on or before end.");
}
boolean emptyRange = false;
int startIndex = getIndex(start);
if (startIndex < 0) {
startIndex = -(startIndex + 1);
if (startIndex == this.data.size()) {
emptyRange = true;
}
}
int endIndex = getIndex(end);
if (endIndex < 0) {
endIndex = -(endIndex + 1);
endIndex = endIndex - 1;
}
if (endIndex < 0) {
emptyRange = true;
}
if (emptyRange) {
TimeSeries copy = (TimeSeries) super.clone();
copy.data = new java.util.ArrayList();
return copy;
} else {
return createCopy(startIndex, endIndex);
}
}, 具体的缺陷行是: if (endIndex < 0) {
, 缺陷行行号是:20,请解释该缺陷行缺陷导致方法出现缺陷的原因。bothuman
请给出修复该缺陷方法的策略。优先考虑修复缺陷行使缺陷方法正确,如不行,再考虑增加、删除行等操作。bothuman
请根据之前给出的错误原因和修复策略,修复这个缺陷方法。请以如下格式给出修复后的缺陷行和完整的缺陷方法。// fixed line: // fixed method:bothuman
请逐行解释修复后的缺陷方法,并确认是否修复成功。若不成功,请重新修复,并以如下格式给出重新修复后的缺陷行和完整的缺陷方法。// fixed line: // fixed method:bot
but it only gengrate at the last round: 修复后的缺陷行:
if (endIndex < 0) {
endIndex = -(endIndex + 1);
endIndex = endIndex - 1;
}
修复后的完整方法:
public TimeSeries createCopy(RegularTimePeriod start, RegularTimePeriod end) throws CloneNotSupportedException {
if (start == null) {
throw new IllegalArgumentException("Null 'start' argument.");
}
if (end == null) {
throw new IllegalArgumentException("Null 'end' argument.");
}
if (start.compareTo(end) > 0) {
throw new IllegalArgumentException("Requires start on or before end.");
}
boolean emptyRange = false;
int startIndex = getIndex(start);
if (startIndex < 0) {
startIndex = -(startIndex + 1);
if (startIndex == this.data.size()) {
emptyRange = true;
}
}
int endIndex = getIndex(end);
if (endIndex < 0) {
endIndex = -(endIndex + 1);
endIndex = endIndex - 1;
}
if (endIndex < 0) {
emptyRange = true;
}
if (emptyRange) {
TimeSeries copy = (TimeSeries) super.clone();
copy.data = new java.util.ArrayList();
return copy;
} else {
return createCopy(startIndex, endIndex);
}
}
修复后的方法解释:
修复后的缺陷行解决了当endIndex小于0时,endIndex的值不正确的问题。修复后的完整方法在原有基础上增加了对endIndex的处理,确保了方法的正确性。
I would like to know how to design a prompt that will allow the model to generate multiple rounds of answers.
hello,
This is the multi-round prompt format:
"""
<s>system
System instruction
<s>human
Human 1st round input
<s>bot
Bot 1st round output<|end▁of▁sentence|>
<s>human
Human 2nd round input
<s>bot
Bot 2nd round output<|end▁of▁sentence|>
...
...
...
<s>human
Human nth round input
<s>bot
"""