Downloads · 30 days
2.1K
12% of all-time downloads
deepseek-coder-1.3b-instruct-imatrix-GGUF
duyntnet/deepseek-coder-1.3b-instruct-imatrix-GGUF
deepseek-coder-1.3b-instruct-imatrix-GGUF is a text generation model from duyntnet. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Quantizations of https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct
All-time downloads
17.1K
Public
Repo size
18.1 GB
Likes
1
Trending 1
Repo makeup
Click a slice to open those files.
.gguf18.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Access
- Public
- Created
- Apr 24, 2024
- Updated
- Apr 26, 2024
- SHA
- df4de508
Try a prompt
Model card
From the Hugging Face model README
Quantizations of https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct
From original readme
3. How to Use
Here give some examples of how to use our model.
Chat Model Inference
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/deepseek-coder-1.3b-instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/deepseek-coder-1.3b-instruct", trust_remote_code=True, torch_dtype=torch.bfloat16).cuda()
messages=[
{ 'role': 'user', 'content': "write a quick sort algorithm in python."}
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
# tokenizer.eos_token_id is the id of <|EOT|> token
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, top_k=50, top_p=0.95, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True))
- Task
- Text Generation
- Library
- transformers
- License
- other
- Languages
- en
- Created
- Apr 24, 2024
- Updated
- Apr 26, 2024