Downloads · 30 days
29
2% of all-time downloads
cnmoro/TinyLlama-1.1B-intermediate-1.5T-PTBR-Instruct-v3-8k
TinyLlama-1.1B-intermediate-1.5T-PTBR-Instruct-v3-8k is a text generation model from cnmoro. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
Finetuned version of PY007/TinyLlama-1.1B-intermediate-step-715k-1.5T, on a Portuguese instruct dataset, using axolotl.
Downloads · 30 days
29
2% of all-time downloads
All-time downloads
1.2K
Public
Repo size
4.4 GB
Likes
8
Public
Click a slice to open those files.
.bin2.2 GB · 100%
From the Hugging Face model README
Finetuned version of PY007/TinyLlama-1.1B-intermediate-step-715k-1.5T, on a Portuguese instruct dataset, using axolotl.
v0, v1 and v2 were finetuned for the default 2048 context length. For this v3, I have used the existing v2 and finetuned the model on a 8k context length dataset. It works fairly well, but it's reasoning capabilities are not so strong. It works well for basic RAG / question answering on retrieved content.
Prompt format:
f"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\n{instruction}\n\n### Response:\n"