damerajee_-_tinyllama-sft-small-v2-awq

Quantization made by Richard Erkhov.

Github

Discord

Request more models

tinyllama-sft-small-v2 - AWQ

• Model creator: https://huggingface.co/damerajee/
• Original model: https://huggingface.co/damerajee/tinyllama-sft-small-v2/

Original model description:

library_name: transformers base_model: unsloth/tinyllama-bnb-4bit license: mit datasets:

• yahma/alpaca-cleaned language:
• en pipeline_tag: text-generation tags:
• Instruct
• TinyLlama

---

Steps to try the model:

prompt Template

alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

load the model

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("damerajee/tinyllama-sft-small-v2")
model = AutoModelForCausalLM.from_pretrained("damerajee/tinyllama-sft-small-v2")

Inference

inputs = tokenizer(
[
    alpaca_prompt.format(
        "best places to visit in india", # instruction
        "", # input
        "", # output
    )
]*1, return_tensors = "pt")

outputs = model.generate(**inputs, max_new_tokens = 128, use_cache = True)
tokenizer.batch_decode(outputs)

Model Information

The base model unsloth/tinyllama-bnb-4bit was Instruct finetuned using Unsloth

Model Limitations

The model was trained on a very small dataset so it might not be as good ,will be training on larger dataset soon

Training Details

The model was trained for 1 epoch on a free goggle colab which took about 1 hour and 30 mins approximately