aisg-senti-llama

Uploaded model

• Developed by: kuross
• License: apache-2.0
• Finetuned from model : unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit

To use this model:

from unsloth import FastLanguageModel
from unsloth.chat_templates import get_chat_template

# Import the model and tokenizer
ft_model, ft_tokenizer = FastLanguageModel.from_pretrained(
    "kuross/aisg-senti-llama",
    max_seq_length=max_seq_length,
    dtype=dtype,
    load_in_4bit=load_in_4bit
)

# Sample chat
tokenizer = get_chat_template(
    ft_tokenizer,
    chat_template = "llama-3", # Supports zephyr, chatml, mistral, llama, alpaca, vicuna, vicuna_old, unsloth
    mapping = {"role" : "from", "content" : "value", "user" : "human", "assistant" : "gpt"}, # ShareGPT style
)

FastLanguageModel.for_inference(ft_model) # Enable native 2x faster inference

text_msg = "I love that movie!" # Edit to what you want

messages = [
    {"from": "human", "value": f"Determine the sentiment of the following text: {text_msg}"},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True, # Must add for generation
    return_tensors = "pt",
)
inputs = inputs.to("cuda")

outputs = ft_model.generate(input_ids = inputs, max_new_tokens = 64, use_cache = True)
tokenizer.batch_decode(outputs)

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>