Downloads · 30 days
50
2% of all-time downloads
kaiimran/malaysian-llama2-7b-32k-instructions-lora-sentiment-analysis-v2-gguf
malaysian-llama2-7b-32k-instructions-lora-sentiment-analysis-v2-gguf is a machine learning model from kaiimran. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
This repository contains a GGUF (GPT-Generated Unified Format) version of the kaiimran/malaysian-llama2-7b-32k-instructions-lora-sentiment-analysis-v2 model, specifically adapted for sentiment analysis of Malay text f…
Downloads · 30 days
50
2% of all-time downloads
All-time downloads
2.5K
Public
Repo size
13.5 GB
Likes
0
Public
Click a slice to open those files.
.gguf27 GB · 100%
From the Hugging Face model README
This repository contains a GGUF (GPT-Generated Unified Format) version of the kaiimran/malaysian-llama2-7b-32k-instructions-lora-sentiment-analysis-v2 model, specifically adapted for sentiment analysis of Malay text from social media. This GGUF version allows for efficient inference on various platforms and devices.
Lakukan analisis sentimen bagi teks di dalam tanda sempang berikut.
———
### Teks: tulis teks, tweet, atau ayat yang anda ingin analisa di ruangan ini.
———
Kenal pasti sama ada teks ini secara keseluruhannya mengandungi sentimen positif atau negatif.
Jawab dengan hanya satu perkataan: "positif" atau "negatif".
Sentimen:
Example:
Lakukan analisis sentimen bagi teks di dalam tanda sempang berikut.
———
### Teks: alhamdulillah terima kasih sis support saya 🥹
———
Kenal pasti sama ada teks ini secara keseluruhannya mengandungi sentimen positif atau negatif.
Jawab dengan hanya satu perkataan: "positif" atau "negatif".
Sentimen:
Clone the llama.cpp repository and build it:
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
make
Download the GGUF model file from this repository.
Run inference using the following command:
./main -m path/to/your/model.gguf -n 256 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt
Replace path/to/your/model.gguf with the actual path to this downloaded GGUF file.
This GGUF model can be used with various UI-based systems for an easier, more user-friendly experience:
GPT4All:
Jan.AI:
Ollama:
malaysian-sentiment.Ollama) with the following content:
FROM /path/to/your/model.gguf
/path/to/your/model.gguf with the actual path to this downloaded GGUF file.ollama create malaysian-sentiment -f malaysian-sentiment.Ollamaollama run malaysian-sentimentFor those who prefer using Python, you can use the following code to load and run inference with the model:
from unsloth import FastLanguageModel
# Model configuration
max_seq_length = 4096 # Extended from TinyLlama's 2048 using RoPE Scaling
dtype = None # Auto-detection (Float16 for Tesla T4, V100; Bfloat16 for Ampere+)
load_in_4bit = True # Use 4-bit quantization to reduce memory usage
# Load the model
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="kaiimran/malaysian-llama2-7b-32k-instructions-lora-sentiment-analysis-v2",
max_seq_length=max_seq_length,
dtype=dtype,
load_in_4bit=load_in_4bit,
)
# Enable faster inference
FastLanguageModel.for_inference(model)
# Prepare the prompt template
alpaca_prompt = """Lakukan analisis sentimen bagi teks di dalam tanda sempang berikut.
———
### Teks: {}
———
Kenal pasti sama ada teks ini secara keseluruhannya mengandungi sentimen positif atau negatif.
Jawab dengan hanya satu perkataan: "positif" atau "negatif".
Sentimen:
{}"""
# Example tweet for analysis
tweet = """
alhamdulillah terima kasih sis support saya ☺️ semoga sis dimurahkan rezeki dipanjangkan usia dan dipermudahkan segala urusan https://t.co/nSfNPGpiW8
"""
# Tokenize input
inputs = tokenizer(
[alpaca_prompt.format(tweet, "")],
return_tensors="pt"
).to("cuda")
# Generate output
outputs = model.generate(**inputs, max_new_tokens=10, use_cache=True)
# Print result
print(tokenizer.batch_decode(outputs)[0])
Feel free to open issues or submit pull requests if you have suggestions for improvements or encounter any problems.