Downloads · 30 days
19
4% of all-time downloads
barbaroo/gptsw3-6.7B-translation-en-fo
gptsw3-6.7B-translation-en-fo is a text generation model from barbaroo. Use it when you need the model to write or continue text. It is set up for transformers.
- Developed by: Barbara Scalvini - Model type: Fully merged model for English → Faroese translation - Languages: English, Faroese - License: Inherits license from the base model (GPT-SW3 6.7B) - Finetuned from: AI-Swe…
Downloads · 30 days
19
4% of all-time downloads
All-time downloads
461
Public
Parameters
6.7B
26.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors26.9 GB · 100%
From the Hugging Face model README
This model is the merged version of the PEFT adapter barbaroo/gptsw3_translate_synth_6.7B with its base model.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import re
import pandas as pd
# Model repo
MODEL_NAME = "barbaroo/gptsw3-6.7B-translation-en-fo"
# Quantization config (8-bit)
bnb_config = BitsAndBytesConfig(
load_in_8bit=True
)
# Initialize tokenizer & model
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForCausalLM.from_pretrained(
MODEL_NAME,
quantization_config=bnb_config,
device_map="auto",
)
model.eval()
# Alpaca-style prompt template
alpaca_prompt = """
### Instruction:
{}
### Input:
{}
### Response:
{}"""
EOS_TOKEN = tokenizer.eos_token
print("EOS token:", EOS_TOKEN)
# Example sentences
sentences = ["I love Faroese!"]
translations = []
for sentence in sentences:
inputs = tokenizer(
[
alpaca_prompt.format(
"Translate this sentence from English to Faroese:",
sentence,
"",
)
],
return_tensors="pt"
).to("cuda")
outputs = model.generate(
**inputs,
max_new_tokens=500,
use_cache=True,
do_sample=True,
temperature=0.1,
top_p=1,
)
output_string = tokenizer.batch_decode(outputs, skip_special_tokens=False)[0]
try:
response = output_string.split("Response:\n", 1)[1]
translation = response.replace(EOS_TOKEN, "")
except IndexError:
translation = ""
translations.append(translation)
print(translation)
Hyperparameters: