llama-3.1-8b-egyptian-german

Egyptian-German Translation Model

This model translates Ancient Egyptian transliterations to German. It's a LoRA fine-tuned version of Llama-3.1-8B-Instruct.

Model Details

• Base Model: meta-llama/Llama-3.1-8B-Instruct
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Training Data: ~4,000 transliteration-translation pairs from Thesaurus Linguae Aegyptiae
• Task: Neural Machine Translation (Ancient Egyptian → German)
• Language Pair: Egyptian (transliteration) → German

Training Details

• LoRA Rank: 16
• LoRA Alpha: 32
• Target Modules: q_proj, v_proj
• Training Epochs: 2
• Batch Size: 4 (effective: 32 with gradient accumulation)
• Learning Rate: 2e-4
• Sequence Length: 128 tokens

Performance Metrics

See evaluation_metrics.json for detailed scores:

• BLEU Score: 29.51
• chrF Score: 50.49
• Test Set Size: 1143 examples

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    device_map="auto",
    torch_dtype="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "thomasajai/llama-3.1-8b-egyptian-german")
tokenizer = AutoTokenizer.from_pretrained("thomasajai/llama-3.1-8b-egyptian-german")

# Generate translation
def translate(text):
    messages = [
        {"role": "system", "content": "Translate Ancient Egyptian transliteration to German."},
        {"role": "user", "content": text}
    ]
    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    outputs = model.generate(**inputs, max_new_tokens=64, temperature=0.1)
    return tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)

# Example
print(translate("ḥr.w n.ꜥ nṯr"))  # Output: "Horus Ninetjer"

Example Translations

Dataset

The model was trained on cleaned data from the Thesaurus Linguae Aegyptiae corpus, focusing on Earlier Egyptian texts.

Data preprocessing included:

• Removal of text-critical symbols and editorial markup
• Normalization of transliteration conventions
• Filtering of incomplete or damaged texts

Limitations

• Ancient Egyptian is a low-resource language with inherent translation ambiguity
• Model trained on formal/religious texts; may not generalize to all text types
• German translations reflect scholarly conventions and may vary from source
• Limited to transliterated text; does not work with hieroglyphic images

Intended Use

• Academic research in Egyptology
• Digital humanities projects
• Educational tools for learning Middle Egyptian
• Assisted translation for scholars (not replacement for expert translation)

Citation

If you use this model, please cite:

@misc{egyptian-german-llama,
  author = {Your Name},
  title = {Egyptian-German Translation Model},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/thomasajai/llama-3.1-8b-egyptian-german}}
}

License

This adapter is released under the same license as the base Llama-3.1 model. Please refer to Meta's license terms.

Acknowledgments

• Base model: Meta's Llama-3.1-8B-Instruct
• Dataset: Thesaurus Linguae Aegyptiae
• Training framework: Hugging Face Transformers + PEFT