belt_road_hungarian_beta2

Belt_Road_Hungarian

Model Description

This model is a conversational and instruction-following large language model, fine-tuned from the foundational open-source Qwen2.5-72B-Instruct model using supervised fine-tuning (SFT).

---

Key Features & Use Cases

• Exceptional Hungarian Language Proficiency: The model has been deeply optimized for Hungarian, demonstrating excellent fluency, accuracy, and a strong understanding of cultural context in conversations.
• Multilingual Translation and Dialogue: With extensive training data that includes Hungarian, English, and Chinese content, the model excels in translation, multilingual Q&A, and cross-language communication.
• Advanced Instruction Following: The model shows a strong ability to comprehend and execute complex instructions, including those with multiple steps and specific constraints.
• Creative Content Generation: It is highly suitable for a wide range of creative tasks, such as writing articles, reports, scripts, and marketing copy.

---

System Requirements

Hardware

• GPU VRAM: For BF16/FP16 inference (recommended), at least 4 x NVIDIA A100 (80GB) GPUs are required. The model weights alone are approximately 136GB, so device_map="auto" is necessary to distribute them across multiple cards.
• System RAM: A minimum of 200GB is recommended.

Software

• Python: Version 3.10 or higher.
• Key Libraries:
  • torch: 2.1 or higher
  • transformers: 4.41.0 or higher
  • accelerate: 1.7.0 or higher
  • einops: 0.8.1 or higher
  • sentencepiece: 0.2.0 or higher

---

How to Use

The recommended method for loading and running the model is by using the transformers library.

# Example code snippet for inference using transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Specify the model path or your Hugging Face Hub repository
model_path = "your-huggingface-repo/your-model-name" # e.g., "your-user/your-qwen-model"

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_path)

# Load the model with device_map to distribute it across available GPUs
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16, # or torch.float16
    device_map="auto"
)

# Example conversation prompt
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello, can you translate 'hello' to Hungarian and Chinese?"}
]

# Apply the chat template and generate a response
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.7,
    top_p=0.9
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)