Arcee-VyLinh-imatrix-GGUF

Quantizations of https://huggingface.co/arcee-ai/Arcee-VyLinh

Inference Clients/UIs

• llama.cpp
• KoboldCPP
• ollama
• text-generation-webui
• GPT4All
• jan

---

From original readme

Arcee-VyLinh is a 3B parameter instruction-following model specifically optimized for Vietnamese language understanding and generation. Built through an innovative training process combining evolved hard questions and iterative Direct Preference Optimization (DPO), it achieves remarkable performance despite its compact size.

Model Details

• Architecture: Based on Qwen2.5-3B
• Parameters: 3 billion
• Context Length: 32K tokens
• Training Data: Custom evolved dataset + ORPO-Mix-40K (Vietnamese)
• Training Method: Multi-stage process including EvolKit, proprietary merging, and iterative DPO
• Input Format: Supports both English and Vietnamese, optimized for Vietnamese

Intended Use

• Vietnamese language chat and instruction following
• Text generation and completion
• Question answering
• General language understanding tasks
• Content creation and summarization

Performance and Limitations

Strengths

• Exceptional performance on complex Vietnamese language tasks
• Efficient 3B parameter architecture
• Strong instruction-following capabilities
• Competitive with larger models (4B-8B parameters)

Benchmarks

Tested on Vietnamese subset of m-ArenaHard (CohereForAI), with Claude 3.5 Sonnet as judge:

Limitations

• Might still hallucinate on cultural-specific content.
• Primary focus on Vietnamese language understanding
• May not perform optimally for specialized technical domains

Training Process

Our training pipeline consisted of several innovative stages:

1. Base Model Selection: Started with Qwen2.5-3B
2. Hard Question Evolution: Generated 20K challenging questions using EvolKit
3. Initial Training: Created VyLinh-SFT through supervised fine-tuning
4. Model Merging: Proprietary merging technique with Qwen2.5-3B-Instruct
5. DPO Training: 6 epochs of iterative DPO using ORPO-Mix-40K
6. Final Merge: Combined with Qwen2.5-3B-Instruct for optimal performance

Usage Examples

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained("arcee-ai/Arcee-VyLinh")
tokenizer = AutoTokenizer.from_pretrained("arcee-ai/Arcee-VyLinh")

prompt = "Một cộng một bằng mấy?"
messages = [
    {"role": "system", "content": "Bạn là trợ lí hữu ích."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=1024,
    eos_token_id=tokenizer.eos_token_id,
    temperature=0.25,
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids)[0]
print(response)