CohereForAI_aya-expanse-32b-exl2

exl2 quant (measurement.json in main branch)

---

check revisions for quants

---

Model Card for Aya-Expanse-32B

<img src="aya-expanse-32B.png" width="650" style="margin-left:'auto' margin-right:'auto' display:'block'"/>

Aya Expanse is an open-weight research release of a model with highly advanced multilingual capabilities. It focuses on pairing a highly performant pre-trained Command family of models with the result of a year’s dedicated research from Cohere For AI, including data arbitrage, multilingual preference training, safety tuning, and model merging. The result is a powerful multilingual large language model serving 23 languages.

We cover 23 languages: Arabic, Chinese (simplified & traditional), Czech, Dutch, English, French, German, Greek, Hebrew, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian, and Vietnamese

This model card corresponds to the 32-billion version of the Aya Expanse model. We also released an 8-billion version which you can find here.

• Developed by: Cohere For AI
• Point of Contact: Cohere For AI: cohere.for.ai
• License: CC-BY-NC, requires also adhering to C4AI's Acceptable Use Policy
• Model: Aya Expanse 32B
• Model Size: 32 billion parameters

Try Aya Expanse

Before downloading the weights, you can try out Aya Expanse (32B) in our hosted Hugging Face Space.

Usage

Please install transformers from the source repository.

# pip install 'git+https://github.com/huggingface/transformers.git'
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "CohereForAI/aya-expanse-32b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Format message with the chat template
messages = [{"role": "user", "content": "Anneme onu ne kadar sevdiğimi anlatan bir mektup yaz"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Anneme onu ne kadar sevdiğimi anlatan bir mektup yaz<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>

gen_tokens = model.generate(
    input_ids, 
    max_new_tokens=100, 
    do_sample=True, 
    temperature=0.3,
    )

gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)

Example Notebooks

Fine-Tuning:

• This notebook showcases a detailed use of fine-tuning Aya Expanse on more languages.

Example Use cases:

The following notebooks contributed by Cohere For AI Community members show how Aya Expanse can be used for different use cases:

• Mulitlingual Writing Assistant
• AyaMCooking
• Multilingual Question-Answering System

Model Details

Input: Models input text only.

Output: Models generate text only.

Model Architecture: Aya Expanse 32B is an auto-regressive language model that uses an optimized transformer architecture. Post-training includes supervised finetuning, preference training, and model merging.

Languages covered: The model is particularly optimized for multilinguality and supports the following languages: Arabic, Chinese (simplified & traditional), Czech, Dutch, English, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian, and Vietnamese

Context length: 128K

Evaluation

<img src="winrates_marenahard.png" width="650" style="margin-left:'auto' margin-right:'auto' display:'block'"/>

Model Card Contact

For errors or additional questions about details in this model card, contact [email protected].

Terms of Use

We hope that the release of this model will make community-based research efforts more accessible, by releasing the weights of a highly performant multilingual model to researchers all over the world. This model is governed by a CC-BY-NC License with an acceptable use addendum, and also requires adhering to C4AI's Acceptable Use Policy.

Try the model today

You can try Aya Expanse in the Cohere playground here. You can also use it in our dedicated Hugging Face Space here.