yi-alpaca-2-34b-swahili

yi-alpaca-2-34b-swahili

Released with the paper "Global Gallery: The Fine Art of Painting Culture Portraits through Multilingual Instruction Tuning" (NAACL 2024).

Paper (NAACL 2024) | Project page | Code

This repo holds PEFT (QLoRA) adapter weights only. The base model is 01-ai/Yi-34B, and the adapter was instruction-tuned on the Swahili translation of the cleaned Alpaca dataset (iamshnoo/alpaca-cleaned-swahili).

Training procedure

The following bitsandbytes quantization config was used during training:

• load_in_8bit: False
• load_in_4bit: True
• llm_int8_threshold: 6.0
• llm_int8_skip_modules: None
• llm_int8_enable_fp32_cpu_offload: False
• llm_int8_has_fp16_weight: False
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: True
• bnb_4bit_compute_dtype: bfloat16

Framework versions

• PEFT 0.4.0

Citation

@inproceedings{mukherjee-etal-2024-global,
    title = "Global Gallery: The Fine Art of Painting Culture Portraits through Multilingual Instruction Tuning",
    author = "Mukherjee, Anjishnu  and
      Caliskan, Aylin  and
      Zhu, Ziwei  and
      Anastasopoulos, Antonios",
    editor = "Duh, Kevin  and
      Gomez, Helena  and
      Bethard, Steven",
    booktitle = "Proceedings of the 2024 Conference of the {N}orth {A}merican Chapter of the {A}ssociation for {C}omputational {L}inguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = jun,
    year = "2024",
    address = "Mexico City, Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.naacl-long.355/",
    doi = "10.18653/v1/2024.naacl-long.355",
    pages = "6398--6415"
}