Llama-3.1-Tango-8b-int4

Uploaded model

• Developed by: sandbox-ai
• License: apache-2.0
• Finetuned from model : unsloth/meta-llama-3.1-8b-bnb-4bit

Evaluation Metrics

Usage:

You can use the model using HuggingFace Transformers library with 2 or more 80GB GPUs (NVIDIA Ampere or newer) with at least 150GB of free disk space to accomodate the download.

This code has been tested on Transformers v4.44.0, torch v2.4.0 and 2 A100 80GB GPUs, but any setup that supports meta-llama/Llama-3.1-70B-Instruct should support this model as well. If you run into problems, you can consider doing pip install -U transformers.

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sandbox-ai/Llama-3.1-Tango-8b-int4")

References(s):

• TODO

Model Architecture:

Architecture Type: Transformer <br> Network Architecture: Llama 3.1 <br>

Input:

Input Type(s): Text <br> Input Format: String <br> Input Parameters: One Dimensional (1D) <br> Other Properties Related to Input: Max of 128k tokens<br>

Output:

Output Type(s): Text <br> Output Format: String <br> Output Parameters: One Dimensional (1D) <br> Other Properties Related to Output: Max of 4k tokens <br>

Training & Evaluation:

• TODO

Dataset:

MessIRve: A Large-Scale Spanish Information Retrieval Dataset <br>

• spanish/-ir/messirve <br> ** messi_mod-v0.0.2 tatakof/messi_mod-v0.0.2

Citation

@article{valentini2024messirve,
      title={MessIRve: A Large-Scale Spanish Information Retrieval Dataset}, 
      author={Francisco Valentini and Viviana Cotik and Damián Furman and Ivan Bercovich and Edgar Altszyler and Juan Manuel Pérez},
      year={2024},
      eprint={2409.05994},
      journal={arxiv:2409.05994},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2409.05994}, 
}

@misc{wang2024helpsteer2preferencecomplementingratingspreferences,
      title={HelpSteer2-Preference: Complementing Ratings with Preferences}, 
      author={Zhilin Wang and Alexander Bukharin and Olivier Delalleau and Daniel Egert and Gerald Shen and Jiaqi Zeng and Oleksii Kuchaiev and Yi Dong},
      year={2024},
      eprint={2410.01257},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2410.01257}, 
}

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)