sethuiyer_-_Chikuma_10.7B_v2-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Chikuma_10.7B_v2 - GGUF

• Model creator: https://huggingface.co/sethuiyer/
• Original model: https://huggingface.co/sethuiyer/Chikuma_10.7B_v2/

Original model description:

license: apache-2.0 library_name: transformers tags:

• dpo datasets:
• argilla/distilabel-intel-orca-dpo-pairs base_model: sethuiyer/Chikuma_10.7B pipeline_tag: text-generation model-index:
• name: distilabled_Chikuma_10.7B results:
  • task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics:
    • type: acc_norm value: 66.38 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/distilabled_Chikuma_10.7B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics:
    • type: acc_norm value: 85.14 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/distilabled_Chikuma_10.7B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics:
    • type: acc value: 64.7 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/distilabled_Chikuma_10.7B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics:
    • type: mc2 value: 59.2 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/distilabled_Chikuma_10.7B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics:
    • type: acc value: 79.4 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/distilabled_Chikuma_10.7B name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics:
    • type: acc value: 58.38 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=sethuiyer/distilabled_Chikuma_10.7B name: Open LLM Leaderboard

---

Chikuma_10.7B - V2 (Enhanced with DPO) [For Experiments]

<p align="center"> <img src="https://huggingface.co/sethuiyer/distilabled_Chikuma_10.7B/resolve/main/chikuma_v2.webp" height="256px" alt="Chikuma"> </p>

This model is the DPO fine tuned version of Chikuma_10.7B, which was a depth upscaled merge of:

• sethuiyer/SynthIQ-7b
• openchat/openchat-3.5-0106

The name "Chikuma" is inspired by the Chikuma River, the longest in Japan, known for its continuous flow and meandering path. This metaphorically represents the model's depth, fluidity, and adaptability in processing and understanding language.

Dataset used for Fine Tuning

Dataset: /argilla/distilabel-intel-orca-dpo-pairs

The dataset was roughly ~3000 samples but they were high quality (according to the chosen_score).

The following filters were applied to the original dataset:

dataset = dataset.filter(
    lambda r:
        r["status"] != "tie" and
        r["chosen_score"] >= 8 and
        not r["in_gsm8k_train"]
)

Chat Template

The chat template for Chikuma_10.7B - V2 is a modified version of ChatML, optimized for improved interaction and engagement:

<|im_start|>GPT4 Correct system:
{system} Always use <|end_of_turn|> when you want to end the answer. <|im_end|>
<|im_start|>GPT4 Correct user:
{user}<|im_end|>
<|im_start|>GPT4 Correct Assistant:
{asistant}<|im_end|>

Nous Benchmark Evaluation

OpenLLM Leaderboard

Training Environment

• Hardware: Single A100 80GB GPU in a runpod, utilized for approximately 1.5 hours.
• Training Script: Accessible via Google Colab Notebook. Special thanks to mlabonne for providing the template.

Usage

# Format prompt
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(new_model)

# Create pipeline
pipeline = transformers.pipeline(
    "text-generation",
    model=new_model,
    tokenizer=tokenizer,
    device="cuda"
)

# Generate text

message = [
    {"role": "system", "content": "You are a helpful assistant chatbot."},
    {"role": "user", "content": "Who invented LLMs?"}
]

prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)

sequences = pipeline(
    prompt,
    max_new_tokens=512
)
print(sequences[0]['generated_text'])

Acknowledgements

A heartfelt appreciation goes to the vibrant open-source community, particularly:

• The Intel team for publishing a great open dataset and show how well it worked in the first place
• Teknium and NousResearch for their awesome work and models.
• Maxime for sharing such great resources.
• Argilla for publishing argilla/distilabel-intel-orca-dpo-pairs

Open LLM Leaderboard Evaluation Results

Detailed results can be found here