macadeliccc_-_laser-polyglot-4x7b-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

laser-polyglot-4x7b - GGUF

• Model creator: https://huggingface.co/macadeliccc/
• Original model: https://huggingface.co/macadeliccc/laser-polyglot-4x7b/

Original model description:

language:

• ja
• en
• zh license: apache-2.0 model-index:
• name: laser-polyglot-4x7b results:
  • task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics:
    • type: acc_norm value: 64.16 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-polyglot-4x7b name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics:
    • type: acc_norm value: 84.98 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-polyglot-4x7b name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics:
    • type: acc value: 63.88 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-polyglot-4x7b name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics:
    • type: mc2 value: 55.47 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-polyglot-4x7b name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics:
    • type: acc value: 77.82 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-polyglot-4x7b name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics:
    • type: acc value: 48.45 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=macadeliccc/laser-polyglot-4x7b name: Open LLM Leaderboard

---

Polyglot-4x7b-24b

Polyglot-4x7b is a Mixture of Experts approach to a multilingual model.

This project is an experiment to see if each expert can be of a different language. The answer is yes.

The model is a merge of models that are capable of Chinese and Japanese output.

• teknium/OpenHermes-2.5-Mistral-7B
• oshizo/japanese-e5-mistral-7b_slerp
• cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser
• s3nh/Mistral-7B-Evol-Instruct-Chinese

TODO:

1. [] polyglot tokenizer

Other polyglot models

• macadeliccc/Polyglot-8x7b-v0.1 (adds 3 more languages)

Code Example

Inference Colab Live demo available on Spaces

from transformers import AutoModelForCausalLM, AutoTokenizer

def generate_response(prompt):
    """
    Generate a response from the model based on the input prompt.

    Args:
    prompt (str): Prompt for the model.

    Returns:
    str: The generated response from the model.
    """
    # Tokenize the input prompt
    inputs = tokenizer(prompt, return_tensors="pt")

    # Generate output tokens
    outputs = model.generate(**inputs, max_new_tokens=256, eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.pad_token_id)

    # Decode the generated tokens to a string
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)

    return response

# Load the model and tokenizer
model_id = "macadeliccc/laser-polyglot-4x7b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True)

# Example prompts in different languages
english_prompt = "Write a quicksort algorithm in python"
chinese_prompt = "用Python写一个快速排序算法"
japanese_prompt = "Pythonでクイックソートアルゴリズムを書いてください"

# Generate and print responses for each language
print("English Response:")
print(generate_response(english_prompt), "\n")

print("Chinese Response:")
print(generate_response(chinese_prompt), "\n")

print("Japanese Response:")
print(generate_response(japanese_prompt), "\n")

Example Output

English:

Write a quicksort algorithm in python.

def quicksort(arr):
    if len(arr) <= 1:
        return arr
    else:
        pivot = arr[0]
        less = [i for i in arr[1:] if i <= pivot]
        greater = [i for i in arr[1:] if i > pivot]
        return quicksort(less) + [pivot] + quicksort(greater)

arr = [5, 2, 9, 1, 5, 7, 4, 8, 6, 3]
print(quicksort(arr))

This is a simple implementation of the quicksort algorithm in python. The function quicksort takes an array as input and returns a sorted array. The algorithm works by selecting a pivot element from the array and partitioning the other elements into two sub-arrays, according to whether they are less than or greater than the pivot. The process is then repeated recursively on the sub-arrays until the entire array is sorted.

Chinese Response:

用Python写一个快速排序算法

def quick_sort(arr):
    if len(arr) <= 1:
        return arr
    else:
        pivot = arr[0]
        less = [i for i in arr[1:] if i <= pivot]
        greater = [i for i in arr[1:] if i > pivot]
        return quick_sort(less) + [pivot] + quick_sort(greater)

arr = [3, 5, 2, 1, 4, 6, 8, 7]
print(quick_sort(arr))

这个程序的时间复杂度为O(nlogn)，空间复杂度为O(n)。

Japanese Response:

Pythonでクイックソートアルゴリズムを書いてください。

def quicksort(arr):
    if len(arr) <= 1:
        return arr
    pivot = arr[0]
    left = [x for x in arr[1:] if x < pivot]
    right = [x for x in arr[1:] if x >= pivot]
    return quicksort(left) + [pivot] + quicksort(right)

print(quicksort([3,6,8,10,1,5,9,2,4,7]))

このコードはクイックソートアルゴリズムを実装しています。クイックソートは一種の分割と conquers アルゴリズムで、配列を分割し、それぞれの部分配列を再帰的にソートします。

この実装では、配列の最初の要素をピボットとして使用します。そして、配列を2つの

Evaluations

Open LLM Leaderboard Evaluation Results

Detailed results can be found here