Felladrin_-_Llama-160M-Chat-v1-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-160M-Chat-v1 - GGUF

• Model creator: https://huggingface.co/Felladrin/
• Original model: https://huggingface.co/Felladrin/Llama-160M-Chat-v1/

Original model description:

language:

• en license: apache-2.0 tags:
• text-generation base_model: JackFram/llama-160m datasets:
• ehartford/wizard_vicuna_70k_unfiltered
• totally-not-an-llm/EverythingLM-data-V3
• Open-Orca/SlimOrca-Dedup
• databricks/databricks-dolly-15k
• THUDM/webglm-qa widget:
  • messages:
    • role: system content: You are a helpful assistant, who answers with empathy.
    • role: user content: Got a question for you!
    • role: assistant content: "Sure! What's it?"
    • role: user content: Why do you love cats so much!? 🐈
  • messages:
    • role: system content: "You are a helpful assistant who answers user's questions with empathy."
    • role: user content: Who is Mona Lisa?
  • messages:
    • role: system content: You are a helpful assistant who provides concise responses.
    • role: user content: Heya!
    • role: assistant content: Hi! How may I help you today?
    • role: user content: I need to build a simple website. Where should I start learning about web development?
  • messages:
    • role: user content: Invited some friends to come home today. Give me some ideas for games to play with them!
  • messages:
    • role: system content: "You are a helpful assistant who answers user's questions with details and curiosity."
    • role: user content: What are some potential applications for quantum computing?
  • messages:
    • role: system content: You are a helpful assistant who gives creative responses.
    • role: user content: Write the specs of a game about mages in a fantasy world.
  • messages:
    • role: system content: "You are a helpful assistant who answers user's questions with details."
    • role: user content: Tell me about the pros and cons of social media.
  • messages:
    • role: system content: "You are a helpful assistant who answers user's questions with confidence."
    • role: user content: What is a dog?
    • role: assistant content: 'A dog is a four-legged, domesticated animal that is a member of the class Mammalia, which includes all mammals. Dogs are known for their loyalty, playfulness, and ability to be trained for various tasks. They are also used for hunting, herding, and as service animals.'
    • role: user content: What is the color of an apple? inference: parameters: max_new_tokens: 250 penalty_alpha: 0.5 top_k: 4 repetition_penalty: 1.01 model-index:
• name: Llama-160M-Chat-v1 results:
  • task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics:
    • type: acc_norm value: 24.74 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics:
    • type: acc_norm value: 35.29 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics:
    • type: acc value: 26.13 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics:
    • type: mc2 value: 44.16 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics:
    • type: acc value: 51.3 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics:
    • type: acc value: 0.0 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Felladrin/Llama-160M-Chat-v1 name: Open LLM Leaderboard

---

A Llama Chat Model of 160M Parameters

• Base model: JackFram/llama-160m
• Datasets:
  • ehartford/wizard_vicuna_70k_unfiltered
  • totally-not-an-llm/EverythingLM-data-V3
  • Open-Orca/SlimOrca-Dedup
  • databricks/databricks-dolly-15k
  • THUDM/webglm-qa
• Availability in other ML formats:
  • GGUF: Felladrin/gguf-Llama-160M-Chat-v1
  • ONNX: Felladrin/onnx-Llama-160M-Chat-v1
  • MLC: Felladrin/mlc-q4f16-Llama-160M-Chat-v1
  • MLX: mlx-community/Llama-160M-Chat-v1-4bit-mlx

Recommended Prompt Format

<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant

Recommended Inference Parameters

penalty_alpha: 0.5
top_k: 4
repetition_penalty: 1.01

Usage Example

from transformers import pipeline

generate = pipeline("text-generation", "Felladrin/Llama-160M-Chat-v1")

messages = [
    {
        "role": "system",
        "content": "You are a helpful assistant who answers user's questions with details and curiosity.",
    },
    {
        "role": "user",
        "content": "What are some potential applications for quantum computing?",
    },
]

prompt = generate.tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

output = generate(
    prompt,
    max_new_tokens=1024,
    penalty_alpha=0.5,
    top_k=4,
    repetition_penalty=1.01,
)

print(output[0]["generated_text"])

Open LLM Leaderboard Evaluation Results

Detailed results can be found here