AIGym_-_Llama-3-8B-Instruct-Gradient-1048k-Agent-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3-8B-Instruct-Gradient-1048k-Agent - GGUF

• Model creator: https://huggingface.co/AIGym/
• Original model: https://huggingface.co/AIGym/Llama-3-8B-Instruct-Gradient-1048k-Agent/

Original model description:

library_name: transformers license: apache-2.0

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

This model is being made to enhance our work within crewai. We started with a high context length (1048K) version of Llama 3. We then fine-tuned on top of that to get a base agent.

Model Description

Built on the following:

• Models:
  • Meta-Llama-3-8B-Instruct
  • Llama-3-8B-Instruct-Gradient-1048k
• Datasets:
  • m-a-p/CodeFeedback-Filtered-Instruction
  • RomanTeucher/awesome_topic_code_snippets
  • dair-ai/emotion
  • mzbac/function-calling-llama-3-format-v1.1
  • gretelai/synthetic_text_to_sql

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: [More Information Needed]
• Model type: [More Information Needed]
• License: [More Information Needed]
• **Finetuned from model [Llama-3-8B-Instruct-Gradient-1048k]

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

Uses

Use inside with crewai to get better results and to generaete starting data.

Direct Use

Use as a chat bot, ai agent, etc.

Out-of-Scope Use

Anything outside of using with crewai while vary possiable will be out of scope.

Recommendations

Self host or host this model in the cloud and use with crew ai for best results.

Code Examples

Use the following format when using the model for inference:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are the helpful assistant. <|eot_id|><|start_header_id|>user<|end_header_id|>

{prompt} <|eot_id|><|start_header_id|>assistant<|end_header_id|>

Example of multi turn

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful AI assistant for travel tips and recommendations<|eot_id|><|start_header_id|>user<|end_header_id|>

What is France's capital?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Bonjour! The capital of France is Paris!<|eot_id|><|start_header_id|>user<|end_header_id|>

What can I do there?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Paris, the City of Light, offers a romantic getaway with must-see attractions like the Eiffel Tower and Louvre Museum, romantic experiences like river cruises and charming neighborhoods, and delicious food and drink options, with helpful tips for making the most of your trip.<|eot_id|><|start_header_id|>user<|end_header_id|>

Give me a detailed list of the attractions I should visit, and time it takes in each one, to plan my trip accordingly.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

How to Get Started with the Model

Use the code below to get started with the model.

import transformers
import torch

model_id = "meta-llama/Meta-Llama-3-70B-Instruct"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]

prompt = pipeline.tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
)

terminators = [
    pipeline.tokenizer.eos_token_id,
    pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = pipeline(
    prompt,
    max_new_tokens=256,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])