InterSync_-_Mistral-7B-Instruct-v0.2-Function-Calling-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Mistral-7B-Instruct-v0.2-Function-Calling - GGUF

• Model creator: https://huggingface.co/InterSync/
• Original model: https://huggingface.co/InterSync/Mistral-7B-Instruct-v0.2-Function-Calling/

Original model description:

license: cc tags:

• function-call
• mistral

---

Fine-tuned Mistral 7B Instruct v0.2 with OpenAI Function Call Support

Finetuned version of Mistral-7B-Instruct-v0.2 to support direct function calling. This new capability aligns with the functionality seen in OpenAI's models, enabling Mistral 7B Instruct v0.2 to interact with external data sources and perform more complex tasks, such as fetching real-time information or integrating with custom databases for enriched AI-powered applications.

Features

• Direct Function Calls: Mistral 7B Instruct v0.2 now supports structured function calls, allowing for the integration of external APIs and databases directly into the conversational flow. This makes it possible to execute custom searches, retrieve data from the web or specific databases, and even summarize or explain content in depth.

Usage

Importing Libraries

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

Initializing Model and Tokenizer

device = "cuda" 

model = AutoModelForCausalLM.from_pretrained("InterSync/Mistral-7B-Instruct-v0.2-Function-Calling")
tokenizer = AutoTokenizer.from_pretrained("InterSync/Mistral-7B-Instruct-v0.2-Function-Calling")

Creating the Text Streamer

streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

Defining Tools

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "format": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The temperature unit to use. Infer this from the user's location.",
                    },
                },
                "required": ["location", "format"],
            },
        }
    }
]

Setting up the Messages

messages = [
    {
        "role": "user",
        "content": (
            "You are Mistral with function-calling supported. You are provided with function signatures within <tools></tools> XML tags. "
            "You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. "
            "Here are the available tools:\n"
            "<tools>\n"
            f"{tools}\n"
            "</tools>\n\n"
            "For each function call, return a JSON object with the function name and arguments within <tool_call></tool_call> XML tags as follows:\n"
            "<tool_call>\n"
            "{'arguments': <args-dict>, 'name': <function-name>}\n"
            "</tool_call>"
        )
    },
    {
        "role": "assistant",
        "content": "How can I help you today?"
    },
    {
        "role": "user",
        "content": "What is the current weather in San Francisco?"
    },
]

Preparing Model Inputs

inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
model_inputs = inputs.to(device)

Generating the Response

model.to(device)
generate_ids = model.generate(model_inputs, streamer=streamer, do_sample=True, max_length=4096)
decoded = tokenizer.batch_decode(generate_ids)

Expected Output

<tool_call>
{"arguments": {"location": "San Francisco, CA", "format": "celsius"}, "name": "get_current_weather"}
</tool_call>