Mistral-7b-Pathway-128k-3

Pseudo-Deterministic Chatbot with Mistral 7B

Overview

This repository contains a fine-tuned version of the Mistral 7B model, specifically designed for creating pseudo-deterministic chatbots. The goal of this project is to enhance the predictability and consistency of chatbot responses while maintaining the flexibility and adaptability of the Mistral 7B model.

Features

• Fine-tuned Mistral 7B Model: Leveraging the power of the Mistral 7B, our model is fine-tuned to offer more deterministic responses, ensuring consistency in conversational contexts.
• Scalable Hugging Face Endpoint: We provide a handler script for deploying the chatbot model on a scalable endpoint using Hugging Face's infrastructure. This setup is ideal for handling varying loads with efficient resource management. This can be deployed for public, protected, or private use with ASW privatelink. This handler script can also be used to serve the model on custom hardware.
• Gradio Interface: A Gradio demo is included, offering a user-friendly interface to interact with the chatbot. This demo can connect not only to our provided backend but also to any alternative backend setup.

Getting Started

1. Deploying the Model: You can deploy from the model repo (here) by clicking "Deploy" in the upper right corner. or with Inference Endpoints SDK

2. Running the Gradio Demo: You can deploy directly from hugginface or with Python:

import gradio as gr

gr.load("models/codys12/Mistral-7b-Pathway-128k-3").launch()

You can embed the space with the URL found in the upper right of the space with "Share"

<iframe
	src="https://your.hf.space"
	frameborder="0"
	width="850"
	height="450"
></iframe>

Usage

• Model Interaction:

def  generate(
message: str,
chat_history: list[tuple[str, str]],#Conversation history
system_prompt: str  =  "",
instruction: str  =  None,#The goal of the current conversation
conclusions: list[tuple[str, str]] =  None,#AI classification of conversation ending
#^ Formatted: [["CONCLUSION_KEY","Conclusion criteria"]]
context: list[str] =  None,#List of strings to be used as context. Indexes that were used will be returned.
max_new_tokens: int  =  1024,#Max new tokens to generate
temperature: float  =  0.6,#Temperature hyperparameter
top_p: float  =  0.9,#Top-p hyperparameter
top_k: int  =  50, #Top-k hyperparameter
repetition_penalty: float  =  1.2, #Repitition hyperparameter
end_sequences: list[str] = ["[INST]", "[/INST]", "\n"]#Sequences that break the generation and return

• Customization: conversation topics and their possible ansers/paths are stored in topics.json. You can freely change this to fit a desired use case.

License

This project is licensed under the Apache License, Version 2.0 - see the LICENSE file for details.