Downloads · 30 days
4
7% of all-time downloads
jfaltermeier/llama3-theia-workshop-johannes-with-config
llama3-theia-workshop-johannes-with-config is a text generation model from jfaltermeier. Use it when you need the model to write or continue text. It is set up for transformers.
Downloads · 30 days
4
7% of all-time downloads
All-time downloads
55
Public
Repo size
32.6 GB
Likes
0
Public
Click a slice to open those files.
.bin16.1 GB · 97%
From the Hugging Face model README
The repo currently consists out of
Use Node >= 20 with npm >= 10.
npm ci
Set OPEN_API_KEY in your environment variables.
You can set an arbitrary value like foobar in case you don't intend to use Open AI's GPT models, e.g. export OPEN_API_KEY=foobar.
Configure the models to chat with in bots.config.json
npm run build
npm run start
Open app at localhost:5173.
The Evaluation App works against Open AI's API.
We recommend vllm for deploying own models.
A simple configuration may look like this:
vllm/vllm-openai:latest--model mistralai/Mistral-7B-Instruct-v0.1.
HUGGING_FACE_HUB_TOKEN to your pod with your token800050 GB/root/.cache/huggingface.
Use this Runpod link to start with a configuration for Mistral-7B-Instruct-v0.2 model. You can use "Edit Pod Template" to adjust the template before using it.
Once the pod is started the first time, it will get a random id assigned by Runpod, e.g. g9q3ycbfk2yorr.
Configure the pod in bots.config.json
id must be unique between podstype: runpodmodelId must be the same as used in the Container Start Command aboverunpodId is the id assigned by RunpodIn case of Mistral based models, disable the system prompt with systemPrompt: null as these models don't support it.