Downloads · 30 days
23
2% of all-time downloads
tigerbot-7b-sft-v1-4bit
TigerResearch/tigerbot-7b-sft-v1-4bit
tigerbot-7b-sft-v1-4bit is a text generation model from TigerResearch. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<div style="width: 100%;" <img src="https://github.com/TigerResearch/TigerBot/blob/main/image/logocore.png" alt="TigerBot" style="width: 20%; display: block; margin: auto;" </div <p align="center" <font face="黑体" size…
All-time downloads
1.2K
Public
Repo size
5.2 GB
Likes
6
Public
Repo makeup
Click a slice to open those files.
.pt5.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- bloom
- Access
- Public
- Created
- Jun 1, 2023
- Updated
- Aug 10, 2023
- SHA
- 20d1f052
Try a prompt
Model card
From the Hugging Face model README
<div style="width: 100%;">
<img src="https://github.com/TigerResearch/TigerBot/blob/main/image/logo_core.png" alt="TigerBot" style="width: 20%; display: block; margin: auto;">
</div>
<p align="center">
<font face="黑体" size=5"> A cutting-edge foundation for your very own LLM. </font>
</p>
<p align="center">
🌐 <a href="https://tigerbot.com/" target="_blank">TigerBot</a> • 🤗 <a href="https://huggingface.co/TigerResearch" target="_blank">Hugging Face</a>
</p>
This is a 4-bit GPTQ version of the Tigerbot 7B sft.
It was quantized to 4bit using: https://github.com/TigerResearch/TigerBot/tree/main/gptq
How to download and use this model in github: https://github.com/TigerResearch/TigerBot
Here are commands to clone the TigerBot and install.
conda create --name tigerbot python=3.8
conda activate tigerbot
conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
git clone https://github.com/TigerResearch/TigerBot
cd TigerBot
pip install -r requirements.txt
Inference with command line interface
cd TigerBot/gptq
CUDA_VISIBLE_DEVICES=0 python tigerbot_infer.py TigerResearch/tigerbot-7b-sft-4bit-128g --wbits 4 --groupsize 128 --load TigerResearch/tigerbot-7b-sft-4bit-128g/tigerbot-7b-4bit-128g.pt
- Task
- Text Generation
- Library
- transformers
- Type
- bloom
- License
- apache-2.0
- Created
- Jun 1, 2023
- Updated
- Aug 10, 2023