Downloads · 30 days
27
2% of all-time downloads
tigerbot-13b-chat-8bit
TigerResearch/tigerbot-13b-chat-8bit
tigerbot-13b-chat-8bit is a text generation model from TigerResearch. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<div style="width: 100%;" <img src="http://x-pai.algolet.com/bot/img/logocore.png" alt="TigerBot" style="width: 20%; display: block; margin: auto;" </div <p align="center" <font face="黑体" size=5" A cutting-edge founda…
All-time downloads
1.1K
Public
Repo size
29.7 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors15.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Aug 10, 2023
- Updated
- Aug 31, 2023
- SHA
- e9e1b286
Try a prompt
Model card
From the Hugging Face model README
<div style="width: 100%;">
<img src="http://x-pai.algolet.com/bot/img/logo_core.png" alt="TigerBot" style="width: 20%; display: block; margin: auto;">
</div>
<p align="center">
<font face="黑体" size=5"> A cutting-edge foundation for your very own LLM. </font>
</p>
<p align="center">
🌐 <a href="https://tigerbot.com/" target="_blank">TigerBot</a> • 🤗 <a href="https://huggingface.co/TigerResearch" target="_blank">Hugging Face</a>
</p>
This is a 8-bit GPTQ version of the Tigerbot 13b chat.
It was quantized to 8bit using: https://github.com/PanQiWei/AutoGPTQ
How to download and use this model in github: https://github.com/TigerResearch/TigerBot
Here are commands to clone the TigerBot and install.
conda create --name tigerbot python=3.8
conda activate tigerbot
conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
git clone https://github.com/TigerResearch/TigerBot
cd TigerBot
pip install -r requirements.txt
Inference with command line interface
# 安装auto-gptq
pip install auto-gptq
# 启动推理
CUDA_VISIBLE_DEVICES=0 python other_infer/gptq_infer.py --model_path TigerResearch/tigerbot-13b-chat-8bit
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Created
- Aug 10, 2023
- Updated
- Aug 31, 2023