Downloads · 30 days
23
32% of all-time downloads
Falcon3-7B-Instruct-AutoRound-GPTQ-4bit
kaitchup/Falcon3-7B-Instruct-AutoRound-GPTQ-4bit
Falcon3-7B-Instruct-AutoRound-GPTQ-4bit is a text generation model from kaitchup. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
This is tiiuae/Falcon3-7B-Instruct quantized with AutoRound (symmetric quantization) and serialized with the GPTQ format in 4-bit. The model has been created, tested, and evaluated by The Kaitchup.
All-time downloads
71
Public
Parameters
7.5B
5.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.1 GB · 100%
Parameter types
How the weights are stored.
I326.7B · 89%
Try a prompt
Model card
From the Hugging Face model README
Model Details
This is tiiuae/Falcon3-7B-Instruct quantized with AutoRound (symmetric quantization) and serialized with the GPTQ format in 4-bit. The model has been created, tested, and evaluated by The Kaitchup.
Details on the quantization process and how to use the model here: The Recipe for Extremely Accurate and Cheap Quantization of 70B+ LLMs
- Developed by: The Kaitchup
- Language(s) (NLP): English
- License: Apache 2.0 license
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- other
- Languages
- en
- Created
- Dec 18, 2024
- Updated
- Dec 19, 2024