Downloads · 30 days
325
10% of all-time downloads
ToRoLaMa-7B-GGUF
ndthai/ToRoLaMa-7B-GGUF
ToRoLaMa-7B-GGUF is a machine learning model from ndthai. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as llama2.
- Author: All By AI - Original model: ToRoLaMa 7b v1.0
All-time downloads
3.3K
Public
Repo size
52.2 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf52.2 GB · 100%
At a glance
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Jan 9, 2024
- Updated
- Jan 10, 2024
- SHA
- eb2073d0
Base models
Model card
From the Hugging Face model README
ToRoLaMa-7B-GGUF
- Author: All By AI
- Original model: ToRoLaMa 7b v1.0
Description
GGUF format model files for ToRoLaMa 7b v1.0.
Prompt template: (Chat Format)
Cuộc hội thoại giữa người dùng và một trí thông minh nhân tạo. Đưa ra câu trả lời chính xác, giúp ích cho người dùng.
USER: chào
ASSISTANT: Xin chào! Tôi có thể giúp gì cho bạn hôm nay?</s>
USER:...
Quantisation methods (using llama.cpp b1796)
- GGML_TYPE_Q2_K - "type-1" 2-bit quantization in super-blocks containing 16 blocks, each block having 16 weight. Block scales and mins are quantized with 4 bits. This ends up effectively using 2.5625 bits per weight (bpw)
- GGML_TYPE_Q3_K - "type-0" 3-bit quantization in super-blocks containing 16 blocks, each block having 16 weights. Scales are quantized with 6 bits. This end up using 3.4375 bpw.
- GGML_TYPE_Q4_K - "type-1" 4-bit quantization in super-blocks containing 8 blocks, each block having 32 weights. Scales and mins are quantized with 6 bits. This ends up using 4.5 bpw.
- GGML_TYPE_Q5_K - "type-1" 5-bit quantization. Same super-block structure as GGML_TYPE_Q4_K resulting in 5.5 bpw
- GGML_TYPE_Q6_K - "type-0" 6-bit quantization. Super-blocks with 16 blocks, each block having 16 weights. Scales are quantized with 8 bits. This ends up using 6.5625 bpw
How to use GGUF files
Note: Don't clone the entire repo! Only pick and download a single file.
Automatically download models using:
- LM Studio
- LoLLMS Web UI
- Faraday.dev
- Library
- transformers
- Type
- llama
- License
- llama2
- Languages
- vi, en
- Created
- Jan 9, 2024
- Updated
- Jan 10, 2024