Downloads · 30 days
35
2% of all-time downloads
SuperCOT-L2-13B-GGUF
Peeepy/SuperCOT-L2-13B-GGUF
SuperCOT-L2-13B-GGUF is a text generation model from Peeepy. Use it when you need the model to write or continue text. It is set up for transformers.
GGUF of a merged checkpoint 4320 ausboss/llama2-13b-supercot-loras2 with base Llama 2 13B. It is currently only quantised to Q5KM as this is the smallest size with comparable accuracy to 8bit (almost lossless). I have…
All-time downloads
2.1K
Public
Repo size
9.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf9.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Sep 14, 2023
- Updated
- Sep 14, 2023
- SHA
- 24a0c62a
Try a prompt
Model card
From the Hugging Face model README
GGUF of a merged checkpoint 4320 ausboss/llama2-13b-supercot-loras2 with base Llama 2 13B. It is currently only quantised to Q5_K_M as this is the smallest size with comparable accuracy to 8bit (almost lossless). I have a fp16 GGUF and will probably quant to 8bit and 4bit GGUF soon.
Ausboss' original model card with the LoRA training info. See his model page for further information.
Training procedure
The following bitsandbytes quantization config was used during training:
quant_method: bitsandbytes
load_in_8bit: False
load_in_4bit: True
llm_int8_threshold: 6.0
llm_int8_skip_modules: None
llm_int8_enable_fp32_cpu_offload: False
llm_int8_has_fp16_weight: False
bnb_4bit_quant_type: nf4
bnb_4bit_use_double_quant: True
bnb_4bit_compute_dtype: bfloat16
Framework versions
PEFT 0.6.0.dev0
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Sep 14, 2023
- Updated
- Sep 14, 2023