Downloads · 30 days
150
2% of all-time downloads
RichardErkhov/pszemraj_-_stablelm-4e1t-2b-v0.1-gguf
pszemraj_-_stablelm-4e1t-2b-v0.1-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
150
2% of all-time downloads
All-time downloads
8.9K
Public
Repo size
23.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf26.8 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
stablelm-4e1t-2b-v0.1 - GGUF
| Name | Quant method | Size |
|---|---|---|
| stablelm-4e1t-2b-v0.1.Q2_K.gguf | Q2_K | 0.74GB |
| stablelm-4e1t-2b-v0.1.IQ3_XS.gguf | IQ3_XS | 0.81GB |
| stablelm-4e1t-2b-v0.1.IQ3_S.gguf | IQ3_S | 0.85GB |
| stablelm-4e1t-2b-v0.1.Q3_K_S.gguf | Q3_K_S | 0.85GB |
| stablelm-4e1t-2b-v0.1.IQ3_M.gguf | IQ3_M | 0.89GB |
| stablelm-4e1t-2b-v0.1.Q3_K.gguf | Q3_K | 0.94GB |
| stablelm-4e1t-2b-v0.1.Q3_K_M.gguf | Q3_K_M | 0.94GB |
| stablelm-4e1t-2b-v0.1.Q3_K_L.gguf | Q3_K_L | 1.01GB |
| stablelm-4e1t-2b-v0.1.IQ4_XS.gguf | IQ4_XS | 1.03GB |
| stablelm-4e1t-2b-v0.1.Q4_0.gguf | Q4_0 | 1.08GB |
| stablelm-4e1t-2b-v0.1.IQ4_NL.gguf | IQ4_NL | 1.09GB |
| stablelm-4e1t-2b-v0.1.Q4_K_S.gguf | Q4_K_S | 1.09GB |
| stablelm-4e1t-2b-v0.1.Q4_K.gguf | Q4_K | 1.14GB |
| stablelm-4e1t-2b-v0.1.Q4_K_M.gguf | Q4_K_M | 1.14GB |
| stablelm-4e1t-2b-v0.1.Q4_1.gguf | Q4_1 | 1.19GB |
| stablelm-4e1t-2b-v0.1.Q5_0.gguf | Q5_0 | 1.3GB |
| stablelm-4e1t-2b-v0.1.Q5_K_S.gguf | Q5_K_S | 1.3GB |
| stablelm-4e1t-2b-v0.1.Q5_K.gguf | Q5_K | 1.33GB |
| stablelm-4e1t-2b-v0.1.Q5_K_M.gguf | Q5_K_M | 1.33GB |
| stablelm-4e1t-2b-v0.1.Q5_1.gguf | Q5_1 | 1.41GB |
| stablelm-4e1t-2b-v0.1.Q6_K.gguf | Q6_K | 1.53GB |
| stablelm-4e1t-2b-v0.1.Q8_0.gguf | Q8_0 | 1.98GB |
license: cc-by-sa-4.0 base_model: stabilityai/stablelm-3b-4e1t tags:
This is a layer pruning experiment based off of stablelm-3b-4e1t:
axolotl version: 0.4.0
base_model: pszemraj/stablelm-3b-4e1t-prune10
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
strict: false
seed: 80085
# dataset
datasets:
- path: BEE-spoke-data/KI-smorgasbord_fw-small
type: completion # format from earlier
field: text # Optional[str] default: text, field to use for completion data
val_set_size: 0.015
sequence_len: 4096
sample_packing: true
pad_to_sequence_len: false
train_on_inputs: false
group_by_length: false
# WANDB
wandb_project: llama3-pruning
wandb_entity: pszemraj
wandb_watch: gradients
wandb_name: stablelm-4e1t-2b-v0.1
hub_model_id: pszemraj/stablelm-4e1t-2b-v0.1
hub_strategy: every_save
gradient_accumulation_steps: 16
micro_batch_size: 1
num_epochs: 2
optimizer: adamw_torch_fused # paged_adamw_32bit
weight_decay: 0.05
lr_scheduler: cosine
learning_rate: 5e-5
warmup_ratio: 0.1
load_in_8bit: false
load_in_4bit: false
bf16: true
tf32: true
flash_attention: true
torch_compile: true # requires >= torch 2.0, may sometimes cause problems
torch_compile_backend: inductor # Optional[str]
gradient_checkpointing: true
gradient_checkpointing_kwargs:
use_reentrant: false
# hyperparams for freq of evals, saving, etc
evals_per_epoch: 5
saves_per_epoch: 3
save_safetensors: true
save_total_limit: 1
output_dir: ./output-axolotl/output-model-2b
logging_steps: 8
deepspeed:
special_tokens:
pad_token: <|end_of_text|>
</details><br>
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 0.0006 | 1 | 4.4344 |
| 2.6558 | 0.2004 | 332 | 2.7150 |
| 2.6548 | 0.4007 | 664 | 2.6196 |
| 2.5435 | 0.6011 | 996 | 2.5981 |
| 2.5133 | 0.8014 | 1328 | 2.5502 |
| 2.489 | 1.0018 | 1660 | 2.5106 |
| 2.2671 | 1.1898 | 1992 | 2.4944 |
| 2.2038 | 1.3902 | 2324 | 2.4843 |
| 2.2513 | 1.5905 | 2656 | 2.4761 |
| 2.1654 | 1.7909 | 2988 | 2.4769 |