A1.1

schneewolflabs/A1.1

A1.1 is a text generation model from schneewolflabs. Use it when you need the model to write or continue text. It is set up for transformers.

Downloads · 30 days

21% of all-time downloads

All-time downloads

Public

Parameters

12.2B

24.5 GB on disk

Likes

Public

Repo makeup

Click a slice to open those files.

.safetensors24.5 GB · 100%

From the Hugging Face model README

Parameter	Value
Training Mode	SFT
Base Model	`schneewolflabs/A1`
Learning Rate	2e-05
Epochs	1
Batch Size	2
Gradient Accumulation	16
Effective Batch Size	32
Max Sequence Length	8192
Optimizer	paged_adamw_8bit
LR Scheduler	cosine
Warmup Ratio	0.05
Weight Decay	0.01
Max Grad Norm	0.5
Seed	42
LoRA Rank (r)	128
LoRA Alpha	256
LoRA Dropout	0.05
Target Modules	up_proj, down_proj, gate_proj, k_proj, q_proj, v_proj, o_proj
Quantization	4-bit (NF4)
GPU	NVIDIA GB10

Trained on 3 concatenated datasets:

Trained with Merlina