Hemlock2-Coder-7B

hemlang/Hemlock2-Coder-7B

Hemlock2-Coder-7B is a text generation model from hemlang. Use it when you need the model to write or continue text. It is set up for transformers.

Downloads · 30 days

5% of all-time downloads

All-time downloads

472

Public

Parameters

7.6B

15.2 GB on disk

Likes

Public

Repo makeup

Click a slice to open those files.

.safetensors15.2 GB · 100%

From the Hugging Face model README

image/png

Parameter	Value
Training Mode	ORPO
Base Model	`Qwen/Qwen2.5-Coder-7B-Instruct`
Learning Rate	9e-05
Epochs	2
Batch Size	2
Gradient Accumulation	8
Effective Batch Size	16
Max Sequence Length	2048
Optimizer	paged_adamw_8bit
LR Scheduler	cosine
Warmup Ratio	0.05
Weight Decay	0.01
Max Grad Norm	0.3
Seed	42
Beta	0.1
Max Prompt Length	1024
LoRA Rank (r)	128
LoRA Alpha	64
LoRA Dropout	0.05
Target Modules	up_proj, down_proj, gate_proj, k_proj, q_proj, v_proj, o_proj
Quantization	4-bit (NF4)
GPU	NVIDIA RTX A6000

Trained with Merlina