Downloads · 30 days
15
0% of all-time downloads
yunsung-llama-2-koen-13b-linkbricks-sft-basic-v1
Saxo/yunsung-llama-2-koen-13b-linkbricks-sft-basic-v1
yunsung-llama-2-koen-13b-linkbricks-sft-basic-v1 is a text generation model from Saxo. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<div align="center" <img src="http://www.linkbricks.com/wp-content/uploads/2024/11/fulllogo.png" / </div
All-time downloads
36.5K
Public
Parameters
13.2B
26.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors26.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Mar 12, 2024
- Updated
- Dec 5, 2024
- SHA
- f3f27136
Model card
From the Hugging Face model README
Model Card for Model ID
<div align="center"> <img src="http://www.linkbricks.com/wp-content/uploads/2024/11/fulllogo.png" /> </div>AI 와 빅데이터 분석 전문 기업인 Linkbricks의 데이터사이언티스트인 지윤성 박사(Saxo)가 beomi/llama-2-koen-13b 베이스모델을 GCP상의 A100-40G 4개를 통해 SFT 훈련을 한(2048 Tokens) 인스트럭션 모델. Accelerate, Deepspeed Zero-3 라이브러리를 사용했으며 Flash Attention 은 Disable 로 설정
Dr. Yunsung Ji (Saxo), a data scientist at Linkbricks, a company specializing in AI and big data analytics, trained the beomi/llama-2-koen-13b base model on 4 A100-40Gs on GCP for 4 hours of instructional training (2048 Tokens). Accelerate, Deepspeed Zero-3 libraries were used.
www.linkbricks.com, www.linkbricks.vc
Configuration including BitsandBytes
learning_rate = 2e-4 num_epochs = 5 batch_size = 4 block_size = 2048 trainer = "sft" warmup_ratio = 0.1 weight_decay = 0.01 gradient_accumulation = 4 mixed_precision = "fp16" peft = True quantization = "int4" lora_r = 64 lora_alpha = 16 lora_dropout = 0.1 model_max_length = 2048
Dataset Format
Alpaca Format Prompt Text
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- ko, en
- Created
- Mar 12, 2024
- Updated
- Dec 5, 2024