heegyu

KoLIMA-5.8b

heegyu/KoLIMA-5.8b

KoLIMA-5.8b is a text generation model from heegyu. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

polyglot-ko-5.8b를 lora로 학습 후 가중치 병합. ko-lima 데이터 사용. 10에폭 1e-4 - 1e-5 cosine decay. 배치 128, 최대시퀀스길이 2048