Downloads · 30 days
44
20% of all-time downloads
google-gemma-10B-ko-chang
lcw99/google-gemma-10B-ko-chang
google-gemma-10B-ko-chang is a text generation model from lcw99. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
The Gemma Self-Attention Merged model is a large language model created by merging the self-attention layers of an English-based Gemma 7B model and a Korean-based Gemma 7B model. This merger allows the model to levera…
All-time downloads
218
Public
Parameters
9.9B
19.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors19.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- gemma
- Access
- Public
- Created
- Apr 19, 2024
- Updated
- Apr 20, 2024
- SHA
- 1f52b21f
Model card
From the Hugging Face model README
Model Card for Model ID
Model Details
Model Description
The Gemma Self-Attention Merged model is a large language model created by merging the self-attention layers of an English-based Gemma 7B model and a Korean-based Gemma 7B model. This merger allows the model to leverage the capabilities of both the English and Korean models, resulting in a more versatile and capable language model that can perform well on tasks involving both English and Korean text.
The key features of this merged model include:
- Increased self-attention capacity with doubled number of attention heads
- Ability to handle both English and Korean language input
- Potential for improved performance on a wide range of natural language processing tasks
Chat template
system: system message...
B: user message...
A: assistant message...
Model Sources
- Repository: https://github.com/lcw99/merge-gemma-attn.git
- Task
- Text Generation
- Library
- transformers
- Type
- gemma
- License
- apache-2.0
- Languages
- ko
- Created
- Apr 19, 2024
- Updated
- Apr 20, 2024