Downloads · 30 days
8
7% of all-time downloads
Aeonium-v1-Base-1.6B-checkpoint-20B
aeonium/Aeonium-v1-Base-1.6B-checkpoint-20B
Aeonium-v1-Base-1.6B-checkpoint-20B is a text generation model from aeonium. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A state-of-the-art language model for Russian language processing. This checkpoint contains a preliminary version of the model with 1.6 billion parameters. Trained only on web pages.
All-time downloads
116
Public
Parameters
1.6B
16.4 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors6.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- May 2, 2024
- Updated
- May 5, 2024
- SHA
- de0f1a71
Model card
From the Hugging Face model README
Aeoinum v1 BaseWeb 1B
A state-of-the-art language model for Russian language processing. This checkpoint contains a preliminary version of the model with 1.6 billion parameters. Trained only on web pages.
Models
| Name | N of parameters | N of dataset tokens | Context window |
|---|---|---|---|
| Aeonium-v1-BaseWeb-1B | 1.6B | 32B | 4K |
| Aeonium-v1-Base-1B | 1.6B | In training | 4K |
| Aeonium-v1-Chat-1B | 1.6B | In training | 4K |
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("aeonium/Aeonium-v1-Base-1.6B-checkpoint-20B")
model = AutoModelForCausalLM.from_pretrained("aeonium/Aeonium-v1-Base-1.6B-checkpoint-20B").cuda()
input_ids = tokenizer("Искусственный интеллект - это", return_tensors='pt').to(model.device)["input_ids"]
output = model.generate(input_ids, max_new_tokens=48, do_sample=True, temperature=0.7)
print(tokenizer.decode(output[0]))
Output:
Искусственный интеллект - это основа современной науки и техники. Его потенциал позволяет решать задачи, которые выходят за пределы человеческих возможностей. В работе над ними участвуют все: от ученых до инженеров и даже военных. В своей книге "Искусственный интеллект" автор книги, профессор Л
Dataset Detail
The dataset for pre-training is collected from public data, most of which are web pages in Russian. The total size of the data is 20B tokens.
Training Detail
The training is performed thanks to a grant from TPU Research Cloud on a TPU v4-32 node.
Copyright
The model is released under the Apache 2.0 license.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- ru
- Created
- May 2, 2024
- Updated
- May 5, 2024