Downloads · 30 days
17
15% of all-time downloads
LLaMA-1B-dj-refine-100B
datajuicer/LLaMA-1B-dj-refine-100B
LLaMA-1B-dj-refine-100B is a text generation model from datajuicer. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Our first data-centric LLM competition begins! Please visit the competition's official websites, FT-Data Ranker (1B Track, 7B Track), for more information. This is a reference LLM from Data-Juicer.
All-time downloads
112
Public
Repo size
10.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin5.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Oct 30, 2023
- Updated
- Nov 10, 2023
- SHA
- 1bd6974a
Try a prompt
Datasets
- datajuicer/redpajama-wiki-refined-by-data-juicer
- datajuicer/redpajama-arxiv-refined-by-data-juicer
- datajuicer/redpajama-c4-refined-by-data-juicer
- datajuicer/redpajama-book-refined-by-data-juicer
- datajuicer/redpajama-cc-2019-30-refined-by-data-juicer
- datajuicer/redpajama-cc-2020-05-refined-by-data-juicer
- datajuicer/redpajama-cc-2021-04-refined-by-data-juicer
- datajuicer/redpajama-cc-2022-05-refined-by-data-juicer
Model card
From the Hugging Face model README
News
Our first data-centric LLM competition begins! Please visit the competition's official websites, FT-Data Ranker (1B Track, 7B Track), for more information.
Introduction
This is a reference LLM from Data-Juicer.
The model architecture is LLaMA-1.3B and we adopt the OpenLLaMA implementation. The model is pre-trained on 100B tokens of Data-Juicer's refined RedPajama and Pile. It achieves an average score of 33.07 over 16 HELM tasks, beating LLMs trained on original RedPajama and Pile datasets.
For more details, please refer to our paper.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Created
- Oct 30, 2023
- Updated
- Nov 10, 2023