Downloads · 30 days
17
2% of all-time downloads
pangu-350M
sunzeyeah/pangu-350M
pangu-350M is a text generation model from sunzeyeah. Use it when you need the model to write or continue text. It is set up for transformers.
All-time downloads
752
Public
Repo size
1.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin691 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gpt_pangu
- Access
- Public
- Created
- Apr 6, 2023
- Updated
- Apr 6, 2023
- SHA
- 8eaf114a
Model card
From the Hugging Face model README
Link to github: here
Model Description
Pangu-α is proposed by a joint technical team headed by PCNL. It was first released in this repository It is the first large-scale Chinese pre-trained language model with 200 billion parameters trained on 2048 Ascend processors using an automatic hybrid parallel training strategy. The whole training process is done on the “Peng Cheng Cloud Brain II” computing platform with the domestic deep learning framework called MindSpore. The PengCheng·PanGu-α pre-training model can support rich applications, has strong few-shot learning capabilities, and has outstanding performance in text generation tasks such as knowledge question and answer, knowledge retrieval, knowledge reasoning, and reading comprehension.
This repository contains PyTorch implementation of PanGu model with 350 million parameters pretrained weights (FP32 precision).
It is slightly different from the original pangu implementation to support the ChatGPT training pipeline in this github repo: sunzeyeah/RLHF.
- Task
- Text Generation
- Library
- transformers
- Type
- gpt_pangu
- Languages
- zh
- Created
- Apr 6, 2023
- Updated
- Apr 6, 2023