Downloads · 30 days
0
readflow-rwkv-4-world-ctx32k
xiaol/readflow-rwkv-4-world-ctx32k
readflow-rwkv-4-world-ctx32k is a machine learning model from xiaol. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is full finetuned model from RWKV 4 world 7B CHNTuned model using data from Readflow tech (readflow.com.cn) ,
Access
Public
Updated Jul 21, 2023
Repo size
15 GB
Likes
12
Public
Repo makeup
Click a slice to open those files.
.pth15 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jul 20, 2023
- Updated
- Jul 21, 2023
- SHA
- 2507c2b4
Model card
From the Hugging Face model README
This is full finetuned model from RWKV 4 world 7B CHNTuned model using data from Readflow tech (readflow.com.cn) ,
finetuned for 32k context length, used to summary news article
using inf-ctx training https://github.com/SynthiaDL/TrainChatGalRWKV with fixed VRAM
you can test summary prompt using RWKV runner(https://github.com/josStorer/RWKV-Runner) in chat mode , and check conversation files in examples folders.
这是和会读Readflow合作的模型（ readflow.com.cn），用于超长微信文章的摘要，训练了32k的context长度，能一次性输入整篇文章进行摘要，如下图以及在example文件夹中的例子，
23k tokens一次性输入摘要，不增显存，如果有state状态，可以秒出，可以反复重试多种可能性，
“Assistant:"是模型输出内容
可以使用runner进行测试 https://github.com/josStorer/RWKV-Runner
RWKV world的token对中文以及各类语言效率很高，token和字比例，基本是1:1,甚至1:n，欢迎测试，并加入微调QQ群讨论439087067
- License
- apache-2.0
- Created
- Jul 20, 2023
- Updated
- Jul 21, 2023