Downloads · 30 days
0
rwkv-7b-jp-lora
OpenMOSE/rwkv-7b-jp-lora
rwkv-7b-jp-lora is a machine learning model from OpenMOSE. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Access
Public
Updated Jun 10, 2023
Repo size
14.8 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.pth14.8 GB · 100%
At a glance
- Access
- Public
- Created
- Jun 6, 2023
- Updated
- Jun 10, 2023
- SHA
- 7c063ea2
Model card
From the Hugging Face model README
license: apache-2.0
a LoRA fine-tuned model with Japanese dataset
LoRA Experiment
rwkv-7b-jp-280.pth is merged model with base
Base Model
https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main
RWKV-4-Raven-7B-v10-Eng89%25-Jpn10%25-Other1%25-20230420-ctx4096.pth
Parameters: Lora Rank 1024 Lora Alpha 2048 ctx length 1024 Lora Size: almost 1.6GB
Dataset
https://huggingface.co/datasets/kunishou/hh-rlhf-49k-ja
https://huggingface.co/datasets/kunishou/databricks-dolly-15k-ja
- Created
- Jun 6, 2023
- Updated
- Jun 10, 2023