mlc-chat-RWKV-4-World-3B-q4f16_2

mlc-chat iOS pre-built RWKV-4-World-3B

https://github.com/mlc-ai/mlc-llm https://mlc.ai/mlc-llm/docs/deploy/ios.html

RWKV-4-World的Hugface格式，因新版World的tokenizer较之前Raven\Pile版本有较大变化，因而需要进行新版HF适配 ringrwkv兼容了原生rwkv库和transformers的rwkv库，同时新添入World版本的配置及代码（支持1.5B，3B，7B全系列），并修复了原HF的RWKV在 Forward RWKVOutput时的细微问题，主要是引入和明确last_hidden_state。以下是轻量级使用代码，比较方便：<br>

RingRWKV GIT开源地址：https://github.com/StarRing2022/RingRWKV <br>

import torch<br> from ringrwkv.configuration_rwkv_world import RwkvConfig<br> from ringrwkv.rwkv_tokenizer import TRIE_TOKENIZER<br> from ringrwkv.modehf_world import RwkvForCausalLM<br>

model = RwkvForCausalLM.from_pretrained("StarRing2022/RWKV-4-World-3B") #或将本模型下载至本地文件夹<br> tokenizer = TRIE_TOKENIZER('./ringrwkv/rwkv_vocab_v20230424.txt')<br>

text = "你叫什么名字？"<br>

question = f'Question: {text.strip()}\n\nAnswer:'<br>

input_ids = tokenizer.encode(question)<br> input_ids = torch.tensor(input_ids).unsqueeze(0)<br>

out = model.generate(input_ids,max_new_tokens=40)<br><br>

outlist = out[0].tolist()<br>

for i in outlist:<br>     if i==0: #要删除tokenid为0的元素 <br>     outlist.remove(i)<br>

answer = tokenizer.decode(outlist)<br>

print(answer)<br>