rwkv-novel-agent-g1h-13b-ctx10240

RWKV Novel Agent G1h 13B (10,240 context)

This is an experimental Chinese novel-agent supervised fine-tune of RWKV-7 G1h 13.3B. It targets novel continuation/instruction following and structured narrative tasks such as speaker attribution, narrative-type classification, and scene-boundary detection.

This release is the full-pass checkpoint after one nominal packed-data pass over the local packed training stream. Use it for research and evaluation rather than production.

Model details

The starting checkpoint was BlinkDL/rwkv7-g1/rwkv7-g1h-13.3b-20260710-ctx10240.pth. This repository does not currently include a Transformers config or Safetensors conversion.

Prompt format

Use the fine-tuning prompt format below. Keep the single spaces after role labels.

System: 你是一个中文小说归因辅助助手。只能从候选角色中选择，并只输出 JSON。

User: 候选角色：甲、乙
上下文：……

Assistant:

Generation should stop on token 0 (EOD) or the text sequence \n\nUser:. Consume the stop sequence instead of displaying it.

Inference

The official rwkv runtime can load this raw checkpoint directly. One GPU with about 30 GiB of available VRAM is recommended for BF16 inference.

python -m pip install "torch>=2.5" "rwkv==0.8.32" "tokenizers>=0.13.2" ninja

import os

os.environ["RWKV_V7_ON"] = "1"
os.environ["RWKV_JIT_ON"] = "1"
os.environ["RWKV_CUDA_ON"] = "1"

from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS

# The RWKV-v7 loader appends ".pth", so omit the suffix here.
model = RWKV(model="/path/to/rwkv-final", strategy="cuda bf16")
pipeline = PIPELINE(model, "rwkv_vocab_v20230424")

prompt = """System: 场景边界检测。仅输出 {"boundaries":[N]}

User: [P1] 雨停了。\n[P2] 三天后，他回到故乡。\n[P3] 他推开老宅的门。

Assistant:"""

args = PIPELINE_ARGS(
    temperature=0,
    top_p=0,
    alpha_frequency=0,
    alpha_presence=0,
    token_stop=[0],
    chunk_len=256,
)
text = pipeline.generate(prompt, token_count=128, args=args)
text = text.split("\n\nUser:", 1)[0]
print(text)

CUDA inference JIT-compiles the RWKV-7 extension on first use, so a working CUDA toolkit, C++ compiler, and ninja executable are required. The model path passed to RWKV must omit the .pth suffix.

For deterministic JSON-style evaluation, use greedy decoding: temperature=0, top_p=0, alpha_frequency=0, alpha_presence=0, chunk_len=256, stop token [0], and stop text \n\nUser:.

Training

The published checkpoint was produced in two stages:

The full-pass continuation used 6 GPUs, BF16, DeepSpeed ZeRO-2, global batch 12 sequences (122,880 tokens/update), assistant-only loss masking, constant learning rate 5e-7 after 10 warmup steps, weight decay 0.01, and gradient clipping 0.5.

The packed training stream contains 131,920 conversations and 413,123,655 tokens, of which 329,536,148 were marked as supervised assistant targets. Because packed sampling used the training script's magic-prime schedule, "one nominal pass" should not be read as every raw token being seen exactly once.

Training data was derived from mikuhhn1239/novel-agent-sft-dataset at revision 5d3040d21f51b3ce90b9396b058e552c47f43cd5 and includes continuation, instruction, attribution, narrative-analysis, and scene-boundary examples.

Evaluation

The checkpoint was evaluated with rwkv==0.8.32, PyTorch 2.7.1+cu128, BF16 CUDA inference, and greedy decoding on NVIDIA A800 80 GB GPUs. Prompts were rendered with the same local packing code used for training, ending at Assistant:. Invalid JSON counted as wrong or empty predictions.

Local held-out test split:

Local benchmark comparison

Additional full-pass metrics:

*The dataset README's published scene F1 is for its v4-590 evaluation setup, not this local merged v4/v4.1 held-out split, so it is a directional reference rather than a strict apples-to-apples baseline.

The last recorded training loss is not an evaluation metric and is intentionally omitted.

Intended use

• Research on Chinese fiction generation and continuation.
• Experiments in speaker attribution, narrative classification, and scene boundary detection.
• Further fine-tuning and evaluation of RWKV-7 recurrent inference.

Limitations and risks

• This is an experimental SFT checkpoint; output quality may be unstable.
• The configured 10,240-token training segment length is not evidence of reliable performance across that full length.
• Generated JSON may be malformed or violate the requested schema.
• The model has not been evaluated for factuality, bias, memorization, harmful content, or prompt-injection resistance.
• Training data contains third-party Chinese novel text. Per-book rights and provenance are not fully established, so users must assess copyright, privacy, and downstream-use obligations for their jurisdiction and use case.
• Do not rely on generated text for legal, medical, financial, safety-critical, or other high-impact decisions.

License and attribution

The upstream RWKV-7 G1h checkpoint is distributed under Apache-2.0, and this repository retains that model-repository license metadata. The dataset's Apache-2.0 label does not establish rights to every underlying novel. The data rights caveat above remains applicable to this derivative checkpoint.

RWKV implementation and base-model credit: BlinkDL/RWKV-LM and BlinkDL/rwkv7-g1.