talkie-1930-13b-base-vllm

talkie-1930-13b-base (vLLM-servable repackage)

This is a HuggingFace + vLLM-ready repackage of talkie-lm/talkie-1930-13b-base. The original release ships as a raw torch checkpoint (final.ckpt) plus a tiktoken vocab (vocab.txt), with no config.json, tokenizer.json, or HF modeling code, so it can't be loaded by transformers or served by vLLM out of the box.

talkie-1930-13b-base is a 13B parameter base (completion) language model from the talkie-lm project, pretrained on ~260B tokens of pre-1931 English text. This is the base model — it is not instruction-tuned and has no chat template; prompt it with raw text and let it continue.

What this repo adds

The architecture and tokenizer BPE ranks are identical to the IT repackage; the only differences are vocab_size (65536 vs 65540), the absence of the 4 chat special tokens, and no chat template.

Serving with vLLM

vllm serve agaralon/talkie-1930-13b-base-vllm \
    --model-impl transformers \
    --trust-remote-code \
    --dtype bfloat16 \
    --max-model-len 4096

Then hit it as an OpenAI-style completions endpoint (base model — use /v1/completions, not chat):

curl http://localhost:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "agaralon/talkie-1930-13b-base-vllm",
        "prompt": "In the year 1925, the great city of London",
        "temperature": 0.8,
        "max_tokens": 120
    }'

Sampling notes

• Use temperature ≥ 0.5 — greedy decoding (temperature=0) can collapse into single-token loops on this architecture.
• bf16 only — fp8 is broken on this architecture.

Plain HuggingFace usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tok = AutoTokenizer.from_pretrained("agaralon/talkie-1930-13b-base-vllm", trust_remote_code=True)
m = AutoModelForCausalLM.from_pretrained(
    "agaralon/talkie-1930-13b-base-vllm", trust_remote_code=True, dtype=torch.bfloat16,
).cuda().eval()

ids = tok(["In the year 1925, the great city of London"], return_tensors="pt").to("cuda")
out = m.generate(**ids, max_new_tokens=120, do_sample=True, temperature=0.8, top_p=0.9,
                 pad_token_id=tok.pad_token_id, eos_token_id=tok.eos_token_id)
print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))

Provenance

• Weights: final.ckpt from talkie-lm/talkie-1930-13b-base, with lm_head_gain.w_g baked into lm_head.weight and cast to bf16.
• Tokenizer: built from vocab.txt from the same release (ranks < 65535, then <|endoftext|> at id 65535).
• Modeling code: adapted from the IT repackage awilliamson/talkie-1930-13b-it-vllm.

License

Apache 2.0, matching the upstream talkie-lm/talkie-1930-13b-base release.