harmonic-gpt-128m-byte-agent-multiparty

Harmonic GPT 128M Byte Agent Multiparty

An experimental 127.6M-parameter causal byte Transformer trained to represent dialogue as a room of peer agents rather than a privileged user/assistant pair. Turns explicitly encode the speaker, addressee, audience, and participant registry.

The checkpoint starts from Transformer pretraining step 90,000 (5.898B ordered UTF-8 bytes) and then receives 500 completion-only SFT steps at peak LR 3e-5. Its mixture contains 25% No Robots Q&A, 25% everyday conversations, 10% practical dialogue, 5% reasoning, 25% MultiLIGHT multiparty dialogue, and 10% pretraining replay.

Evaluation snapshot

• Held-out ordinary-chat BPB: 1.244777 (starting value 1.294554)
• Held-out multiparty BPB: 1.216836 (starting value 1.382048)
• Matched base-corpus BPB: 0.950660 (starting value 0.940957)
• Response-only linguistic composite: 76.6
• 768-byte repetition probe: 0/6 detected loops under both greedy and T=0.8/top-k 40
• EOS: 5/6 greedy and 6/6 stochastic long-form probes

These are behavioral diagnostics, not claims of factual reliability. The model frequently gives incorrect or nonsensical answers and must not be used for consequential decisions.

Native byte framing

<|room|> scope=direct members=p0,p1
<|participants|>
p0=Agent.Ajax
p1=Agent.Steve
<|turn|> p0>p1 audience=p0,p1
Agent.Ajax:
Hello!
<|end|>
<|turn|> p1>p0 audience=p0,p1
Agent.Steve:

The learned response terminator is \n<|end|>\n. The included handler maps OpenAI-style user/assistant histories onto the two peer-agent slots while preserving turn boundaries.

The No Robots component makes this release CC BY-NC 4.0. MultiLIGHT is used through its public ShareGPT conversion and is primarily fantasy roleplay; the checkpoint should be treated as an exploratory group-dialogue experiment.