pondllm-qwen3-0.6b-action-sft

PondLLM Qwen3-0.6B Action SFT Adapter

This is a small LoRA adapter trained to make Qwen/Qwen3-0.6B select structured actions for PondLLM, an experimental deterministic grid ecology. Organisms perceive only their local surroundings and can forage, move, rest, share, signal, reproduce, and die under scarce energy.

This repository now contains the V2 balanced-action adapter. It is a controlled follow-up to V1: the base model, simulator, prompts, LoRA configuration, optimizer, and seed were held fixed while rare-action training coverage was increased.

It is not a standalone language model. The frozen Qwen base model must be loaded and this adapter attached with PEFT. The simulator itself is not included in this repository.

Intended use

The adapter maps a structured local observation to one legal JSON action:

{"action":"move","target":[2,3]}
{"action":"forage"}
{"action":"share","target":"organism-00002","amount":1}
{"action":"signal","message":"short message"}
{"action":"rest"}
{"action":"reproduce","target":[2,3]}

It is intended for reproducing this bounded action-policy experiment and studying small-model behavior in simulated environments. It is not a general chat model or a policy for real-world autonomous decisions.

Loading the adapter

import json
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-0.6B"
adapter_id = "Xnizzorg/pondllm-qwen3-0.6b-action-sft"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

system_prompt = """You are one bounded organism in a resource ecology.
You know only the supplied local observation and your own memory. Never invent hidden world state.
Choose exactly one legal action and return only one JSON object.

Legal forms:
{"action":"move","target":[x,y]}
{"action":"forage"}
{"action":"share","target":"organism_id","amount":1}
{"action":"signal","message":"short message"}
{"action":"rest"}
{"action":"reproduce","target":[x,y]}

Move and reproduce targets must be listed in open_neighbors. Share targets must be visible agents.
Foraging only succeeds when current_food is positive. Energy is scarce and reaching zero is death."""

observation = {
    "tick": 4,
    "self": {
        "id": "organism-00001",
        "lineage": "lineage-01",
        "position": [2, 2],
        "energy": 7,
        "age": 4,
        "drives": {
            "reproduction_threshold": 16,
            "share_threshold": 12,
            "curiosity": 0.35,
            "sociality": 0.25,
            "fecundity": 0.2,
        },
    },
    "current_food": 1,
    "visible_food": [{"position": [3, 2], "amount": 1}],
    "visible_agents": [],
    "open_neighbors": [[1, 2], [3, 2], [2, 1], [2, 3]],
    "memory": [],
}

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": json.dumps(observation, separators=(",", ":"))},
]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=64,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )

generated = output[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))

For this observation, a legal response is:

{"action":"forage"}

V2 training data

All examples were generated inside the deterministic PondLLM simulator. No personal, user-conversation, or scraped private data was used.

The original dataset contained 11,614 records. For V2, unique examples of reproduce, share, and signal were generated from disjoint simulator seeds until each action reached 1,000 records.

Training configuration

Evaluation

The behavioral evaluation used greedy decoding on a deterministic held-out set containing 100 examples of each action.

V2 per-action recall:

In four fixed-seed, 40-tick worlds, V2 made 832/832 valid decisions, produced 12 births and 3 deaths, and preserved all 16 founder-lineage instances. This demonstrates a behavioral change, not general ecological superiority.

Limitations

• Signalling remains effectively unlearned under greedy decoding.
• Sharing improved substantially but remains inconsistent.
• One of 600 held-out actions attempted to share the organism's final unit of energy.
• Evaluation covers a synthetic simulator, a limited seed set, and one base model.
• The policy is prompt- and schema-specific.
• The adapter has not undergone preference optimization, reinforcement learning, or evolutionary weight updates.
• Survival or reproduction in a simulator is not evidence of sentience or subjective experience.

The next supervised experiment should target conditional communication while checking that V2's reproduction and sharing gains are preserved.

Version history

• V2 balanced action SFT — current default revision; rare actions topped up to 1,000 examples.
• V1 natural action SFT — preserved in this repository's commit history.

Reproducibility files

• adapter_model.safetensors: learned V2 LoRA weights
• adapter_config.json: PEFT configuration and base-model reference
• tokenizer.json, tokenizer_config.json, chat_template.jinja: tokenizer snapshot
• run_manifest.json: exact training configuration, metrics, and package versions

V2 adapter SHA-256:

64A24413099F71DEBC10AEE5CF863EC9A83C51C3B9EE8F4F1BDB76C96D3308DB

License and attribution

This adapter is released under the Apache License 2.0. It is derived from Qwen/Qwen3-0.6B, which is also distributed under the Apache License 2.0.

Training used Transformers, PEFT, and TRL.