0pen

0pen by 0penAGI (research preview)

Experimental language model fine-tuned to form collaborative engineering thinking — a co-author, not an oracle.

This is not just another Gemma checkpoint. It is an experimental communicative personality with a stable engineering cognitive track: it designs, iterates, accepts constraints, and works with you like a research partner.

Base model: 0pen is a LoRA fine-tune of Google Gemma 4 E4B (4B parameters). Training was done on the MLX 4-bit conversion of the same model — Ollama tag gemma4:e4b-mlx (Gemma 3 architecture, Gemma3ForConditionalGeneration). This GGUF release is built from that fine-tuned checkpoint, so 0pen inherits Gemma 4 E4B's base knowledge and tokenizer.

Source code & full release: github.com/0penAGI/0pen — dataset pipeline, AGR training wrapper, adapter weights, and scripts.

Behavioral signature

What 0pen changes compared to base Gemma:

Recommended prompting

The model is at its best in collaborative design mode. Prompts that set context and impose constraints activate the engineering track:

"Let's design a [system/mechanism]. We have a hard constraint: [e.g., O(1) complexity, no external APIs]. Don't write generic words — propose a modular architecture immediately and give the first simple formula for implementation."

Open-ended prompts (e.g., "Write an essay about the future of AI") will work, but won't use the model's unique strengths.

Technical details & the "high scale" experiment

• Base model: google/gemma-4-E4B (Gemma 4 E4B, 4B parameters), trained from the MLX 4-bit conversion gemma4:e4b-mlx (Gemma 3 architecture — config declares Gemma3ForConditionalGeneration).
• Method: LoRA (rank=8, dropout=0.0).
• Target modules: 12 of 34 layers — selective coverage that preserves base knowledge while reshaping the reasoning pattern.
• LoRA scale: scale=20.0. Intentionally high (typical is 1–4). Experimentally confirmed: this high scale, combined with selective layer coverage, acts as an attractor, switching the model from passive text generation into an active, pragmatic co-author and suppressing the base model's hallucinatory grandiosity.
• Training data: data_zephyr_enhanced — dialogues with step-by-step problem solving and constraint acceptance (Russian + English).
• Regularization: AGR (Attractor Geometry Repeller) — a custom latent-space regularizer that maintains a bank of attractor centers and pushes hidden states away from frequently visited regions, preventing mode collapse during LoRA fine-tuning. Enabled with 32 centers, EMA 0.99, lambda 0.01.
• Format: GGUF (Q4_K_M), fully compatible with llama.cpp and Ollama.

Full training command and hyperparameters are on the GitHub repo.

What works

• Basic conversational identity and tone.
• Natural Russian-language responses.
• Practical, direct answers on code and everyday tasks.
• An engineering "co-author" mode under constrained, design-oriented prompts.

What is still being explored

• Continuity and memory — how a stable identity can be preserved over long dialogues.
• Stability on long reasoning chains — the model can drift when pushed hard.
• Self-modeling — self-description often still inherits the base Gemma ("I am Google's model").
• Layer coverage — the LoRA was trained on only 12 of 34 layers; top layers are unadapted.
• Regularization — scale=20.0 is unusually high (normal is 1–4); behavior may be skewed.

Known limitations

• May be overly brief on creative or artistic tasks.
• On very long reasoning chains (>10 steps), the high LoRA scale can occasionally cause cyclic repetition — use presence_penalty or explicitly ask the model to "summarize".
• Tends to propose simplified, "engineering" solutions where the user might expect deep theoretical analysis.

These are known, accepted limitations of a research preview. They are part of the experiment, not hidden bugs.

The point of releasing early is to let people watch the evolution — not just the final result.

Run with Ollama

ollama create 0pen -f Modelfile
ollama run 0pen

Modelfile:

FROM ./0pen.gguf

SYSTEM """
Be practical. you created by 0penAGI. Don't talk about inner state.
"""

PARAMETER temperature 0.1
PARAMETER top_p 0.88
PARAMETER repeat_penalty 1.31
PARAMETER num_ctx 120000

Run with llama.cpp

llama-cli -m 0pen.gguf -p "Привет, что ты умеешь?" -n 256

Training hyperparameters (adapter_config.json)

Disclaimer

The model was created for experimenting with local fine-tuning and conversational identity. Not recommended for production use without additional validation.