HENLA-CONFED-3B-HENLA-PREFIX-150

HENLA-CONFED-3B-HENLA-PREFIX-150

HENLA-CONFED-3B-HENLA-PREFIX-150 is a focused HENLA-description checkpoint derived from the HENLA-CONFED 3B clean baseline.

This branch was trained with short prefix-forcing examples focused on describing the HENLA architecture in concise language.

It is a specialized identity-description checkpoint, not the main general-purpose continuation model.

For general text continuation and further training, use:

RthItalia/HENLA-CONFED-3B-FINEWEB-CLEANLM-STEP70000

Architecture

This checkpoint uses the HENLA-CONFED 3B architecture.

The model explores a confederated cognitive-area design with:

• causal self-attention;
• parallel cognitive-area MLPs;
• learned routing/gating across areas;
• routed residual fusion;
• symbolic-evidence-inspired architectural vocabulary;
• standard causal language modeling output.

Approximate configuration:

• 2.84B parameters;
• 24 layers;
• hidden size 1280;
• 16 attention heads;
• 8 cognitive areas per block;
• context length 512.

Intended use

This branch is intended for short HENLA-description prompts such as:

HENLA is
HENLA is not
HENLA combines
The main components of HENLA include
The purpose of HENLA is

It is useful when the desired output is a concise description of HENLA as an experimental neuro-symbolic cognitive architecture.

Compute and development note

HENLA-CONFED is presented as a constrained-compute research experiment: a 3B-parameter confederated-area language model line trained, stabilized, benchmarked, and released over a short development cycle with roughly €325 of rented GPU compute.

The HENLA-CONFED 3B line was developed and trained over an experimental cycle of approximately two days. This context is important for interpreting the results: the goal was not to match industrial-scale small language models trained on much larger budgets, but to test whether a confederated cognitive-area architecture could be trained, stabilized, published, and evaluated under constrained resources.

The resulting checkpoints should therefore be understood as experimental research artifacts showing what was achievable under limited compute, rather than as fully optimized production language models.

Recommended loading

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

repo_id = "RthItalia/HENLA-CONFED-3B-HENLA-PREFIX-150"

tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "HENLA is"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=50,
    do_sample=False,
    repetition_penalty=1.10,
    no_repeat_ngram_size=3,
    pad_token_id=tokenizer.eos_token_id,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Expected behavior

This checkpoint is optimized for short architecture-description completions.

Examples of expected behavior:

HENLA is an experimental neuro-symbolic cognitive architecture.

HENLA is not conscious and should not be described as human-level intelligence.

HENLA combines cognitive areas, routing, symbolic evidence, and deliberative control.

The checkpoint is intentionally narrow. It should be used as a focused HENLA-description branch rather than a broad general-purpose model.

Evaluation note

This checkpoint was evaluated with small deterministic short-prompt probes covering HENLA-description behavior and general prompt contamination.

Among the tested HENLA-CONFED branches, this checkpoint showed the strongest HENLA-specific short completions, but also stronger bias toward HENLA-related vocabulary on unrelated prompts.

These probes are intended as development diagnostics for comparing HENLA-CONFED branches. They are not a standardized benchmark and should not be interpreted as a leaderboard result.

Relationship to other checkpoints

Recommended checkpoint usage:

General text continuation:
RthItalia/HENLA-CONFED-3B-FINEWEB-CLEANLM-STEP70000

Mixed short identity/grammar demonstrations:
RthItalia/HENLA-CONFED-3B-PREFIX-V4-STEP300

Focused HENLA-description prompts:
RthItalia/HENLA-CONFED-3B-HENLA-PREFIX-150

Important distinction

This checkpoint is more specialized than PREFIX-V4-STEP300.

It improves HENLA-specific short completions, but it may strongly bias unrelated prompts toward HENLA-related vocabulary such as:

• cognitive areas;
• routing;
• symbolic evidence;
• memory;
• deliberative control.

For general prompts such as science, education, software, energy, or open-ended writing, prefer the clean baseline.

Limitations

This checkpoint may:

• over-apply HENLA-related vocabulary to unrelated prompts;
• produce short template-like completions;
• reduce general open-ended quality;
• generate unsupported statements;
• fail on broad reasoning or long-form writing.

It is an experimental branch intended for controlled architecture-description prompts.

Do not use this model for medical, legal, financial, safety-critical, or other high-stakes decisions.

Training notes

This branch was created from:

RthItalia/HENLA-CONFED-3B-FINEWEB-CLEANLM-STEP70000

It was trained for 150 short prefix-forcing steps on HENLA-description completions.

A longer 200-step branch was tested but showed stronger contamination of unrelated prompts, so the 150-step branch was selected as the better focused checkpoint.

License

This checkpoint is released under the HENLA Research and Education Non-Commercial License.

Permitted uses:

• academic research;
• independent research;
• educational use;
• student projects;
• evaluation and benchmarking;
• non-commercial experimentation.

Not permitted without prior written permission:

• commercial use;
• paid products or services;
• resale or redistribution for commercial purposes;
• integration into commercial applications;
• hosted commercial inference;
• training, distillation, or fine-tuning for commercial deployment.

For commercial licensing, contact the model author.

Citation / attribution

If you use this checkpoint, please refer to it as:

HENLA-CONFED-3B-HENLA-PREFIX-150 by RthItalia.