Lfm2-MaleCNS-Titans-VL-3B-V2-HashHop

LFM2–MaleCNS–Titans VL 3B V2 — HashHop memory training

Repository: win10/Lfm2-MaleCNS-Titans-VL-3B-V2-HashHop. Checkpoint ID: 0b5ec95eac3849e38867dcba9048c623. This repository contains a complete merged model, including the native vision/language backbone and custom dual-memory runtime.

The two training stages are separately versioned: visual/agent SFT and subsequent HashHop memory learning. A linked stage may still be in progress; only a populated checkpoint repository represents a completed upload.

Completed HashHop stage

After the separately published SFT, this checkpoint received 32 native optimizer updates over 256 synthetic shuffled-hash examples (one epoch, physical batch 8, outer LoRA rank 8 / alpha 16). Native language/vision/output parameters are frozen in this continuation; memory interfaces, graph/FFN priors, VAE and controller adapters are trainable. The native Trainer and optimizer remain in LlamaFactory. This is a small memory-learning experiment, not evidence of maximum capacity or a finished long-context curriculum.

Training uses disjoint random graphs with 1/2/4/8-hop trace and final-value tasks and 8/16-character ASCII-letter hashes. All source pairs are observed before any question or answer is supplied. The source KV cache is discarded. A fresh query then reads VAE posterior reconstructions plus the updated Titans and physical FFN weights. Raw source embeddings and exact correction patches are excluded from this training read; production archives retain their full correction data. No runtime dictionary or external hash-chain solver supplies answers.

The model-owned objective is L_query + 0.25 L_physical_only + 0.1 L_contrast + 0.05 L_dream. Contrast compares correct memory with empty and unrelated memory, with gradients through both sides. Dream losses train existing feature/weight VAE heads. Source/query isolation and nonzero Titans, FFN and Dream gradients are checked in the initial two real optimizer updates.

The task follows Magic’s HashHop description and reference generator, with explicitly local arrow notation, native chat serialization and disk-memory controls. This is not a reproduction of a 100-million-token benchmark.

Merge verification: 424 effective tensors audited; probe logit KL 0.00035086, maximum absolute difference 0.12500000. BF16 merge is not claimed to preserve every prediction exactly.

Measured memory and generation behavior

The primary result uses the model’s own memory routing. Oracle-unit and source-visible controls are diagnostics. Forty-two held-out hash questions include unseen graphs, 10-hop extrapolation, and cross-unit chains. Complete generated outputs are retained, including errors and repetitions. These development tests informed implementation repairs; they are not an untouched final benchmark.

The suite also produced 32 multi-turn coherence outputs; 53 outputs across the full suite reached their configured limit. Exact hash scores do not certify general coherent generation. Repetition and invented details were observed in SFT diagnostics; reliable multi-hop recall and universal factual consistency are not established.

On the matched native-routing protocol, parent SFT scored 0/42; this HashHop checkpoint scored 0/42. The stage name describes the training performed, not a guarantee of improvement.

Manual review of generated answers

All 32 recorded coherence outputs were manually reviewed. With correctly selected complete memories, both manuals' initial summaries preserve the deployment identifier, filename, timeout and retry count. However, the first manual's full-memory summary invents checksum 0x00000000; the requested three-paragraph structure is not followed. Follow-up answers omit required identifiers and either miscount retries versus total attempts or omit the total. The second native-routed follow-up still repeats to 1,024 tokens. Relative to matched SFT, native-routed limit hits decrease from 2/4 to 1/4 and oracle limit hits from 2/4 to 0/4; this small-sample reduction in looping is not a pass for factual coherence. Corrected latent-only recall reproduces the initial source facts, while physical-only and posterior-codes-plus-weights controls invent identifiers and settings. Latent-only and source-visible follow-ups also fail identifier retention/attempt-count requirements. Reliable multi-turn memory use is not established.

Training objective and dataset audit

Checked 256 training labels; 0 are arithmetically incorrect. 256 queries start at chain roots and 256 end at terminal nodes. A solver that ignores the requested hop count solves 256/256 training rows but only 10/42 evaluation queries. There are 0 same-query/different-memory target pairs. The implemented contrast loss can decrease by raising empty/wrong-context NLL. This pilot also omits retrieval/cross-unit supervision and trains a compressed-only read that differs from production corrected recall. These are training-design defects; the results do not isolate an architectural capacity failure.

Read-interface repair and evidence scope

The release indexes the complete raw question separately from chat role/control tokens. At read time, the native query BOS precedes the recalled feature sequence; no stored feature is removed. These implementation repairs did not alter SFT weights or the archived baseline units. An earlier, unrepaired SFT test scored 0/42 with native routing and 0/42 with oracle units; its source-visible control scored 4/42. The reader version associated with the table is identified above; the old and repaired protocols must be interpreted separately.

The initial SFT LlamaFactory template omitted the native leading BOS. The subsequent HashHop template includes it and its token IDs were checked against the native HF chat template. The completed SFT history is preserved rather than described as having used the corrected serialization.

Post-SFT J-lens calibration and functional interventions were also run. Their limited controls provide evidence of controllable internal representations, not proof of every global-workspace property. The English architecture, experiment code and checkpoint-separated evidence are maintained at LFM2-JSpace-Memory.

Loading and ordinary generation

Use the recorded PyTorch/Transformers environment from the linked research repository. The model uses custom architecture code and requires trust_remote_code=True. FlashAttention-2 is used for training; install a compatible native FA2 or the pinned community kernel used in the experiments.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "win10/Lfm2-MaleCNS-Titans-VL-3B-V2-HashHop"
model = AutoModelForImageTextToText.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16, device_map="cuda",
    attn_implementation="kernels-community/flash-attn2@f50dc99ed079b35990bc895d43fd353ea0cb376d",
).eval().requires_grad_(False)
processor = AutoProcessor.from_pretrained(repo, trust_remote_code=True)

Opening an archive is explicit; ordinary model.generate() does not silently choose a disk archive.

archive = model.open_physical_archive("./my_memory")
source = processor(text="A newly observed fact belongs here.", return_tensors="pt").to("cuda")
archive.session.observe(**source, use_cache=False, logits_to_keep=1)
# Optional physical FFN learning: labels must come from the observed source.
archive.session.learn(**source, labels=source["input_ids"].clone(), learning_rate=0.001)
saved = archive.append()  # Native concept address + complete immutable physical unit.
query = processor.tokenizer.apply_chat_template(
    [{"role": "user", "content": "What was the newly observed fact?"}],
    tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True,
).to("cuda")
address = processor(text="What was the newly observed fact?", return_tensors="pt").to("cuda")
result = archive.generate(address, generation_inputs=dict(query), top_k=1,
                          max_new_tokens=64, do_sample=False)
answer_ids = result["tokens"][0, query["input_ids"].shape[1]:]
print(processor.tokenizer.decode(answer_ids, skip_special_tokens=True))
archive.close()

The 64-token output limit above is an example argument, not a built-in maximum. Ordered recall checks the complete selected memory/query/output budget against the native context length. Cold memory units are checkpoint- and codec-bound; old addresses are not silently relabeled after fine-tuning.

Architecture: fixed backbone, physical memory units, and native concept addressing

1. Native LFM2.5-VL inheritance

Lfm2TitansForConditionalGeneration inherits Hugging Face's Lfm2VlForConditionalGeneration. The native vision tower, multimodal projector, tokenizer, chat template, hybrid language stack, and output head remain in the model. The inherited language stack has 30 layers, combining 22 short-convolution layers and 8 attention layers, with hidden width 2,048. Language memory ports attach at zero-based layers 4, 14, and 26. The visual port reads native vision patch/tile features; image observations are not replaced by generated captions.

Each port has its own feature projections and gates. There is no shared, narrow cross-modal input/output latent. The port interface is still parameter-intensive; adding dense interface parameters is not presented as an increase in persistent memory capacity.

2. Sparse MaleCNS–Titans substrate

The inherited selected topology has 90,839 nodes, 1,983,608 directed edges, four channels, and two graph microsteps. Edge indices are stored in memory_graph.safetensors, rather than millions of JSON integers. Destination/source order is part of the checkpoint contract. Four channels provide 7,934,432 fast scalars; graph fast weights plus momentum require 63,475,456 bytes in FP32, before other unit components.

For a port feature x, independent projections drive its assigned input nodes. A dendritic graph operator propagates for two steps; the port reads its assigned read nodes and projects back into the native feature space. Compiled finite-step execution plans and CSR sparse multiplication evaluate the reachable computation. Custom sparse autograd preserves the outer derivatives needed to train an online gradient writer. Small compute tiles bound temporary tensor sizes; these are not SFT sample windows.

Within one observation block, every read uses the same snapshot W_t. After the complete causal forward has produced its observations, a single commit creates W_(t+1). Thus a later write cannot alter an earlier read in that block.

The writer uses next-feature associations for language and observed-feature reconstruction for vision. It differentiates a mean squared reconstruction objective with respect to physical synaptic weights. A separate write-key projection and learned value projection are present. In the selected configuration:

g = sum of port gradients masked by edge/channel ownership
g_normalized = g / (RMS over writable scalars + epsilon)
momentum_next = beta * momentum + g_normalized
W_next = W - learning_rate * momentum_next

Non-writable entries preserve both their previous weights and momentum. Automatic decay is disabled. RMS normalization remains enabled, and the optional surprise-magnitude gate is not active in this inherited checkpoint. Accordingly, it would be inaccurate to claim that raw gradient magnitude presently controls write strength.

3. Structural write ownership

For port p, an edge e=(u,v) is structurally eligible when:

distance(input_nodes[p], u) + 1 + distance(v, read_nodes[p]) <= microsteps

Eligible edge/channel scalars receive an owner, rotated across channels with seed 17. A shared-association budget of 0.25 is applied only where multiple ports can reach an edge. The resulting masks restrict plasticity, not reading. This separates independently writable capacity from intentionally shared associations.

Structural eligibility is an upper bound on influence. It is not a measured number of facts the model can remember. The retained visual-protection mask covers 6,210 input-subgraph edges; ownership improves write separation, but that mask alone is not proof that every visual association is protected. This model does not establish that the biological topology outperforms a degree-rewired random graph.

There is no persistent neuron-activation vector in this checkpoint. Persistent graph memory is synaptic weight plus momentum; graph activations are recomputed for each feature. The design must not be described as a complete recurrent simulation of the fly brain.

4. Direct FFN physical weights

Each memory unit also contains rank-16 physical factors for w1, w2, and w3 at the three memory depths. These change the native FFN's computation. They are separate from the rank-8 outer SFT LoRA used to train this release.

The caller owns these factors and their first/second optimizer moments. An observation-derived supervised loss can update them while the backbone remains frozen. Immutable units include their physical FFN state, graph state, VAE codes, ordered observations, and addressing metadata. Loading a memory is therefore loading numerical state used by the forward pass, rather than attaching a textual description to a prompt.

5. Dual memory: VAE observations and Titans weights

The VAE participates in ordinary memory storage, not just dreaming. Independent per-port encoders produce a posterior mean and log-variance, with per-feature normalization statistics. Their hidden width is 64 and latent width is 16. The new native_input head is initialized as an exact copy of the prior first-language-port head, then trained for the actual input-embedding distribution.

Ordered native input embeddings include the native projected visual features at image positions. Their VAE means are decoded in order. Unit-local residual factors and sparse numerical correction values preserve information the shared decoder cannot reconstruct. Each decoded segment is checked against the original feature checksum. Encoding tiles bound SVD scratch space; all observed feature rows are stored.

This is not a fixed-size lossless compressor. Incompressible observations can require more bytes than raw BF16 features; the previous short-source pilot measured roughly 2.1 times raw feature size for its ordered representation. A checksum proves feature reconstruction, not semantic recall or multi-hop reasoning.

At recall, the selected ordered features are decoded and passed through all native language layers, while the selected graph and FFN physical weights contribute to the forward computation. Native attention/convolution caches are rebuilt and remain ephemeral. Stored token IDs, source strings, pixel arrays, or KV caches are not the serialized representation of the ordered memory.

The native query's leading BOS is placed before the recalled feature sequence. Every recalled feature and query token remains present. This avoids a second conversation-start marker after the memory prefix. The same framing helper is used by the production reader and the memory-dependent training objective.

The current ordered reader accepts a fresh, unpadded, batch-one text query; stored observations may include images. Selected features plus query and generation budget must fit the backbone's native 32,768-position context. Exceeding this raises an explicit error. Unbounded disk growth does not imply unlimited simultaneous attention to every archived observation.

6. J-lens-derived addresses

For each calibrated layer, a complete 2,048 × 2,048 averaged Jacobian maps that layer's native residual features toward the final language layer. The estimator sums over valid causal target positions, averages over valid source positions, then averages over calibration prompts. It is not merely a same-position diagonal derivative.

The vocabulary output vectors, final normalization weights, and Jacobian define an overcomplete dictionary of directions. Positive matching pursuit followed by projected nonnegative least-squares refitting selects 16 coordinates per observed position. The stored representation includes vocabulary-coordinate indices, nonnegative coefficients, and residual energy. Vocabulary IDs identify numerical dictionary directions; decoding them to words is only a display operation. No hand-written concept list or generated JSON controls storage.

All observed positions at each memory language depth are retained. Query and memory codes are compared using sparse late interaction on CPU or GPU. The present index performs an exact scan; there is no deployed million-unit ANN throughput claim or alternative pooled-key reader. Physical weight tensors load only after the query chooses candidate units.

For text queries, the address encoder receives the complete question text, separately from the native chat serialization used for generation. Role and conversation control tokens do not contribute to the address. This separation does not require source IDs, manually supplied concept labels, or omission of any question text.

Addresses include a checkpoint identity, lens hash, recorded time, optional event time, logical time, and causal parent hashes. A content-derived SHA-256 reference supports exact programmatic lookup. Agent ownership, permissions, and application goals remain caller concerns; the memory layer does not impose a hard-coded multi-agent policy.

7. Disk units, hot mounting, and composition

The archive is append-only and has no software unit-count ceiling. Disk space grows with retained information. An active reader pins a complete immutable unit; a background loader stages another unit before an atomic swap. Unsaved changes cannot be silently discarded by mounting a different unit. Readers already holding a snapshot finish with that snapshot.

Multiple selected units keep separate graph/FFN evaluations. Their contributions are combined at the read interface; weights are not blindly averaged. Active composition has an explicit resident-unit bound. The default archive generation call selects one unit, and callers may request more within the configured bound. Selection quality and interference still require measurement.

The published runtime accepts the unified format-6 physical unit with ordered observations and version-2 native concept addresses. Old label-driven, pooled, graph-only, and associative-softmax recall branches are absent. Old artifact conversion, if needed, is an offline operation, not an inference fallback.

8. Dream consolidation

A shared weight VAE encodes normalized chunks of 1,024 values through hidden width 128 and latent width 32. It reconstructs graph and FFN deltas. Per-port feature VAEs provide source-conditioned replay. A small MaleCNS-derived sleep controller scores reconstruction-based scheduling signals.

Dreaming constructs a candidate state without mutating the current unit. Ordered decoded observations are replayed through the native model; accepted visual latent replay contributes native visual features. Actual graph commits update the candidate. A caller-provided functional retention validator must accept it before atomic publication. Latent-cycle similarity alone is not treated as a guarantee that an imagined observation is historically true. Existing physical units remain retained.

The sleep-controller topology is biological inspiration. This implementation and its current experiments do not demonstrate biological dreaming or a measured capacity gain from consolidation.

9. Joint SFT and release compatibility

Native LlamaFactory owns preprocessing, batching, Trainer, backward, and optimizer updates. The model returns native supervised cross-entropy plus memory auxiliary loss. Real Titans writes/readbacks train the memory, while VAE reconstruction/KL and scheduling objectives train the Dream components. FFN write gradients are captured from the native supervised backward without modifying those gradients; a detached batch of actual updates trains the weight VAE on the following batch.

The backbone, vision tower, projector, graph interfaces, physical FFN priors, VAE heads, and controller receive outer LoRA updates. The final checkpoint merges those adapters. J-lens calibration is then repeated because the native weights changed. Checkpoint-bound old units and addresses are not silently relabeled as compatible with new weights.

See TRAINING.md for the exact data, auxiliary sampling, timing, and prior-update provenance; see EVIDENCE.md for measured behavior.

Parent SFT training

The complete parent training schedule is documented in the SFT repository.

Attribution and license

The original LFM Open License v1.0 is included verbatim in LICENSE. The model is derived from LiquidAI/LFM2.5-VL-3B. The MaleCNS topology derives from MaleCNS v1.0 under its dataset attribution terms. J-lens methods follow Anthropic research; this is an independent implementation and evaluation.