Lfm2-MaleCNS-Titans-VL-3B

Lfm2-MaleCNS-Titans-VL-3B

LFM2.5-VL-3B with the c042 MaleCNS–Titans neural memory architecture, followed by LoRA training of the native language/vision/projector and full training of the added memory parameters. LoRA has been merged into this complete model. The custom class directly inherits Lfm2VlForConditionalGeneration and saves the native backbone, memory weights, graph, ports, tokenizer, processor, and model code together.

這是小樣本 PT 探索模型。保留原始 LFM 的語言、視覺、投影與生成架構，加入多路神經記憶；此版本包含實際 LoRA 訓練後合併的完整權重。

Architecture

• 11,240 selected MaleCNS v1.0 neurons and 119,997 directed connections; 4 dendritic channels and 2 microsteps.
• Independent native vision port (1,152 features) and language ports at zero-based layers 4, 14, 26 (2,048 features each). Each has its own input/read projection and residual gate.
• 120,024,951 added learned memory parameters; 119,544,960 (99.60%) belong to the ports. Runtime fast weights contain 479,988 FP32 scalars (1.83 MiB), or 3.66 MiB including momentum. These counts do not establish how many facts can be recalled.
• At the current two microsteps, 86,946 of 119,997 edges are potentially observable through at least one port. This corresponds to an upper bound of 347,784 observable channel scalars, not measured independent information capacity.
• Inner gradient writes update fast connection weights with RMS normalization, momentum, forgetting, and a trained slow-weight anchor. Text-only writes preserve 3,460 selected visual input-to-input connections exactly; 2,967 are on the visual port's observable paths. This protects a subset of weights and does not establish that the entire visual association is preserved.
• Write rate, momentum, and forgetting are learned global scalars here. They are not per-input controllers; this is a Titans-inspired update rather than a reproduction of every component of the Titans paper.
• Every port reads the previous block snapshot. A write commits once after the native forward; the caller explicitly owns the returned session state.
• Neuron activity is transient within each read. This checkpoint persists synaptic fast weights and momentum, with no additional persistent neuron-activity state. Ports share the fast-weight bank, with different structural supports.
• Native local context remains 32,768 tokens. Fast memory can persist across cache resets and input blocks, with finite capacity and possible interference/forgetting. This model does not provide lossless unlimited recall.
• The original backbone FFNs remain present. The connectome is the added memory computation, attached through the distributed ports.

Exact graph indices, port selections, and dynamics are in config.json; the searchable graph provenance and diagram are under architecture/. The source-level capacity audit and proposed next experiments are in architecture/capacity_review.md; those proposed changes are not part of this checkpoint.

Training performed

The prepared data were used directly as HF Arrow. No SFT, corpus-wide pass, or deduplication was added. Each episode writes a raw prefix (plus native image features on image episodes), then predicts the suffix with a fresh native cache and retained neural state. Checkpointed inner-gradient chunks retain all features and higher-order training gradients.

Loss: retained PT CE + 0.1 × prefix PT CE. On image episodes, add 0.2 × reset PT CE, 0.1 × max(0, 0.05 + retained CE − reset CE), and 0.02 × KL(original-native distribution || reset-student distribution). The original teacher disables LoRA and memory. The reset CE coefficient exceeds the margin coefficient, so the loss does not encourage increasing reset CE to manufacture a memory gain. This coefficient property alone is not a general guarantee against all evaluation artifacts.

Training used the user's local LlamaFactory fork and its PT CustomTrainer. Its nominal Transformers version ceiling was explicitly bypassed for the installed 5.9.0 backend; actual integration tests and runtime versions are recorded. The historical local training source is in training/, with local dataset/work paths requiring adaptation on another machine. Adapter/optimizer resume checkpoints remain in the original local work directory.

Verification and measured memory effect

One 131,072-token stream: 129,024 writes followed by a 2,048-token held-out read. Only the final read is replayed with reset memory. Both read branches discard native KV/convolution/image caches. The same bounded exploratory evaluation was also used in architecture search; these are development results, not an independent benchmark.

• Retained-memory gain over reset: 0.511086 NLL.
• Changed next-token argmax positions: 948 / 2,047.
• Visual-only section gain: 0.355743 NLL.
• Full exported model reload: maximum target-NLL error 0.00000000, identical argmax True.
• Real neural-state save/load preserved fast weights, momentum, and commit count exactly. Native text and image generation API checks passed; these checks do not measure factual accuracy.

Compared with c042 before native LoRA, absolute retained NLL improved by only 0.005825. Reset NLL changed by +0.057767 (higher is worse). Consequently, most of the 0.063592 increase in retained/reset gap comes from a worse reset branch. The gap increase should not be presented as an equivalent improvement in memory quality. The absolute retained improvement is small and has not been established as statistically significant on an independent test set.

The visual sections score held-out article tokens after image-only observations, so the metric combines visual/text association and language prediction. A change in NLL is not proof of exact visual recall. Previous search generations showed repetition and unverified details. The current small PT run establishes an executable integrated architecture and measured state effects; it does not establish superiority to every random topology or broad downstream quality.

See verification/evaluation.json, verification/reload.json, and verification/api_checks.json for receipts. The pre-LoRA integration receipt is explicitly historical; actual LoRA trainability is in training/protocol.json.

Run

Validated runtime: PyTorch 2.11.0 with CUDA 13.0, Transformers 5.9.0, Accelerate 1.14.0, NVIDIA Blackwell. Memory parameters load in FP32 and native parameters in BF16. The supplied runtime code is required; use trust_remote_code=True and a reviewed/pinned Hub revision. Training checks for actual FlashAttention-2.

python inference.py --model win10/Lfm2-MaleCNS-Titans-VL-3B \
  --observation "這段文字是要先寫入記憶的觀測。" \
  --question "請概括剛才的觀測。" --session-out session.pt

python inference.py --model win10/Lfm2-MaleCNS-Titans-VL-3B \
  --image example.jpg --observation "" \
  --question "請描述先前看過的圖片。" --session-out image-session.pt

inference.py uses forward(memory_write=True) to observe, then generate(memory_state=...) with a fresh native cache. It supports --session-in to restore a state from the same model checkpoint. Model weights and a conversation's fast state are separate artifacts. New sessions start from the trained slow initialization; the evaluation state is not embedded in the published model.

Use torch.no_grad() for observation writes: the memory implementation temporarily enables gradients for its inner update. Do not wrap writes in torch.inference_mode(). Generation reads memory; generated tokens are not automatically written back. One write batch corresponds to one session, preventing accidental mixing of independent conversations. use_memory=False provides the merged native control.

Sources and license

• Base: LiquidAI/LFM2.5-VL-3B, under the included LFM Open License v1.0.
• Topology source: MaleCNS v1.0, Janelia.
• Neural fast-weight inspiration: Titans: Learning to Memorize at Test Time.

This is a modified model, not an official Liquid AI or Janelia release. The connectome is used as a selected computational topology; the implementation is not a full biological brain simulation. See NOTICE and manifest.json for modifications and artifact hashes.