cognitive-nutrition-experiment

The Density Imperative: Cognitive Nutrition Experiment

Fine-tuning data quality is signed. Sparse curation lobotomizes; dense curation teaches the model how to access and articulate its own pre-trained knowledge.

Papers

Author: Tad MacPherson, Metavolve Labs, Inc. Base Model: Llama 3.2 11B Vision-Instruct Dataset: Alexandria Aeternum Genesis (10,090 artworks from The Metropolitan Museum of Art) Interactive Explorer: iaeternum.ai/evaluation — browse all 100 evaluation images × 3 models × 8 evaluation tracks

The Experiment

A controlled ablation study proving that fine-tuning data quality is not a scalar quantity but a signed intervention. We fine-tuned Llama 3.2 11B Vision-Instruct on 9,081 cultural heritage images under three conditions — holding images, architecture, and hyperparameters constant. The sole variable is metadata density.

Results

The Three-Way Gradient

CogBench Cognitive Depth (Track 4)

All 8 dimensions statistically significant: Friedman test, all p < .001, n = 100.

Semantic Density Coverage

GLMM Verbosity Control

Key insight: Group A's destruction is real and independent of verbosity (β = −0.106, p < .001). Group B's advantage is mediated by the structured analytical patterns the model acquires — token count is a mediator, not a confounder.

Training Details

Group A (Sparse)

• Records: 9,081
• Steps: 1,135 (~1 epoch)
• Final loss: 0.865
• Training time: 34.9 hours
• GPU: NVIDIA L4

Group B (Dense NEST)

• Records: 9,081
• Steps: 1,135 (~1 epoch)
• Final loss: 1.164
• Training time: 38.3 hours
• GPU: NVIDIA L4

Shared Hyperparameters

Evaluation Framework

Four tracks, applied to 100 held-out images (909 reserve images available for community replication):

1. Track 1 — VULCA-Bench (L1–L5): Visual perception, technical knowledge, symbolic understanding, historical context, philosophical depth
2. Track 2 — POPE-Style Probes: Binary object presence questions (positive + adversarial + random)
3. Track 3 — Response Quality: Token density, refusal rate
4. Track 4 — Structural Generalization: CogBench cognitive depth (8 dimensions × 100 images), counterfactual resistance, semantic density alignment (6 NEST fields)

Total: 4,400+ question–answer pairs across three model conditions.

Repository Contents

├── README.md                           # This file
├── group_a_adapter/                    # LoRA adapter — Group A (sparse captions)
│   ├── adapter_config.json
│   ├── adapter_model.safetensors       # ~450MB
│   ├── tokenizer.json
│   ├── tokenizer_config.json
│   └── ...
├── group_b_adapter/                    # LoRA adapter — Group B (dense NEST)
│   ├── adapter_config.json
│   ├── adapter_model.safetensors       # ~450MB
│   ├── tokenizer.json
│   ├── tokenizer_config.json
│   └── ...
├── evaluation/                         # Evaluation results from GCS
│   ├── cn_experiment_results.json      # T1-3 results (early — see results/ for merged)
│   ├── track4_results.json             # T4 per-model results
│   ├── track4_results_merged.json      # T4 all 3 conditions merged
│   └── track4_raw_responses.json       # Raw T4 model outputs
├── results/                            # Complete results package
│   ├── cn_experiment_results_t13.json  # Merged T1-3 (Base + A + B)
│   ├── track4_results_merged.json      # Merged T4 (Base + A + B)
│   ├── track4_raw_responses_base_and_a.json  # Raw Q&A: Base + Group A
│   ├── track4_raw_responses_groupb.json      # Raw Q&A: Group B
│   ├── statistical_analysis_results.json     # Friedman, Wilcoxon, GLMM
│   ├── ppp_survival_test_results.json        # Perceptual hash survival
│   ├── STATISTICAL_ANALYSIS.md               # Human-readable stats
│   ├── CN_EXPERIMENT_RESULTS_COMPLETE.md     # Full results narrative
│   └── training_metrics/
│       ├── group_a_training.csv
│       ├── group_b_training.csv
│       └── training_metrics.json
├── scripts/                            # Full reproducibility scripts
│   ├── train_cn_experiment.py          # Training script
│   ├── prepare_data.py                 # Data preparation (Group A/B split)
│   ├── evaluate_cn.py                  # Tracks 1-3 evaluation
│   ├── evaluate_track4.py              # Track 4 evaluation
│   ├── statistical_analysis.py         # Statistical analysis (Friedman, GLMM)
│   └── cn_experiment_startup.sh        # VM orchestrator
├── eval_explorer.html                  # Interactive comparison tool (3.3MB standalone)
├── group_a_stats.json                  # Training stats
└── group_b_stats.json                  # Training stats

Interactive Eval Explorer

The eval_explorer.html file is a self-contained, 3.3MB interactive tool that lets you browse:

• All 100 evaluation images
• Side-by-side responses from Base, Group A, and Group B
• All 8 CogBench dimensions + counterfactual + semantic density tracks
• Per-image scores and comparisons

Open it in any browser — no server required. Also hosted at iaeternum.ai/evaluation.

Using the Adapters

from unsloth import FastVisionModel
from peft import PeftModel

# Load base model
model, tokenizer = FastVisionModel.from_pretrained(
    "unsloth/Llama-3.2-11B-Vision-Instruct",
    load_in_4bit=True,
)

# Apply Group B (NEST-enriched) adapter
model = PeftModel.from_pretrained(
    model,
    "Metavolve-Labs/cognitive-nutrition-experiment",
    subfolder="group_b_adapter"
)
FastVisionModel.for_inference(model)

Reproducing the Experiment

# 1. Get the dataset
pip install datasets
from datasets import load_dataset
ds = load_dataset("Metavolve-Labs/alexandria-aeternum-genesis")

# 2. Prepare training data (creates Group A sparse + Group B NEST splits)
python scripts/prepare_data.py --input alexandria_aeternum_full_v2.jsonl

# 3. Train (requires NVIDIA L4/A100, 24GB+ VRAM)
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
python scripts/train_cn_experiment.py --group A
python scripts/train_cn_experiment.py --group B

# 4. Evaluate
python scripts/evaluate_cn.py           # Tracks 1-3
python scripts/evaluate_track4.py       # Track 4 (CogBench, counterfactual, semantic density)
python scripts/statistical_analysis.py  # Statistical tests

The Key Insight

Dense structured training data teaches the model how to think about a domain, not merely what to say about specific examples. Group B produces superior analyses of images the model has never seen during training. The knowledge about art history, composition, and technique exists in both models' pre-trained weights. Only Group B has learned to access and organize that knowledge into structured analytical responses. This is not knowledge injection — it is cognitive scaffolding.

Citation

@article{macpherson2026density,
  title={The Density Imperative: How Semantic Curation Depth Determines Vision-Language Model Capability},
  author={MacPherson, Tad},
  year={2026},
  institution={Metavolve Labs, Inc.}
}

@article{macpherson2025entropy,
  title={The Entropy of Recursion: A Strategic Framework for Provenance-Verified Data and the Preservation of Signal Integrity in Generative Systems},
  author={MacPherson, Tad},
  year={2025},
  institution={Metavolve Labs, Inc.}
}

@article{macpherson2026architecture,
  title={Cognitive Nutrition: Architecture of a Provenance-Embedded Data Refinery},
  author={MacPherson, Tad},
  year={2026},
  institution={Metavolve Labs, Inc.}
}

Patent

U.S. Provisional Patent Application No. 63/983,304 — "Systems and Methods for Provenance-Embedded Semantic Transport, Multi-Agent Data Enrichment, and Perceptual Compute Offloading in Autonomous Systems"

License

Apache 2.0. The LoRA adapters, evaluation scripts, and results are freely available for research and commercial use. The underlying artworks are public domain (Met Museum Open Access).

---

Built by Metavolve Labs — Cognitive Nutrition for Foundation Models. "The factory is open."