gary-mesh

gary-mesh

A Mesh-NCA-style graph cellular automaton in pure numpy, plus a short design paper on growing a neural net wide (more neurons) instead of deep (more parameters).

One small shared update rule is run on every cell of a graph. Because every cell runs the same rule, the parameter count is set by the rule, not by the number of cells. So you can scale the population without scaling the training bill. This is the idea behind Mesh Neural Cellular Automata (Pajouheshgar et al., TOG 2024), applied to the Gary family of tiny numpy models.

What's measured here

A 2,364-parameter shared rule, trained by backpropagation-through-time (gradient verified against finite differences to 1.4e-8):

• It learns. Correlation between the self-organized pattern and the target reaches 0.979 on a 64-cell ring, with each cell seeing only its two neighbors.
• Parameters stay flat as the mesh grows. The same rule runs on 64 → 65,536 cells with the parameter count fixed at 2,364 and compute staying linear at ~0.006–0.012 ms per cell. Sixty-five thousand neurons run in ~400 ms on the same rule that fit the 64-cell ring.
• Honest limit: global-pattern generalization across scales fails, because a T-step local rule can only coordinate cells within T hops. The fix (a statistical/style loss + rotation-equivariant perception) is exactly what MeshNCA uses, and is the next build.

Files

• gary_mesh.py — the model: ring graph, direction-aware message passing (the 1/cos/sin 2D shadow of MeshNCA's spherical harmonics), shared 2-layer MLP, Bernoulli async masking, hand-derived BPTT, Adam.
• demo.py — reproduces the headline (trains in ~2 s, then prints the scaling table).
• gary_mesh_rule.npz — the trained 2,364-parameter rule.
• PAPER_million_neurons.md — the design note: how a million Gary neurons could learn from each other cheaply, and how to graft a mesh layer into the chat cortex.

Run

pip install numpy matplotlib
python demo.py

Family

• gary-neuron — an asynchronous NCA + mixture-of-experts adder (already message-passing; 7-digit addition).
• gary-neuron-chat — a dialogue model that learns from use (plastic memory + episodic store).
• gary-4-petite — the tiny cortex the others warm-start from.

Citation

Pajouheshgar, E., Xu, Y., Mordvintsev, A., Niklasson, E., Zhang, T., Süsstrunk, S. Mesh Neural Cellular Automata. ACM TOG 43(4), 2024. arXiv:2311.02820. Mordvintsev, A., et al. Growing Neural Cellular Automata. Distill, 2020.