esp32-ai-fly

Fruit fly connectome for ESP32-S3

A selected MaleCNS v1.0 connectivity graph for an ESP32-S3 with 16 MB Flash and 8 MB PSRAM. It preserves 48,311 neurons, 9,462,135 directed edges and their 49,481,754 synaptic contact counts in a compressed graph. The runtime processes every retained edge on the board using two cores, SIMD and a PSRAM cache.

Escaping a looming spider is the included demonstration, and nothing in it is trained. A looming signal computed from the simulated spider's distance drives the fly's own looming detectors, LC4 and LPLC2, on the side it approaches from. When the mean activity of the escape descending neurons on that side passes a threshold, the fly jumps away from it. Those outputs are DNp01, the giant fiber that makes a real fly jump, plus DNp02, DNp04 and DNp11. The graph, the circuit, the world and the OLED all run on the board; no computer takes part while it runs.

The escape circuit is a small text file naming input and output neurons, one input amplitude and one threshold. It was found by stimulating groups of neurons and measuring which descending neurons answer, not by fitting anything to a task.

This is a selected induced subgraph, not the complete fly brain or nervous system. The rate dynamics and the escape interface are engineering choices. These custom binaries use the included C runtime, not transformers.

metadata.json records the package version and SOURCE_CODE.json records the source snapshot and each file hash, including any uncommitted changes relative to the base revision. No precompiled firmware is included.

Contents

An edge represents one source/destination pair, with an integer contact count. The edge count is not a count of individual synapses. Graph compression preserves all retained edges and counts exactly. Q29 state arithmetic is a separate approximation, checked against floating-point gold with explicit tolerances.

The original EM images, neuron morphology and raw connectome tables are not included.

Which part of the nervous system?

The source is MaleCNS v1.0. Selection starts from annotation rows with a nonempty superclass and status != Glia, sorted by body ID. From that population we keep only superclasses beginning with cb_, visual_ or descending_neuron. Connections are retained only when both endpoints are retained. There is no additional edge-strength threshold.

This removes all vnc_* classes, all ol_intrinsic and ol_sensory classes, ascending neurons and the other classes listed in SUBGRAPH.json. It keeps central-brain classes, visual projection/centrifugal classes and descending neurons. It is not simply the whole brain with its nerve cord cut off.

Selection is by cell annotation, not by synapse coordinates. Retained descending cells can have processes in the nerve cord; contacts are not spatially cropped. Our filtered starting population contains 166,700 annotated cells. This count comes from the frozen files and predicate above, rather than a paper headline. See SUBGRAPH.md and neurons.csv for the complete inventory.

It is a part of the nervous system because the whole of it does not fit. The smallest lossless encoding measured for the complete nervous system was 24.49 MiB, and 20.99 MiB with the nerve cord classes removed, against about 14 MiB of Flash available for the graph. This selection is 8.61 MiB at its smallest, and 13.5 MiB in the runtime format, which stores independently decodable blocks.

Download and check

Download the package with the Hugging Face CLI:

hf download slvDev/esp32-ai-fly --local-dir fly-model
cd fly-model
shasum -a 256 -c SHA256SUMS

The runtime snapshot is included so this release can be used without depending on a moving GitHub branch. It includes the relevant runtime, firmware, deployment, research and verification files. SOURCE_CODE.json distinguishes an exact Git revision from a working-tree snapshot and lists every file hash. Code is MIT; included data notices retain their CC BY 4.0 terms. From the downloaded directory:

tar -xzf esp32-ai-fly-source.tar.gz
cd esp32-ai-fly
python3 -m venv .venv
.venv/bin/python -m pip install -r research/fly/requirements.txt
.venv/bin/python firmware/esp32_fly/tools/verify.py host \
  --bundle .. --report release-host-check.json

This verifies the full graph gold in five host modes and the escape gold. The checked-in manifest matches this release exactly; the download's own hashes do not automatically authorize replacing its graph or circuit.

ESP32 build

The target is ESP32-S3 N16R8 with octal PSRAM and a 128 × 64 SH1106 I2C OLED: 3V3, GND, SDA GPIO18, SCL GPIO46, address 0x3C. The repository firmware was tested with these pins.

From the extracted source directory:

.venv/bin/python firmware/esp32_fly/tools/prepare_assets.py --bundle ..

Install the Arduino board package and the display libraries as described in firmware/esp32_fly/README.md, and uv, which deploy.sh uses to run its Python tools. From the extracted source directory:

ARTIFACTS=.. scripts/deploy.sh fly

deploy.sh takes exactly one argument, the model name, because the board holds one model at a time and deploying replaces what is already there. ARTIFACTS points it at this bundle, PORT selects the serial port when the first /dev/cu.usbmodem* is not the board, and MANIFEST selects a different escape circuit. It generates fresh headers, runs full host gold, compiles, writes the graph separately at 0x210000 and only then uploads the application. The circuit's neuron indices are compiled into the application; the graph stays an external partition image. The application compiles to 426,274 bytes of the 2,097,152-byte application partition, which is a build figure from the compiler rather than a device measurement.

The source archive's docs/fly-connectome/README.md explains graph scope, memory layout and reuse independently of the demo. Device numerical and live-action checks are in firmware/esp32_fly/VERIFICATION.md.

On screen, the fly walks a random path and the spider creeps in from an edge. Physics runs at 120 Hz, the display at about 19 frames per second, and a neural decision arrives about every 1.7 seconds, so the spider needs several seconds of looming before the fly reacts. The bar in each top corner is the mean activity of that side's escape neurons, with the escape threshold marked; the sides are the fly's own eyes, not halves of the screen.

What the wiring does by itself

The graph produces the neural response without training; sensory encoding, the threshold, jump rules and the spider's retreat are programmed. Driving the left looming detectors and watching the descending neurons, against 20 random control groups of the same size drawn from other visual projection neurons:

Ranking all 1,316 descending neurons by how specifically they respond puts DNp01, DNp04, DNp02, DNg40, DNp71 and DNp11 at the top, known looming-escape neurons of the fly found by the wiring alone. The response appears two steps after the stimulus because LC4 and LPLC2 contact these cells directly. The full records, including the right eye and the gold threshold crossings, are in the evidence section of escape-circuit.json.

Smell and taste behave differently: stimulating olfactory or gustatory sensory neurons produces no left/right signal in the steering neurons DNa01, DNa02, DNa03 or MDN. Those runs were made while choosing what to port, on graphs this package does not contain, and are not part of the shipped evidence. This release demonstrates the escape circuit only.

Reuse and change the circuit

The graph executor knows nothing about spiders. To point it at different neurons, stimulate candidates with the probe and export a new circuit, gold and candidate manifest. From the extracted source directory:

.venv/bin/python -m research.fly.probe --bundle .. \
  --stimulus 'loom_L: type=LC4,LPLC2 side=L' \
  --control 'superclass=visual_projection* type!=LC4,LPLC2,LPLC1,LPLC4,LC16,LC6' \
  --watch 'type=DNp01,DNp02,DNp04,DNp11' \
  --out probe-loom.json
.venv/bin/python -m research.fly.circuit --bundle .. --out-dir circuit-out

Each --stimulus is a "name: selector" pair, --control names the random comparison pool the response is ranked against, and selectors are space-separated key=value terms over type, side, superclass, class and body_id, where key!=value excludes.

Review the candidate, copy the files into the bundle and select the manifest explicitly with --manifest, or replace the checked-in one. Full commands are in research/fly/README.md in the source archive. NumPy and a C compiler are sufficient; these tools run on the CPU.

Changing which neurons are read does not require retraining anything, because nothing was trained. It does require new gold and new numerical checks: merely changing a hash does not establish that a circuit behaves as intended.

Results and limits

The circuit evidence was corrected on 2026-09-22 to centre laterality z-scores on the control mean. The graph, neuron selections, amplitude, threshold and escape gold are unchanged. validation/release-host-gold-20260922.json checks the current bundle. The device results below are historical runs from 2026-09-16 using the prior circuit fingerprint; this corrected package has not been reflashed or retested on hardware. See validation/README.md for exact scope.

The replay row counts how often the fly escaped during one 60-decision session. It is not a success rate: how often a spider reaches the fly depends on the demo world's random walk, and no quality gate is applied to it. The verifier recomputed each decision from the neuron states the board reported, so the numerical rows describe the board's own arithmetic. The OLED acknowledged at 0x3C and its frame counter advanced; pixel appearance was not inspected.

The release-device-*.json receipts record device-reported identities and the local verification source hashes. They do not attest the installed firmware binary or prove which source it was built from. Host tests do not execute Xtensa SIMD or measure device scheduling.

The dynamics use normalized positive contact counts and a tanh rate update; they do not implement measured neurotransmitter signs, conduction delays, biological plasticity or validated full-fly behaviour. A strong, specific response means the wiring carries that signal in this model, not that a real fly behaves this way. These results describe this selected graph and escape circuit, not a general-purpose biological agent.

Attribution and license

The MaleCNS dataset is provided by FlyEM at HHMI Janelia, the University of Cambridge Department of Zoology, the MRC Laboratory of Molecular Biology and Google Research under CC BY 4.0. See the official project and ATTRIBUTION.md for the paper citation and changes made here.

The model/data package is distributed under CC BY 4.0. The included source code retains its MIT license in LICENSE-CODE and the source archive. This is an independent project by Viacheslav Sierbov / slvDev. The upstream teams do not endorse this port or its escape demonstration.