matrixai-will-it-rain

Will it rain tomorrow? (real data)

The case that CANNOT be fully proven — which is why it is here.

A dense network with six inputs — temp_max, humidity, pressure, pressure_delta, cloud_cover, precip_hours — one hidden layer of 16 ReLU units and a 2-way softmax, which returns a probability for si (it rains the next day) and no. It was fitted to a 2,189-row file of real daily weather observations — 1,751 rows used for training, 438 held out — and it is published because it is the MatrixAI case that does not come out clean: most of what matrixai verify normally checks cannot be checked here, and the package says so itself.

It is one of the three reproducible cases at matrixaistudio.org/casos.

---

Read this first: three of the four verification stages could not be checked

Every MatrixAI package carries a manifest, and matrixai verify reports four stages separately, with four possible verdicts (PASS, FAIL, INCOMPARABLE, NOT_RUN). Run against this package, this is the entire output, unedited:

$ matrixai verify paquete.zip --retrain --locale en
manifest  PASS
R1        INCOMPARABLE  — the package itself declares R1 is not possible: "Cannot regenerate the dataset from the recipe and compare its full sha256: this model has no data recipe, so its dataset cannot be regenerated and compared; why there is none is not recorded in the package; the dataset generation seed is unknown."
training  INCOMPARABLE  — the dataset could not be regenerated, so there is nothing to train on
R3        NOT_RUN  — the retraining run did not complete, so there is no fresh value to contrast: the dataset could not be regenerated, so there is nothing to train on
$ echo $?
3

The exit code is 3, measured here on 2026-08-30 by re-running the published paquete.zip with matrixai-core 1.7.0: output and exit code identical to the published salida.txt. The scale is 0 nothing failed · 2 something does not match · 3 it could not be checked — that line is printed in the README of the two reproducible packages in this set; this bundle's README has no verification section at all, because there is nothing here for verify to rebuild.

Verdicts and reasons: the output above, which matches salida.txt of the published case. INCOMPARABLE means "I could not check this" — a distinct verdict from FAIL, and the wording is quoted from https://matrixaistudio.org/manual/proof, where the four stages are described.

The package declares the same about itself before anyone runs anything. From reproduce.json:

"reproducible": false,
"reproducible_reason": "Not reproducible: this model has no data recipe, so its dataset cannot be regenerated and compared; why there is none is not recorded in the package; the dataset generation seed is unknown.",
"missing": ["recipe", "seed_dataset"]

and its verifiable block records "possible": true for manifest alone, and "possible": false for r1, training and r3, each with the reason quoted above.

On the count. The project page phrases this as "two of the four stages come back INCOMPARABLE", which is the count of INCOMPARABLE verdicts. The fourth, R3, comes back NOT_RUN — a different word for a consequence of the same cause. Counted by what they establish about this model: one stage passes, three establish nothing, and reproduce.json marks all three "possible": false.

Why there is no recipe. In MatrixAI a recipe is a rule that generates the data — the other two cases at /casos have one (predicted_kelvin = 1*celsius + 273.15, for instance), it travels inside the package, and verify re-runs it, rebuilds the dataset, compares its digest, retrains and contrasts the metric. This package has none, and it records that why there is none is itself not written down: "why there is none is not recorded in the package". The reason the authors give, on the project page, is that no rule was invented so the four stages would go green.

"No recipe" is not "no data". The 2,189 rows are published in full (lluvia_nombrada.csv, 168,756 bytes). What is missing is a rule that regenerates them.

---

What is still established, which is not nothing

Each of these was re-measured on 2026-08-30 against the published files, with matrixai-core 1.7.0 (the package was built with 1.6.0). Every command is in Reproducing it.

1. The package is internally consistent — manifest PASS. Recomputing sha256 over the 15 files named in reproduce.json: 15 of 15 match. The only file in the zip not named by the manifest is reproduce.json itself, which is the manifest — a manifest cannot carry its own digest inside.
2. It predicts with no MatrixAI installed. predict.py needs only numpy and onnxruntime. Re-run with imports of matrixai blocked at sys.meta_path, it returned the published probabilities to the last digit.
3. The exported ONNX matches the reference implementation. export_manifest.json: equivalence_check.passed: true, max_abs_diff 7.45e-08, max_rel_diff 4.81e-07, over 20 samples, against atol 1e-05 / rtol 1e-04. Those 20 vectors come from a generator with a fixed default seed of 42 (matrixai/export/equivalence.py) — the same 20 points on every export, not a fresh sample.
4. The published weights do come from that published CSV, and you can check it by hand. This is what verify will not do for you, because it will not guess where a dataset came from. But feeding the published CSV back through the published .mxai and .mxtrain reproduced Best epoch: 11, Best validation loss: 0.533861, Accuracy: 0.762557, and a params.best.json byte-identical to the one in the package (sha256 2d8273df56be1b305e17504691ef3cce79550990fbee0dee146a8a663968f0cf). Re-exporting produced 9 of the 16 bundle files byte-identical, including model.mxai, model.mxtrain, predict.py, params.best.json, example_input.json and expected_output.json. The other seven differ — README.md, export_manifest.json, inference_spec.json, model.onnx, model_manifest.json, reproduce.json and space/requirements.txt — and all seven differ only in the version and timestamp they stamp on themselves (matrixai_version 1.6.0 → 1.7.0, created_at/exported_at, the matrixai== pin in space/requirements.txt, and the environment_sha256 and manifest_sha256 derived from them). The ONNX graph itself serializes identically: one byte differs in the whole file, the 6 of the version string in its metadata.
5. The published metric can be recomputed from the two published files. See The measured numbers.

None of that says the model is good, or that the rows are a faithful record of anything. It says the package holds together and that the number on it is attached to a file with a digest.

---

What this does NOT prove

That the data are what they claim to be. This is the one that matters. reproduce.json declares a dataset digest, and the published CSV matches it, and retraining from that CSV reproduces the weights — so the chain from this file to these weights closes. What no digest can establish is that those 2,189 rows are a faithful record of real weather, at a real place, over a real period. That step is trust in whoever downloaded them, not evidence from the package. The report writes it down instead of covering it with a PASS.

That 76.26% is how often it is right about tomorrow. It is accuracy over one contiguous 20% block of the same file. Measured against that same block: always answering si scores 64.61% (283 si / 155 no in those 438 rows). The same model scores 71.36% over the whole file and 70.13% over the 1,751 rows it trained on — so the held-out block is not a harder sample than the rest, and the headline number is the best of the three, not a typical one.

That it is even-handed between the two answers. On that block it answers si for 363 of 438 days: recall 95.8% on si (271 of 283), recall 40.6% on no (63 of 155). It misses most dry days. Confusion counts are in The measured numbers. Calibration was not measured at all — the probabilities have not been checked against observed frequencies.

Where and when this weather happened. The CSV has no date and no location column, so which station and which years are unrecoverable from the published files. What the file does show, measured: llueve_manana on row i equals rain > 0 on row i+1 for 2,188 of 2,188 consecutive pairs. So the label rule is recoverable exactly from the published CSV even though no case file states it, the rows are in sequence rather than shuffled, and the held-out block is the later end of that sequence, not an interleaved sample. Whether consecutive rows are consecutive calendar days cannot be checked — there is no date column.

That the model saw everything the file holds. The CSV carries 13 feature columns; the training contract uses 6. temp_min, temp_mean, rain, wind_speed, dew_point, wind_dir_sin and wind_dir_cos were not fed to the model. rain in particular is the column the label is derived from, one row ahead.

Several things the package explicitly leaves empty, and an absent value is not a zero: generation.epochs_ran and epochs_effective are null — the contract declares 40 epochs and the training output reports epoch 11 as the best, but the package does not record how many ran (re-running the published files writes epochs: 40 into runs/v1/training_trace.json, so all 40 did); artifacts.dataset.rows_used is null — and that field is the number of rows the training consumed, not the size of the validation split: the core documents the distinction where it builds the block (matrixai/export/reproduce.py, the comment on artifacts.dataset — rows are the raw CSV's rows, the ones R1 has to regenerate; rows_used the ones training consumed; paraphrased here, the source comment is in Spanish). Re-running the published files gives 1,751 training rows and 438 held out, so 1,751 is the number that would fill that hole; seeds.dataset is null; and each metric names its own gaps in an incomplete list — accuracy: evaluator, evaluator_version, tolerance_rel; best_validation_loss adds aggregation, which is itself null.

Authorship or authenticity. In the manifest's own words: "It does NOT prove authorship or authenticity: signatures are out of scope here." Nothing here is signed.

Fitness for any use. This is not a weather service and not a forecast product. Nothing on this page is a claim of regulatory conformity of any kind, in any jurisdiction.

This repository deliberately carries no model-index metrics block. That widget presents a number as a checked result; the point of this card is that this particular number is one you have to recompute yourself, by hand, from files whose provenance stops at a person.

---

The data

Origin, as stated on the project page: "Daily observations from Open-Meteo (open-meteo.com), ERA5 reanalysis. Its API is free for non-commercial use and its data derives from ERA5 (Copernicus/ECMWF), published under CC BY 4.0. The CSV you download here is a selection of columns plus a derived label — «will it rain tomorrow» — computed by us; it is not the original data." The data licence is not the code licence: the AGPL-3.0 below covers MatrixAI and this package, not the observations.

One digest detail worth knowing before you check it

The published CSV has CRLF line endings, so sha256sum on the file as downloaded returns ddec894dee537801eee8cb2eab82a221503b9deef1afa91451827a05a2ce4023, which is not the digest in reproduce.json. That digest matches the same file with its CRLF newlines normalised to LF:

python3 -c "import hashlib;print(hashlib.sha256(open('lluvia_nombrada.csv','rb').read().replace(b'\r\n',b'\n')).hexdigest())"
# b0bb8777559188aad2dc9e817cbd10cf65c54fa2de8a445f900e0230a93480bd

---

The measured numbers

Everything below is either quoted from a published file or was measured by re-running the published files; the "source" column says which, and no number appears here that is not one of the two.

Measured on 2026-08-30 by re-running the published bundle over the published CSV (script below), not read from any file:

reproduce.json records seeds.split: 42; the cut that produced this run is the contiguous last 20% of the file, which is what reproduces the published accuracy exactly.

# recompute the published accuracy. Run from inside the bundle, with a copy of
# lluvia_nombrada.csv next to predict.py. No MatrixAI needed.
import csv, sys, collections
sys.path.insert(0, ".")                       # run from inside the bundle
from predict import MatrixAIModel

rows = list(csv.DictReader(open("lluvia_nombrada.csv", newline="", encoding="utf-8")))
cut = int(len(rows) * 0.8)
model = MatrixAIModel()
fields = ["temp_max", "humidity", "pressure", "pressure_delta", "cloud_cover", "precip_hours"]
cm = collections.Counter()
for r in rows[cut:]:
    p = model.predict({k: float(r[k]) for k in fields})
    cm[(r["llueve_manana"], max(p, key=p.get))] += 1
ok = cm[("si", "si")] + cm[("no", "no")]
print(ok, "/", sum(cm.values()), "=", ok / sum(cm.values()), dict(cm))
# 334 / 438 = 0.7625570776255708

Importing predict.py from inside the bundle, as that script does, writes a __pycache__/ next to it — and matrixai verify counts that as a file the manifest does not name, which turns manifest PASS into INCOMPARABLE. Delete it, or work on a copy, before verifying.

---

Reproducing it

Python 3.10 or newer (requires-python = ">=3.10"). Nothing else: no account, no API key, no GPU.

pip install "matrixai-core[export]"

# the published inputs
mkdir -p caso/datos && cd caso
curl -O https://matrixaistudio.org/casos/lluvia/lluvia.mxai
curl -O https://matrixaistudio.org/casos/lluvia/lluvia.mxtrain
curl -O https://matrixaistudio.org/casos/lluvia/dia.json
curl -o datos/lluvia_nombrada.csv https://matrixaistudio.org/casos/lluvia/lluvia_nombrada.csv

# 1. train  (the .mxtrain points at datos/lluvia_nombrada.csv)
matrixai train lluvia.mxai --training lluvia.mxtrain --output runs/v1

# 2. export the self-contained bundle
matrixai export-bundle lluvia.mxai --params runs/v1/params.best.json \
  --outdir paquete --training lluvia.mxtrain --from-run runs/v1

# 3. predict one day, without MatrixAI on the path
cd paquete && python3 predict.py --input ../dia.json && cd ..

# 4. verify — on the ZIP, not on the folder you just built
zip -r paquete.zip paquete
matrixai verify paquete.zip --retrain --locale en   # exit code 3

export-bundle writes a directory and has no zip option, so the zip line is a real step, not a convenience.

Step 4 verifies the zip on purpose: verifying the freshly built folder leaves datos/ on disk next to it, and the retraining stage would quietly pick it up. The project's build script records that this shortcut is what hid a verifier defect — its comment says so in one sentence, in Spanish, and it names one defect, not a number of them. That script is not part of the published case, so this is the one statement on this page you cannot check from the files in this repository.

What that produced here on 2026-08-30, with matrixai-core 1.7.0 rather than the 1.6.0 recorded in the package: identical training line (Best epoch: 11, Best validation loss: 0.533861, Accuracy: 0.762557), byte-identical params.best.json, identical predict.py output to the last digit, and the identical four-stage report with exit code 3. Only the run id differs (it is per run), plus the seven files listed in What is still established.

The published bundle, its literal salida.txt and this exact report are also downloadable, already run, at https://matrixaistudio.org/casos/lluvia/paquete.zip and https://matrixaistudio.org/casos/lluvia/salida.txt.

---

Using it without MatrixAI

The bundle predicts on its own: ONNX plus a standalone wrapper. requirements.txt is numpy>=1.24 and onnxruntime>=1.16 — MatrixAI is not among them.

python -m venv .venv && . .venv/bin/activate    # Windows: .venv\Scripts\activate
pip install -r requirements.txt
python predict.py --input example_input.json    # reproduces expected_output.json

from predict import MatrixAIModel

model = MatrixAIModel()          # reads inference_spec.json next to predict.py
model.predict({"temp_max": 18.0, "humidity": 92.0, "pressure": 1004.0,
               "pressure_delta": -6.0, "cloud_cover": 95.0, "precip_hours": 8.0})

Feed raw, human-readable values: predict.py applies the normalisation and label mapping the model was trained with, from inference_spec.json. Measured outputs, identical from the published bundle and from a freshly rebuilt one:

The raw graph, if you prefer it, expects an already-normalised float32 vector of shape [batch, 6] on input Parte, in the order temp_max, humidity, pressure, pressure_delta, cloud_cover, precip_hours, and returns [batch, 2] on Pronostico_out2, ordered no, si.

Input ranges declared in inference_spec.json: temp_max [0, 45] °C, humidity [30, 100] %, pressure [980, 1040] hPa, pressure_delta [−20, 20] hPa, cloud_cover [0, 100] %, precip_hours [0, 24] h. A value outside its range is clamped to the edge, not rejected — measured: temp_max: 60.0 returns exactly the same probabilities as temp_max: 45.0. It is not silent, though: predict.py --meta reports "clipped": [{"field": "temp_max", "raw_value": 60.0, "normalized_value": 1.0}] (the command prints that JSON with indent=2; it is quoted inline here). Read that metadata if your inputs can leave the range.

---

The model and its training contract

Both travel inside the bundle and are published as text. This is the whole model:

PROJECT LluviaManana

VECTOR Parte[6]
  temp_max: Scalar[0, 45]
  humidity: Scalar[30, 100]
  pressure: Scalar[980, 1040]
  pressure_delta: Scalar[-20, 20]
  cloud_cover: Scalar[0, 100]
  precip_hours: Scalar[0, 24]
END

NETWORK Pronostico
  INPUT Parte
  LAYER Dense units=16 activation=relu
  LAYER Dense units=2 activation=softmax
  OUTPUT llueve_manana: ProbabilityMap[no, si]
END

GRAPH
  Parte -> Pronostico
END

and this is the whole training contract:

MODEL lluvia.mxai

DATASET Observaciones
  SOURCE csv("datos/lluvia_nombrada.csv")
  INPUT Parte FROM COLUMNS [temp_max, humidity, pressure, pressure_delta, cloud_cover, precip_hours]
  TARGET llueve_manana: Label[no, si]
  SPLIT train=0.8 validation=0.2 seed=42
  BATCH size=32
END

LOSS Perdida
  TYPE cross_entropy
  PREDICTION Pronostico
  TARGET llueve_manana
END

OPTIMIZER Optimizador
  TYPE sgd
  LEARNING_RATE 0.1
  UPDATE Pronostico.*
END

RUN
  EPOCHS 40
END

Identifiers are in Spanish because that is what the published files contain; renaming them here would break the digests they are checked against.

MatrixAI can also build a model from a plain-language description. This case was not made that way: it was made from the .mxai and the .mxtrain published above, over a CSV.

---

What is in the package

model_hash mxai_7b9f3359d895e818 · parameter_schema_hash params_91da9c484c855d71 · manifest sha256 93fe1497e44e6e3b17a6ef820e8916781079159a8abd8c57c0f64ab99b9f9762

Full digests are in reproduce.json; manifest PASS means all 15 of them were checked and matched. There is no data_recipe.txt in this package — that absence is the whole point of the case.

---

Licence and links

• Code and model: AGPL-3.0-only. MatrixAI is open source; predict.py carries the same header inside the bundle.
• Data: not AGPL. Open-Meteo's API is free for non-commercial use; the underlying ERA5 data (Copernicus/ECMWF) is published under CC BY 4.0. The published CSV is a selection of columns plus a label derived by the project's authors, and it is not the original data.
• This case, with its recipe-less report, the whole bundle and every artefact to download: https://matrixaistudio.org/casos#lluvia
• The other two cases: Celsius to Kelvin · Readmission risk (synthetic)
• What a receipt proves and what it does not, stage by stage: https://matrixaistudio.org/manual/proof
• Site: https://matrixaistudio.org
• Source (AGPL-3.0): https://github.com/robertollweb/matrixAI
• PyPI: https://pypi.org/project/matrixai-core/ — pip install "matrixai-core[export]"

MatrixAI builds, trains, audits and deploys a neural network from a plain-language description or a CSV, and emits a cryptographic receipt at each step. It is open source under AGPL-3.0, and every exported package predicts with no MatrixAI installed. This is the case where the receipt has to admit what it could not check.

---

Make your own

This model is not a demo to look at — it is a case you can redo, and the tool that produced it is free.

MatrixAI Studio turns a written description, or a CSV you already have, into a neural network you can question: it builds it, trains it, and emits a cryptographic receipt at every step, so anybody can re-check what you claim. It runs on your own machine — no account, no cloud, no API key.

• Download it and build your own: https://matrixaistudio.org/
• Try it in the browser first, installing nothing: https://matrixaistudio.org/demo/
• This case, with every file and command: https://matrixaistudio.org/casos#lluvia
• What it does — and what it does not: https://matrixaistudio.org/what-you-get
• How the receipts work: https://matrixaistudio.org/manual/proof
• Source (AGPL-3.0): https://github.com/robertollweb/matrixAI · PyPI: pip install matrixai-core

If something on this page is not true, it should be visible from the outside. That is the whole point of publishing the package and not just the numbers.