causalscale

causalscale 4.0.0 — replication package

For the KDD 2027 Datasets & Benchmarks submission

Protocol, Not Physics: Re-measuring the Scalability Limit of Differentiable Causal Discovery

Author: Shuaidong Gao (ORCID 0009-0004-5641-3581)

This is the revision. It replaces the earlier release, in which the paper was organised around a twelve-engine toolkit. That framing is gone: the paper now asks a single question — is the widely reported d ≈ 150 scalability limit a property of the methods or of the protocol used to measure them? — and the toolkit is one of the objects being measured, not the contribution.

The version number is 4.0 rather than 3.5 because the defects fixed here are the kind that made the previous release untrustworthy, not additions to it. They are listed under What changed in 4.0 below, each with the check that now pins it.

---

Quick start

The three checks below need numpy and matplotlib only. They do not import PyTorch, do not import this package's engines, and do not touch a GPU. They read the released per-seed records and rebuild everything the paper reports.

pip install numpy matplotlib

python run_all.py --verify      # package integrity + execute the paper's claims
python run_all.py --tables      # every table, rebuilt from the records
python run_all.py --figures     # all three printed figures, from the records
python run_all.py --audit       # only the claim-by-claim assertions

--verify is designed so that it cannot fail for an environmental reason: if the records are present, it reports on the science; optional dependencies are listed but never required. VERIFY.md gives the exact expected output, so a reader can tell a real pass from a silent one.

Re-running the experiments is a separate, expensive path:

pip install -e ".[solvers]"              # torch, dagma, scipy, scikit-learn
python run_all.py --list                 # list the six suites
python run_all.py --suite boundary       # one suite, resumable
python run_all.py --reproduce            # all suites (hours; GPU recommended)

---

What changed in 4.0

Five defects, each of which could produce a wrong answer that looked like a plausible one. All five were found by reading the implementation against its own objective, and each is now covered by an assertion.

Two further changes are about honesty rather than arithmetic, and they are the reason the package now states what it is producing:

• A rank-r output is not called a DAG. output_kind returns "co-variation network (rank-r; not a DAG)" for lowrank, multi_scale and full. Those engines optimise a correlation-reconstruction objective, not h(W) = 0; reading their output as a directed causal graph is the substantive error the paper reports against this package in §4.4.
• The claims of the earlier README are gone. Every number that release advertised and this one cannot reproduce has been removed rather than reworded. docs/ENGINES_LEGACY.md keeps the old engine descriptions, marked unreplicated.

Two conventions are now written down instead of assumed — the coefficient orientation above, and what each output is. CONVENTIONS.md is one page and is the thing to read before reading any matrix this package returns.

---

API surface, read rather than asserted

An earlier description of this package named five entry points. Two of those names exist; three do not, and the real names are different. They are listed here rather than described, so the mismatch cannot recur silently:

---

Paper → package cross-reference

Every number in the paper is rebuilt from experiments/records/*.jsonl. The records are one JSON object per (method, dimension, seed, condition); nothing is aggregated before it is written.

Tables are identified by their LaTeX label first, because the printed number depends on how the paper is assembled: the protocol card is tab:card, so tab:main prints as Table 2, not Table 1.

The claims themselves are executable:

python run_all.py --audit

This does not merely re-derive the numbers; it states them as predicates over the records and fails if a sentence the paper relies on stops holding. A reproduction script shows that the numbers can be regenerated. An audit shows what would be false if they were not.

---

Layout

---

What is and is not claimed

• Claimed. Under one fixed protocol, a plain NOTEARS solver does not return an empty graph at d = 150; the collapse that is normally reported is dominated by the sparsity penalty, a scalar that was never swept, and not by the two causes usually blamed (an exhausted iteration budget, an unsatisfied constraint).
• Not claimed. That NOTEARS scales to genome-wide resolution, or that this package establishes a new state of the art. The obstruction past d ≈ 500 is stated as identifiability, not compute.
• Not re-measured. The other engines in docs/ENGINES_LEGACY.md are carried over with their original, self-reported numbers. They were not independently re-run and should not be read as evidence.

All experiments were run by one author on a single 8 GB GPU. Timings and out-of-memory thresholds are hardware-specific; the F1 comparisons are not, and every record states the seed that produced it.

---

Citation

@inproceedings{gao2027causalscale,
  title={Protocol, Not Physics: Re-measuring the Scalability Limit of
         Differentiable Causal Discovery},
  author={Gao, Shuaidong},
  booktitle={Proceedings of the 33rd ACM SIGKDD Conference on Knowledge
             Discovery and Data Mining (KDD), Datasets and Benchmarks Track},
  year={2027}
}

License

MIT. See LICENSE.