tetracta-znext-1b

Model X-Ray evidence correction — 6 September 2026. Any structural-location, knowledge-separation, portrait-visualization, lesion-response, legacy simulated-quantization, robustness or prior X-Ray endorsement previously linked from this card has been withdrawn. It is not current evidence. No replacement result is published while validation remains pending. Correction record.

<!-- vg1-card-note-20260922 -->

Model X-Ray is a separate Tetracta service and is not what this card documents. The presence of this repository is not an eligibility grant for it, and nothing measured here is a Model X-Ray result. Its current scope, status and limits: scope and report guide · service status · dated correction record.

<!-- /vg1-card-note-20260922 -->

Tetracta Z-Next 1B (v4.3)

This repository contains no model weights. It is a model information card: architecture class, measured benchmarks, measured memory/concurrency behaviour, training provenance, and an honest list of limitations. The weights are not published at this time.

Full results page (tables, fairness gates, limitations): https://www.tetracta.ai/znext.html You can, however, run the model yourself right now: → https://www.tetracta.ai/zchat

---

"So it's Mamba with a new name."

No — and we are also not claiming to have invented constant-state sequence modelling. That family exists, it is published, and Mamba and RWKV are its best-known members. The claim here is narrower and checkable: a set of measurements that, as far as we can tell, are not published elsewhere in this form.

1. A same-data transformer twin, trained by us, as a control. Architecture papers are usually compared against public checkpoints trained by other teams on other corpora with other tokenizers and recipes. We trained a deliberately ordinary transformer on the same corpus, same tokenizer, same budget, same harness — and we publish it, including the rows where it beats us.
2. Robustness to instruction tuning appears to differ by architecture. One recipe, applied unchanged to three architectures, moved them in three directions: ours +0.51 pp (95% CI [+0.04, +0.98]), our previous generation −0.43 ([−0.89, +0.04]), the transformer twin −1.39 ([−1.97, −0.79]). One run per architecture, one recipe, no seed repeats — we do not know yet whether it generalises.
3. The concurrency claim is a measurement on a real server, not an asymptotic argument. Seats were counted from real free VRAM on one consumer card running a public chat service, and we publish the caveats that cut against the number.
4. We publish where our own measurements come out negative. The limitations section is longer than the results section.

What we do not claim, and would like to fix. We have not run a budget-matched head-to-head against Mamba or RWKV. We ran state-spaces/mamba-1.4b-hf through our own harness and it scores 56.40 macro against our 45.24 — but it is a finished public release on a far larger budget, so that number answers a different question than ours. A matched-budget comparison inside the constant-state family is exactly the experiment we cannot afford alone, and it is near the top of the list of things we would run with support.

Model summary

What it is, in one paragraph

Z-Next is a decoder-only language model that does not use attention and keeps no KV cache. A conversation is carried in a fixed-size session state — the memory a session occupies does not grow with the length of the conversation or the length of the input. Architecturally it is a distant relative of the recurrent / state-space family in the sense that it carries a bounded running state rather than re-reading a growing history, but the internal mechanism differs from published models in that family and is not disclosed here. The claim of this page is not "better quality". It is a measured cost/behaviour profile at matched training budget — and the measurements, including the unflattering ones, are given below.

---

Intended use

Intended. Research into attention-free / constant-memory sequence models; evaluation of the class at 1B scale; benchmarking against budget-matched baselines; technical due diligence by potential research or commercial partners; hands-on inspection of runtime memory behaviour through the live demo.

Not intended. Production deployment, factual question answering, anything safety-critical, or any use that depends on the output being correct. This is a 1B-parameter model trained on 10B tokens. It hallucinates heavily and in a manner appropriate to that scale (see Limitations). There is no safety tuning, no RLHF, and no content filtering beyond what the pre-training corpus implies.

Out of scope. Non-English use. The corpus is English (FineWeb-Edu + English Wikipedia) and every evaluation reported here is English.

---

What is different — stated only through measurements

Three things are measured, and only these three are claimed:

1. Session memory is constant. A session's state occupies 5.227 MiB and does not change with context length. This was verified by raising the input from 952 to 31,926 tokens: the session state was byte-identical at every point (table below). This is a measurement, not a projection.
2. On one 16 GB consumer GPU, that constant cost translates into far more parked sessions than a KV-cache transformer of the same size — 727 vs 14 in the measured head-to-head configuration (≈52×). The conditions under which this ratio holds, and the conditions under which it shrinks, are spelled out below. We state the strongest counter-argument ourselves.
3. At a matched (in fact slightly unfavourable) token budget, quality is not the price paid. The macro score over 10 standard zero-shot tasks is above both budget-matched Pythia reference points with intervals clear of zero, and level with the same-data transformer baseline base-to-base (+0.62, 95% CI [−0.06, +1.28] — a tie, not a win). After the same instruction-tuning recipe the gap against that baseline is +2.51 [+1.79, +3.25], but read that one together with the fact that the recipe cost the twin 1.39 points. The defensible claim is "quality is not the price paid", not "quality is better".

Everything else on this page is context, caveat, or a negative result.

---

Benchmarks

Evaluation contract

• 10 standard zero-shot tasks, full test/validation sets, no sub-sampling (n_limit = 0): LAMBADA 5,153 · HellaSwag 10,042 · PIQA 1,838 · WinoGrande 1,267 · ARC-Easy 2,376 · ARC-Challenge 1,172 · OpenBookQA 500 · SciQ 1,000 · BoolQ 3,270 · MMLU 14,042.
• Single metric contract applied identically to every row: acc for LAMBADA, WinoGrande, ARC-Easy, BoolQ, MMLU; acc_norm for HellaSwag, PIQA, ARC-Challenge, OpenBookQA, SciQ. (Using acc for PIQA instead moves every macro by ≤0.14 p.p and changes no conclusion. We recomputed every row from the raw per-item outputs under one contract rather than copying numbers between reports.)
• Same harness, same seed (1234), pure fp32, TF32 disabled, context ceiling 1024 for all models.
• Macro = unweighted mean of the 10 task scores. Per-model 1σ on the macro ≈ 0.38 p.p. For differences between two models we report paired bootstrap 95% CIs over per-item correctness (4,000 resamples), not the single-model σ.

Main table

Pythia rows were measured by us with the same harness, not copied from the literature (EleutherAI/pythia-1b and EleutherAI/pythia-1.4b, revision step5000 = 10.49B tokens, chosen to match our budget).

Paired differences (95% CI, paired bootstrap over per-item correctness)

The last three rows are the honest form of a frequently over-claimed point. We do not appeal to a literature generalisation about instruction tuning lowering zero-shot benchmarks. We ran the same recipe on three architectures and report what happened: it lowered two of them and raised one.

The budget comparison is tilted against us

Z-Next saw the fewest tokens of any model in the table. It is also smaller than both Pythia reference points. We are not claiming an equal-budget comparison; we are claiming a comparison in which our side had less budget.

Two models we measured and deliberately left out of the table

The same harness, seed, 1,024-token ceiling and gates were also run on two fully trained public releases. Under the identical metric contract:

That is 11.2 and 3.3 points above our base model. They are excluded from the main table because their pre-training budgets are orders of magnitude beyond 10B tokens, so placing them beside a budget-matched comparison would be meaningless — but excluding them silently would be worse, so here they are.

This also answers the obvious question about family membership. Mamba is the closest published member of the constant-state family we say we belong to, and at 1B scale we have no quality argument against a version of it trained on far more data. What this card claims is a matched-budget comparison and a memory profile, not a quality win over the fully-trained field. We did not independently verify either model's token budget, so we quote no number for it.

Additionally, the 1024-token context ceiling in this suite comes from the transformer twin's positional limit and is imposed on everyone. Z-Next's own context behaviour is therefore not exercised at all by this benchmark suite.

Perplexity

WikiText-103 — word-level perplexity and bits-per-byte are tokenizer-independent, so this comparison is methodologically valid across all models including Pythia.

⚠ Version caveat, stated plainly: WikiText-103 was never measured on the finished 10.00B Z-Next v4.3 checkpoint. The most advanced point we measured is 94% of the budget. It is shown here labelled as such and must not be attributed to the final model. Against the budget-matched Pythias, the gap is large and the comparison is sound; against our own twin and our own previous generation, the gap at 94% is practically closed (+0.4%) but not closed in our favour.

LAMBADA perplexity — this is a token-level PPL, so it is comparable only between models sharing our tokenizer. Pythia is deliberately excluded.

Note the asymmetry that runs through this whole card: on LAMBADA accuracy the model is clearly behind Pythia (30.5 vs 36.6 and 42.2), while on LAMBADA perplexity it is well ahead of the twin. Accuracy-based benchmarks do not flatter this architecture; we did not select a metric that does.

Long context — identical windows, last 256 tokens scored (loss, nats, lower is better)

Read this table honestly in both directions. Z-Next v4.3, at 74% of its budget, had lower loss than the finished transformer twin at every length. It also had higher loss than our own previous generation at every one of the five lengths. We are not dropping that column.

⚠ Two limits on this table. (i) The windows are drawn from the same corpus all three models were trained on — this is not a held-out set, so none of these numbers is a generalisation measurement. The text is identical for all three, which keeps the comparison symmetric, but the three models did not consume identical fractions of that corpus (10.00B draws for us, 10.32B for the other two). (ii) No confidence interval was computed. Each cell is 16 windows × 256 scored tokens and the differences being pointed at are 0.02–0.05 nats. The evidence is that the ordering is consistent across all five lengths, not that any single gap was interval-tested.

---

Memory and concurrency (measured, not projected)

Hardware: one NVIDIA GeForce RTX 5070 Ti. Torch reports 15,851 MiB total, 15,608 MiB free at idle. In the head-to-head configuration the measured free VRAM is split evenly between the two models (7,804 MiB each), and 350 MiB per side is reserved for transient generation working memory (peak measured transient usage: 266 MiB). Both models are fp32 — the comparison is dtype-symmetric.

Same card in single-model mode (whole card given to Z-Next): 2,220 sessions (reproduced in 10 separate runs).

Constant memory under growing input — the direct measurement:

A 33× longer input leaves the session state bit-for-bit unchanged. The 266 MiB is shared transient working memory, not per-session.

How to read the 52× — caveats we impose on ourselves

• A seat is a memory admission limit, not throughput. 727 seats means 727 conversations can have their state resident simultaneously. It does not mean 727 concurrent token streams; generation on a single GPU is serialised.
• 727 was never literally opened. It is (measured free budget ÷ measured constant per-session cost). The highest number of Z-Next sessions actually opened live is 20, at which point VRAM linearity was confirmed to within ±1 MiB. By contrast the twin's 14 was driven into the wall end-to-end (capacity+3 sessions attempted; the excess got HTTP 503 and the declared ceiling was never exceeded).
• The single biggest lever is the twin's declared context ceiling, and the ratio scales with it almost linearly. At the same budget: 256 tok → ~13×, 1,024 tok → 52× (measured), 2,048 tok → ~104×, 4,096 tok → ~242×. Everything except the 1,024-token row is arithmetic extrapolation from the measured per-token KV cost, not a measurement.
• The strongest counter-argument, stated by us: the twin in this comparison is a straightforward implementation that allocates the entire declared ceiling at session open. Production serving stacks with paged/block KV allocate as the conversation grows, so on short conversations they will host many more sessions than 14 and the ratio will drop substantially. The constant-memory property remains, but the headline multiplier is configuration-dependent.
• Quantising only the twin's KV cache (int8/fp8) would cut the ratio by 2–4×. Both sides here are fp32.
• Capacity is read from actual free VRAM at startup, not from a nominal table; if something else is on the card, seats are reduced and a warning is printed, and if CUDA OOMs at runtime the declared capacity is lowered to the observed reality. Measured example: with 5,151 MiB of other work on the card the numbers became 258 ⇄ 4 = 64× — counter-intuitively the ratio rises on a tighter card, because the twin's 256 MiB grain rounds the remaining budget more coarsely.
• An earlier published figure of "750 ⇄ 14 = 54×" came from a pre-fix build that read the budget from the card's nominal total rather than its actual free memory. That bug is recorded in our own notes and the corrected number is 727 ⇄ 14 = 52×. We publish the corrected one.

---

Training

Pre-training

Corpus

• 10.33B tokens, from two fully public sources: HuggingFaceFW/fineweb-edu (sample-100BT) and wikimedia/wikipedia (20231101.en).
• No synthetic data — deliberately excluded.
• The corpus is English; every evaluation on this page is English.
• Training windows were sampled at random with replacement. We therefore do not claim "one epoch" or "no data repetition"; expected corpus coverage is ≈62%. The defensible statement is: the corpus holds 10.33B tokens and the model consumed 10.00B, so this run does not rely on volume-scale data repetition.
• The mixture ratio between the two sources was a collection target and the realised ratio was never measured, so no ratio is quoted.

Instruction / chat tuning

The benchmarked instruction-tuned checkpoint saw instruction data equal to 0.033% of the pre-training token budget. The ~$12 figure is the only truly invoiced cost number in this card; the pre-training dollar figure above is a derivation.

---

Evaluation methodology — fairness gates

Every benchmark run records four gates alongside the scores:

Honest note on K4: it only applies to models that have an accelerated path of ours. For the transformer twin and for Pythia the field is null — the question does not exist for them. So the correct statement is not "all four gates passed for all models"; it is "K1–K3 passed for every model, and K4 is a question that only exists for our own runs, where it was measured and passed."

Honest note on K3: the 1,024-token ceiling is the twin's structural limit and it is imposed on everyone including us. Also, "no item was truncated" is true for our models and the twin, but 2 Pythia items were truncated — a negligible effect, but the absolute phrasing would be wrong.

Comparison validity — three mandatory caveats

1. The twin is not shape-identical to Z-Next. It was trained on the same data with the same budget and the same recipe, but the depth/width profile is not matched one-to-one (the numbers are withheld). What was measured is therefore "architecture
   • shape" as a package, not architecture alone.
2. Not all rows come from one sitting. The five base/instruction-tuned rows were measured in a single session on 19 Aug; the twin-base and previous-generation-base rows come from a separate session on 18 Aug — same harness, pure fp32, TF32 off, same seed (1234), but not the same run.
3. Pythia was trained on a different corpus (the Pile). What is equalised is the token budget and the measurement protocol — not the data.

---

★ Limitations and honest notes

This section is deliberately as long as the results section.

1. The live demo does not run the benchmarked checkpoint. The classic benchmark suite was never run on any chat-tuned build. The 45.8 macro belongs to the instruction-tuned checkpoint, not to the model you will chat with, and the model you will chat with has no macro score at all. The demo is started with one of our two most recent chat builds (20 or 21 Aug), chosen at launch, so we cannot promise in advance which one answers you. The chat/tool numbers in items 12–13 were measured on the 20 Aug build; the 21 Aug build has since been through the identical gate suite and its numbers are given inside item 12 — they are materially better than the 20 Aug ones, and we are stating that rather than leaving the more pessimistic figure standing as if it described the current build.

2. Long-range retrieval in free-form natural language is not solved. On a format-neutral natural-language needle test, instruction tuning produced no measurable improvement at all: base 0.306 vs tuned 0.301, Δ = −0.005, p = 0.92, chance = 0.25, n = 216 per model. All three models score near chance. Broken down by depth, all three only work in roughly the last 15% of the context (0.58–0.92 there) and fall below chance at head and middle depths. We can measure a 5.5× gain on a structured retrieval curve, but it does not transfer to the neutral format. We therefore explicitly do not claim that recall is solved.

3. Retrieval decays with the gap after the fact, in the structured test too. Base model, by gap length: 280 tok → 1.000 · 1,210 → 1.000 · 2,202 → 0.583 · 3,194 → 0.292 · 7,007 → 0.083. The best tuned variant reaches 0.458 at a 7,007-token gap (chance ≈ 0.016). Real improvement, still a monotonically falling curve. The problem is not the number of stored facts but the gap after them: with 2 records, FAR placement scores 0.042 and NEAR placement scores 1.000.

4. On the live demo pipeline, needle retrieval in a natural document ends after a few thousand tokens. Measured on the 20 Aug chat-tuned build through the demo pipeline. You can reproduce this yourself:

Our internal reading is that this reflects a curriculum limit rather than an architectural one (the long-gap curriculum was a small fraction of the last tuning run). That reading is a hypothesis, not a measurement; the table above is the measurement.

5. Compositional / multi-hop inference is essentially absent. Two-step chained questions score 0/24 across the base and both tuned checkpoints; direct single-hop scores 1–3/24. This is an open capability gap at this scale, not a measurement artefact.

6. MMLU does not discriminate at this scale. Every 1B-class model in our table scores 22.97–25.11 against a 25.0 chance floor. It drags every macro down equally and distinguishes nobody. Read the macro with that in mind.

7. Instruction tuning made raw-corpus perplexity ~17% worse (2.7438 → 2.9013 nats; PPL 15.5 → 18.2). This is the expected distribution shift, but it belongs next to the "+0.5 macro" line, not hidden from it. The context gain over long windows was retained (−0.0497 → −0.0564 nats).

8. Two comparisons we do not get to call wins. (a) Against our own previous generation: +0.45 p.p, 95% CI [−0.22, +1.11] — equal. That generation is also ahead of us on LAMBADA perplexity (31.59 vs 42.54) and on long-context loss at all five measured lengths. (b) Base against the twin's base: +0.62 p.p, 95% CI [−0.06, +1.28] — the interval contains zero, so that is a tie too. The only architecture comparison against the twin that clears its interval is the one taken after the shared instruction-tuning recipe (+2.51), and a large part of that margin is the twin losing 1.39 points to the recipe rather than us gaining 0.51.

9. WinoGrande — a recall-signature task — is behind the twin. 51.6 vs 53.2 (−1.6 p.p) for the finished base; the gap was −3.6 p.p at the 74% checkpoint.

10. LAMBADA accuracy is well behind budget-matched Pythia (30.5 vs 36.6 and 42.2), even though the macro is ahead. Top-1 hit rate is 46.34% vs the twin's 46.75% (measured at the 88%-of-budget checkpoint, not on the finished base) — the twin is ahead on easy tokens. Accuracy-thresholded benchmarks understate this architecture and perplexity-based ones flatter it; both are shown.

11. Output-format following is a separate, real gap. A required [CODE: <answer>] output format was not produced in any of 5 seeds × 5 settings, although the correct answer still appeared in the surrounding prose in 80% of those runs. The model answers and ignores the imposed format. Measured on the 20 Aug chat-tuned build only; not re-run on the other checkpoints, so we cannot say whether it is a property of the line or of that build.

12. Mid-session tool chaining broke on one build and largely recovered on the next. In the 20 Aug chat-tuned build, end-to-end tool use scores 0.033–0.067 against a pre-registered threshold of 0.40, and 0.000 on the worst seed — while the same task in a fresh context scores 0.600. Measured root cause: a previous turn's result overrides the current argument. A related value-leak metric sits at 0.089–0.097 against a pre-registered threshold of 0.060–0.070 (down from 0.181–0.197 in the prior version — halved, still failing). Thresholds were registered before the run and were not moved afterwards.

Update, and it goes in our favour — which is exactly why it is here. The 21 Aug chat-tuned build has since been run through the identical gate suite (15 seeds, same temperature, same mixed-session harness): mid-session end-to-end tool use 0.567 (clears the 0.40 gate; worst of 15 seeds 0.500), call rate 1.000, chat recall 0.944, refusal rate 0.031. So "the session destroys the capability" was a property of one build, not of the architecture, and we are correcting it rather than letting the gloomier number stand. Still failing on that build: value leakage 0.067 against a 0.060 threshold; topic-holding in a long mixed session 0.658 against 0.680; worst-seed chat recall 0.667 against a 0.700 floor. 3 of 17 pre-registered gates fail. And the classic benchmark has still never been run on it.

13. Conversational memory is limited by format, not distance. In an ~800–920-token, 28-turn chat, recall measured 0.633–0.700, while the same distance presented in a structured format scored 1.000. The 20 Aug chat-tuned build raises chat recall to 0.933–0.967, but that too was measured at ~800 tokens of context, not at long context — and, per item 1, not on the build the demo now serves.

14. It hallucinates, heavily, in a manner appropriate to 1B/10B. Observed: the Moon's orbital period given as "21 days"; "the first three primes are 1, 2, 3"; 8102 × 47 rendered as "380,793 … 3807.680" (correct answer 380,794). Chat quality is not product-grade and we do not present it as such. The claim of this work is the cost/behaviour profile, not the quality.

15. No safety work has been done. No RLHF, no red-teaming, no content filtering, no bias evaluation. Corpus biases from FineWeb-Edu and English Wikipedia pass through unmitigated.

16. Version hygiene. Numbers measured on earlier generations of this architecture (inference-latency figures in particular) are not reproduced on this card, because they were not measured on this model. Where a number comes from an intermediate checkpoint (74% or 94% of budget), it is labelled inline. We do not substitute one version's measurement for another's.

---

The plan we are asking to fund

A compute grant is easier to judge when it buys a specific experiment rather than a direction. Three stages, each with a stop condition written before the run:

We are asking for stage 1 only: ≈720 GPU-hours. It answers one question — what six times the token budget does to this architecture — and we publish the result either way.

Compute figures are extrapolated from the measured run (947M parameters, 10.0B tokens, 29.97 h on 4×H200 ≈ 120 GPU-hours) by scaling throughput with parameter count. They are estimates, not measurements.

Live demo

→ https://www.tetracta.ai/zchat

• Runs on a single consumer GPU at home, with Z-Next and the same-data transformer twin loaded side by side on half the card each, so the seat counts above are visible live.
• CAPTCHA + proof-of-work gated; the demo is off by default and switched on manually, so it may be closed when you visit.
• The demo runs a chat-tuned checkpoint — one of our two most recent (20 or 21 Aug), chosen when the engine is started. No chat-tuned build has ever been run through the classic benchmark suite, so the 45.8 macro is not its score and it has none. Its chat and tool-use gates have been measured on both builds: the 21 Aug build clears the tool-use gate the 20 Aug build failed, and still fails 3 of 17 gates. See Limitation 1 and item 12.
• Useful things to try: paste a very long document and watch the session memory stay flat; then try to make it retrieve a fact from the middle of that document, and watch it fail past a few thousand tokens. Both behaviours are on this page.

---

Contact / collaboration

Tetracta is scaling this architecture beyond 1B and is looking for compute partners, research collaborators, and technical due-diligence conversations.

• Site: https://www.tetracta.ai — research page: https://www.tetracta.ai/znext.html
• Live demo: https://www.tetracta.ai/zchat

What we will share under agreement: full evaluation harness details, raw per-item benchmark outputs, training telemetry, reproduction protocol for every table on this page, and the corpus construction procedure (both sources are public).

What we will not share publicly: the internal mechanism of the architecture, its shape parameters, and its inference implementation details. This card is deliberately built so that its persuasiveness comes from measurement rather than from mechanism — if the numbers above are not convincing on their own, the mechanism would not fix that.

---

Citation

@misc{tetracta_znext_1b_v43_2026,
  title  = {Tetracta Z-Next 1B (v4.3): an attention-free, KV-cache-free language model
            with constant-size session state},
  author = {Tetracta},
  year   = {2026},
  note   = {Model card and benchmarks; weights not released.
            Live demo: https://www.tetracta.ai/zchat},
  url    = {https://www.tetracta.ai/znext.html}
}

Last updated: 2026-08-21. Every figure on this page is traceable to a measurement artefact in our internal records; figures that are derivations, extrapolations, or intermediate-checkpoint readings are labelled as such inline.