Loom-Crucible-Preview

<div align="center"> <img src="banner.jpg" alt="Loom Crucible Preview" width="520"> </div>

Loom Crucible Preview

<img src="logo.jpg" alt="" width="20" height="20" style="border-radius:4px;vertical-align:middle;margin-right:6px;"> 155.0M parameters · 52 layers · 1024 context · Textile Labs

The biggest Loom by far, and a preview of how far our growth method can push a small, disciplined model. We took Loom Tapestry 3 (69.2M) and grew it in both directions at once, wider (384 → 512) and deeper (40 → 52 layers). The growth is function-preserving: the 155M model started out producing exactly what Tapestry 3 produced, then trained for 12 hours on a rebuilt corpus.

It calls itself Loom Crucible. It reads what a search brings back and answers from it, says when it looked something up, says when it can't know something about you, and stays a Loom: plain, brief, and not a story generator.

Trained from-scratch lineage (no third-party checkpoint), on a Kaggle dual-T4. Runs offline via Ollama; llama.cpp-compatible.

you            who wrote dracula
Loom Crucible  <lookup>wrote dracula</lookup>
harness        ← Dracula is an 1897 Gothic horror novel by Irish author Bram Stoker.
Loom Crucible  Bram Stoker. I had to look that up.

What "Preview" means

Crucible clearly beats every earlier Loom on its core skills: reading a passage correctly (9/10 vs Tapestry 3's 4/10) and the acceptance battery (125/133). It is not yet better end to end on live Wikipedia (10/20 held-out vs Tapestry 3's 12/20). On that test the limit is now the search step, not the reading step: when the search found the answer, both models got it right about 85% of the time. A better harness is the next release, and it helps every Loom.

How it was grown

Cold-training a big model on little data fails. Instead we grow a trained small one:

• Wider, 384 → 512. New channels start at zero so they cannot disturb the old ones. Two details made this exact rather than approximate. First, the 2 new attention heads per layer are placed inside their key/value groups, because appending them silently re-pairs the old heads with the wrong keys. Second, every RMSNorm weight is scaled by √(384/512) and ε by 384/512, because zero-padding changes the norm.
• Deeper, 40 → 52. 12 new layers are inserted as no-ops (zeroed output projections).

At step 0 the 155M model matched Tapestry 3 to 3e-4 in the logits and 1e-6 in the loss. All twelve hours went into getting better, not starting over.

Measured behaviour

The family, same harness, 2026-09:

† Crucible's score is the first on a fully clean held-out battery. While building its corpus we found that some battery prompts had leaked into the training data used since Tapestry 3. We removed them for Crucible, so earlier scores are somewhat optimistic by comparison.

The acceptance battery, row by row:

Held-out reading bench (hand-written passages with distractors, e.g. a country's capital next to its largest city, a start year next to an end year; none in any training data):

End to end, everyday questions it had never seen, live Wikipedia, model writing its own query, scored on the final answer: tuning 11/20, held-out 10/20 (Tapestry 3: 12/20, 12/20). When the search found the answer, Crucible got it right 10 times out of 12.

Read this before you use it

Every point measured.

• It is not better than Tapestry 3 end to end yet. It reads better, but live results are capped by the search step. Run harness.py --show to see the sentence it read.
• When a search comes back without the answer, it usually answers anyway. Given an on-topic passage that lacks the fact, it says so only 3 times in 10. If the passage holds something of the right type (a number for "how tall", a year for "when did it end"), it tends to grab that. Trust the sentence shown by --show, not just the reply.
• With tools off, it sometimes answers famous facts from half-memory (3/8 held). Those answers are often wrong. Keep tools on for facts.
• It once refused a passage that did hold the answer (1 of 5 battery reading rows).
• It is not a calculator. Arithmetic is out of scope; use a tool.
• Harness search is Wikipedia only: no time, weather, news or prices.
• Plain, brief voice by design. It is a small reader and assistant, not a chat stylist.

What it does reliably: knows what it is, stops on its own, holds a conversation, answers follow-ups from the same result (5/5), never claims a lookup it didn't make (16/16), and reads a supplied passage better than any earlier Loom.

Usage — the harness

python3 harness.py "who wrote dracula"
python3 harness.py --show "how tall is ben nevis"
python3 harness.py --no-tools "who are you"

Stdlib + certifi. Wikipedia needs no API key.

Usage — Ollama

ollama run hf.co/textilelabs/Loom-Crucible-Preview "who are you"

template and params are read automatically.

Usage — transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("textilelabs/Loom-Crucible-Preview")
model = AutoModelForCausalLM.from_pretrained("textilelabs/Loom-Crucible-Preview").eval()
eot = tok.convert_tokens_to_ids("<|eot|>")
def ask(message, tools=False):
    p = f"<tools:{'on' if tools else 'off'}>\n<user>\n{message}\n<|eot|>\n<loom>\n"
    ids = tok(p, return_tensors="pt", add_special_tokens=False).input_ids
    with torch.no_grad():
        out = model.generate(ids, max_new_tokens=64, do_sample=False, eos_token_id=eot,
                             pad_token_id=tok.convert_tokens_to_ids("<|pad|>"))[0]
    return tok.decode(out[ids.shape[1]:], skip_special_tokens=False).replace("<|eot|>","").strip()

Prompt format is exact: <tools:off>\n<user>\n{message}\n<|eot|>\n<loom>\n.

How it was built

Files

config.json / model.safetensors           the model
tokenizer.json / tokenizer_config.json    custom BPE tokenizer, 16,384 tokens
loom-crucible-preview-f16.gguf            for Ollama / llama.cpp (lookup tags USER_DEFINED)
harness.py                                 runnable search harness
template / params                          read automatically by `ollama run hf.co/...`
Modelfile                                  for building locally
ATTRIBUTION.md                             required credits for the training corpora

License

Model: MIT. Training data retains its original licences and attribution.