filedfact-qwen3-8b

FiledFact-Qwen3-8B-LoRA

A research-preview specialist model for SEC/XBRL financial fact extraction. FiledFact-Qwen3-8B-LoRA is a LoRA adapter for Qwen/Qwen3-8B that reads a passage of SEC filing text and emits the XBRL-aligned numeric facts it contains as structured JSON - concept, full-unit value, unit, period, and segment dimensions. It was fine-tuned on the FiledFact corpus, the parent of the StockAlloy/filedfact-100k and StockAlloy/filedfact-passages datasets.

It is a specialist, not a general model: it is not a question-answering system, not an advisor, and it is not production-ready. Its outputs require human review before use.

On a held-out-by-company benchmark, fine-tuning improved Fact-F1 from 0.325 to 0.829 and paired concept accuracy from 0.035 to 0.461.

🚀 Try it live: FiledFact demo Space - paste a filing passage, get the facts back as JSON. No setup.

Intended use

• Research on XBRL grounding, financial information extraction, and structured generation.
• A fact-layer generator inside a pipeline: extractor (this model) → reasoner (a general model) → verifier (checks values against the source document).
• Error analysis and benchmarking alongside the FiledFact-100K and FiledFact-Passages datasets.

Not intended for:

• Investment advice or investment decisions of any kind.
• Chat or general question answering.
• Arithmetic or multi-hop reasoning over filings.
• Fully autonomous filing-processing systems.

Quick start

Transformers + PEFT

The adapter's config records its training base as unsloth/qwen3-8b-unsloth-bnb-4bit (a 4-bit mirror of Qwen/Qwen3-8B); it loads fine onto Qwen/Qwen3-8B.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3-8B"
tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "StockAlloy/filedfact-qwen3-8b", subfolder="adapter")

# PROMPT_TEMPLATE is the training prompt from the "Prompt format" section below
prompt = PROMPT_TEMPLATE.format(form_type="10-K", filed_at="2024-11-01",
                                pre_context="(none)", heading="Results of Operations",
                                text=passage)
text = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,  # REQUIRED - without this, Qwen3 thinking blocks pollute the JSON
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=8192, do_sample=False)  # greedy; dense passages can need >4096 output tokens
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Ollama (local)

Convert the adapter with llama.cpp and layer it on the stock qwen3:8b:

python llama.cpp/convert_lora_to_gguf.py <adapter-dir> --base-model-id Qwen/Qwen3-8B --outtype f16

# Modelfile
FROM qwen3:8b
ADAPTER ./filedfact-qwen3-8b-adapter.gguf
PARAMETER temperature 0
PARAMETER num_ctx 8192

ollama create filedfact-qwen3:8b -f Modelfile

Append /no_think to the end of the prompt when running through Ollama (this disables Qwen3 thinking, matching how the model was trained and evaluated). The Ollama build applies the adapter over a Q4_K_M-quantized base; expect somewhat lower accuracy than the benchmarked configuration (see Evaluation setup).

Prompt format

The model was trained and evaluated with exactly this prompt. Use it verbatim; other phrasings are out of distribution:

You are a financial data extraction system. Below is a passage from an SEC filing ({form_type}, filed {filed_at}). Extract EVERY numeric financial fact that appears in the passage.

Return ONLY valid JSON in this exact shape:
{"facts": [{"concept": "...", "value": 0, "unit": "...", "period": "...", "dimensions": [{"axis": "...", "member": "..."}]}]}

Rules:
- concept: the US-GAAP/IFRS XBRL concept name if you know it (e.g. "RevenueFromContractWithCustomerExcludingAssessedTax"); otherwise a precise CamelCase label.
- value: the numeric value in FULL units. Expand the table scale: if the table is "in thousands" and a cell shows 1,634 the value is 1634000. Use negative numbers for values shown in parentheses.
- unit: ISO currency code (USD, EUR, ...) for monetary values; "shares" for share counts; "pure" for ratios.
- period: "instant:YYYY-MM-DD" for point-in-time balances, or "duration:YYYY-MM-DD..YYYY-MM-DD" for flows over a period.
- dimensions: the slice the value belongs to (business segment, geography, product, plan type, share class, ...) as axis/member pairs. Use [] for company-wide totals.

CONTEXT IMMEDIATELY BEFORE THE PASSAGE (may contain the table caption and scale, e.g. "in thousands"):
{pre_context}

SECTION HEADING: {heading}

PASSAGE:
{text}

Fill {form_type}, {filed_at}, {pre_context} (use (none) if unavailable), {heading}, and {text}.

Note on the prompt's wording: despite the "EVERY numeric financial fact" instruction, the model returns only the XBRL-tagged facts. Other numbers - dates, page numbers, untagged figures - are skipped, but the model reads them as context: a year in a column header or an "in thousands" cue tells it which period or scale a fact belongs to. Two quirks to expect: XBRL sign conventions can flip a value's polarity versus the displayed sign (see Limitations), and dimension objects sometimes carry an extra member_label field, which parsers should tolerate.

Evaluation

Held-out-by-company benchmark: a fixed 200-passage subset spanning companies never seen in training, scored with value-anchored fact pairing and per-field accuracy. "Full-fact" means concept, value, unit, period, and dimensions all correct at once, with no partial credit.

Evaluation setup

All models were run with the same prompt and scored with the same scorer on the same passages; confidence intervals are computed via passage-block bootstrap. The baseline configuration is a reasonable default rather than a per-model optimization; a higher reasoning-effort setting or model-specific prompt tuning could move the baseline numbers.

Results (best value per row in bold)

Against gpt-5.5, the differences in precision, concept accuracy, dimensions, and full-fact exactness are statistically significant; overall Fact-F1 is a statistical tie - we describe it as matching gpt-5.5, not beating it - and gpt-5.5 keeps higher recall. In practice: the fine-tune extracts fewer wrong facts and identifies which fact a number is far more reliably, while gpt-5.5 recovers more of the long tail.

Cross-architecture check (Gemma-2-9B)

The identical recipe applied to a second 9B architecture produces the same pattern, indicating the improvement is data-driven rather than architecture-specific (measured with an earlier adapter build on an earlier benchmark revision; retained as an architecture-generality check):

The Gemma adapter weights are not released because Gemma derivatives carry the Gemma Terms of Use.

Example

A real passage from a Johnson Controls 10-Q (filed 2023-02-01) - a segment-revenue table. The pre-context carries the scale cue:

Pre-context: …The following table presents further disaggregation of Global Products segment revenues by product type (in millions):

Passage:

| Three Months Ended December 31, |
| 2022 | 2021 |
HVAC | $ | 1,440 | $ | 1,483 |
Fire & Security | 570 | 544 |
Industrial Refrigeration | 70 | 49 |
Total | $ | 2,080 | $ | 2,076 |

Verbatim output from the Ollama build (temperature 0), abridged to the first, third, and seventh of eight extracted facts:

{"facts": [
  {"concept": "RevenueFromContractWithCustomerExcludingAssessedTax", "value": 1440000000, "unit": "USD", "period": "duration:2022-10-01..2022-12-31", "dimensions": [{"axis": "ProductOrServiceAxis", "member": "ext-0001524472#HVACMember", "member_label": "HVAC"}, {"axis": "StatementBusinessSegmentsAxis", "member": "ext-0001524472#GlobalProductsMember", "member_label": "Global Products"}]},
  {"concept": "RevenueFromContractWithCustomerExcludingAssessedTax", "value": 570000000, "unit": "USD", "period": "duration:2022-10-01..2022-12-31", "dimensions": [{"axis": "ProductOrServiceAxis", "member": "ext-0001524472#FireAndSecurityMember", "member_label": "Fire And Security"}, {"axis": "StatementBusinessSegmentsAxis", "member": "ext-0001524472#GlobalProductsMember", "member_label": "Global Products"}]},
  {"concept": "RevenueFromContractWithCustomerExcludingAssessedTax", "value": 2080000000, "unit": "USD", "period": "duration:2022-10-01..2022-12-31", "dimensions": [{"axis": "StatementBusinessSegmentsAxis", "member": "ext-0001524472#GlobalProductsMember", "member_label": "Global Products"}]}
]}

All eight table cells are extracted (both years). Values are expanded to full units from the pre-context's "(in millions)" cue; "Three Months Ended December 31" becomes the correct quarter duration; each product row carries two dimensions - the product axis and the Global Products segment axis, the latter inferred from the pre-context - using the company's own extension members (ext-0001524472#…); the Total rows correctly carry only the segment axis. Note the extra member_label field and that quarter start dates are inferred (the passage states only the end date).

Limitations

• Extracts XBRL-aligned facts only. It does not return every number visible in a passage; dates, page numbers, and untagged narrative figures are excluded by design.
• Concept errors remain, especially on long-tail and company-specific concepts (paired concept accuracy is 0.461).
• Dimension attribution is not production-grade (dimensions-F1 0.360); verify segment/axis assignments before use.
• Period boundaries may be inferred when the passage does not state them explicitly, as in the example above.
• Fact-dense passages need a large output budget - the benchmark's densest passage generated 7,588 tokens; at an 8192-token budget no passage truncated, but a 4096 budget can clip the densest tables.
• Not designed for arithmetic, multi-hop reasoning, or question answering (see Additional diagnostics).
• Outputs require human review.

Training data

The adapter was trained with LoRA SFT on ~105K private passage-complete records from the parent FiledFact corpus, using corrected alignment labels - each training example is one filing passage paired with all of its XBRL facts. Prompt tokens were masked from the loss (completion-only); max sequence length 12,288; one full epoch. The evaluation companies were excluded from training.

A public, passage-complete sample of the training format is FiledFact-Passages - a slice of the training distribution. Its passages are included in this model's training data, so do not use it to benchmark this model. For individual grounded facts (one fact per row), see FiledFact-100K; the model was not trained on those rows.

Additional diagnostics

A TAT-QA transfer probe (n=200, simplified EM scoring, measured with an earlier adapter build) confirms the adapter should not be used for general QA: overall exact-match fell from 0.390 (base) to 0.265 (tuned), with arithmetic and multi-span answers degrading most while single-span extraction improved slightly. This is reported to clarify scope; it is not part of the headline benchmark.

License

• These adapter weights: Apache-2.0.
• Base model Qwen/Qwen3-8B: Apache-2.0.
• The FiledFact-100K and FiledFact-Passages public samples: CC BY-NC 4.0.
• For the full FiledFact corpus: [email protected] · stockalloy.com/datasets.

Citation

@misc{stockalloy_filedfact_qwen3_2026,
  title  = {FiledFact-Qwen3-8B-LoRA: A Specialist Model for SEC/XBRL Financial Fact Extraction},
  author = {StockAlloy Research},
  year   = {2026},
  url    = {https://huggingface.co/StockAlloy/filedfact-qwen3-8b}
}

@dataset{stockalloy_filedfact_2026,
  title   = {FiledFact-100K: Span-Grounded, Value-Verified SEC Financial Facts},
  author  = {StockAlloy Research},
  year    = {2026},
  url     = {https://huggingface.co/datasets/StockAlloy/filedfact-100k}
}