Downloads · 30 days
11
26% of all-time downloads
cakradana-app/cakradana-v1-docie-deepseek-r1-distill-llama-8b-adapter
cakradana-v1-docie-deepseek-r1-distill-llama-8b-adapter is a text generation model from cakradana-app. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as llama3.1.
Downloads · 30 days
11
26% of all-time downloads
All-time downloads
42
Public
Repo size
353 MB
Likes
0
Public
Click a slice to open those files.
.safetensors336 MB · 95%
From the Hugging Face model README
A LoRA adapter that turns Indonesian campaign-finance filing text into structured donation records
Built with Llama
</div>Indonesian candidates and parties file campaign-finance reports — LADK, LPSDK, LPPDK — with the KPU (Komisi Pemilihan Umum). Turning those filings into auditable data means reading a donor table and emitting one record per donation, without inventing any.
This is a LoRA adapter for
unsloth/DeepSeek-R1-Distill-Llama-8B
(83.9 M trainable parameters) that does that. It is inert on its own — it needs
the base model.
It pairs with
cakradana-v1-ocr-deepseek-3b-adapter,
which produces the text this model reads. Both were built for
Cakradana, an AI system for transparency in
Indonesian election financing, which placed 3rd in the Open Sector at the
TrackAML 2.0 hackathon run by PPATK, Indonesia's Financial Transaction
Reports and Analysis Center.
Compare only at equal token budget. The base model spends its output budget
reasoning: at 1,024 tokens, 44 of 100 baseline outputs never closed </think>
and so never reached any JSON. Scoring those as failures credits this adapter
for R1-Distill's verbosity rather than for extraction.
The middle column re-runs 30 of the identical chunks at 3,072 tokens (the eval seeds its selection, so those rows are a strict subset). All three columns below are those same 30 chunks.
| metric | base @1024 | base @3072 | tuned @1024 |
|---|---|---|---|
| parse rate | 0.333 | 0.733 | 1.000 |
| fabrication rate | 0.444 | 0.643 | 0.000 |
| span validity | 0.867 | 0.935 | 0.918 |
| F1 (strict) | 0.050 | 0.198 | 0.418 |
| recall | 0.025 | 0.136 | 0.322 |
| cut off mid-reasoning | 13 | 6 | 0 |
89% of real filing chunks carry no donation at all — covers, decision letters, expenditure tables. A model that invents records on those is worse than useless no matter how it scores elsewhere.
The base model fabricates on roughly two thirds of the empty chunks it manages to answer, and giving it three times the budget does not help (0.444 → 0.643; more budget means more outputs parse, which reveals more fabrication). The tuned model fabricates on none — 0 of 60 hard-null chunks in the full run, while still emitting 218 records on the chunks that do carry them. It learned to discriminate, not to refuse.
At equal budget the base model scores 0.935 against this adapter's 0.918. The apparent 0.792 → 0.931 gain in the naive 1,024-token comparison is a budget artefact, and it is not quoted here as a win.
Both columns at 1,024 tokens, so the parse-rate and recall gaps are inflated by the budget effect above. Reported for completeness, not as the headline.
| metric | base | tuned |
|---|---|---|
| parse rate | 0.370 | 0.990 |
| fabrication rate | 0.636 | 0.000 |
| F1 (strict) | 0.030 | 0.578 |
| F1 (amount-only) | 0.039 | 0.745 |
The matcher is not doing the work: folding titles, legal forms and punctuation moves strict F1 by 0.007. The gap to amount-only matching is real signal — where this model errs, it more often has the donation right and the donor name wrong than the reverse.
Reproduce either run:
python scripts/eval_extraction.py # tuned
python scripts/eval_extraction.py --adapter '' # baseline
The published figures were measured against the private pre-redaction corpus. The script reads the public release — a redacted subset, 6,174 pairs against 8,016 — so it selects different rows and will not reproduce them to the digit. It reproduces the comparison.
The prompt is not a suggestion. This adapter was trained under
vocabulary.build_system_prompt() / build_user_prompt(), shipped in this
repo. Substituting your own instructions serves the model something it never
saw during training.
pip install "transformers==4.57.3" peft unsloth
pip uninstall -y torchao # peft calls is_torchao_available(), which raises
# rather than returning False on some builds
python scripts/extract.py filing.txt
from scripts.extract import load, extract, check_spans
model, tokenizer = load() # base + this adapter
records, raw = extract(model, tokenizer, document_text)
bad = check_spans(records, document_text) # every *_span must be quotable
add_generation_prompt=True ends the prompt at <|Assistant|><think>\n, and
the model continues </think>\n[...]. Training used exactly that state, so the
standard chat-template call is the correct one — no prompt surgery, and no
reasoning trace to pay for. The <think> block is deliberately empty: this task
teaches an output format, and the span rule is what guards correctness.
A JSON array of objects, each carrying only the fields the document supports:
[{"sender": "Hj. Nessy Ariyani", "sender_type": "individual",
"amount": 750000, "date": "2015-10-15",
"sender_span": "Hj. Nessy Ariyani", "amount_span": "750.000",
"date_span": "15 OKTOBER 2015"}]
Every field carries a *_span, and a span is the raw quoted substring — not
an offset. The value is normalised (750000); the span is what is printed on
the page ("750.000"). This is the contract that makes an extraction checkable
rather than merely plausible: check_spans() in scripts/extract.py verifies
each one against the source, and a span that is not in the document is a
fabricated quote however right the value looks.
An empty array is a valid and common answer. It is the correct answer for most filing pages.
| base | unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit, 4-bit |
| method | LoRA, r=32, alpha=32, dropout 0, bias none |
| trainable | 83,886,080 parameters, 448 tensors |
| scope | all 7 projections × all 32 layers |
| schedule | 2 epochs, 338 steps, batch 2 × grad-accum 4 |
| optimiser | adamw_8bit, lr 1e-4 linear, warmup 0.03, weight decay 0.01 |
| precision | bf16, max_seq_length 4096 |
| seed | 20260819 |
| hardware | 1× A100-SXM4-40GB |
| runtime | 1,114.6 s (18 m 35 s) |
| final train loss | 0.0679 |
Data. 8,016 pairs → 7,984 after deduplicating chunk placements → 343 record-bearing and 6,179 hard-null in train. Hard nulls are downsampled to 3:1 against record-bearing chunks (the corpus ships at 17:1), giving 1,372 rows, then 1,349 after dropping 23 over the sequence limit.
That 3:1 ratio is what produced the fabrication result. It was not swept; a different ratio would likely trade fabrication against recall.
Loss falls on the answer only, via train_on_responses_only with the
<|User|> / <|Assistant|> markers. Note those are U+FF5C FULLWIDTH VERTICAL
LINE, not ASCII | — an ASCII pipe matches nothing and silently supervises
zero tokens. scripts/train_extraction.py decodes the supervised span and
aborts unless </think> appears in it, rather than training for hours on
nothing.
The published dataset is a redacted subset of what this run saw. Training
used the private pre-redaction corpus; the public release
(cakradana-kpu-filings-14k-pages)
withholds pages carrying personal identifiers, leaving 6,174 pairs against the
8,016 used here. It is the corresponding citable dataset.
max_seq_length=4096 filter dropped 13 record-bearing training rows whose
median was 60 records (max 172), leaving a training set that tops out at
45. Of 40 record-bearing eval outputs only 1 was truncated by the output
cap; 23 emitted fewer records than expected and then closed the array —
142 records never emitted. Raising --max-seq to 8192 is the obvious v2, and
the F1 above is its baseline.max_new_tokens for dense pages. One eval output was truncated
mid-record at 1,024. scripts/extract.py defaults to 2,048.eval_clean covers 4 publishing hosts, 166 documents. Nothing here
supports a claim about an unseen publisher.Built with Llama. Governed by the Llama 3.1 Community
License, inherited through
DeepSeek-R1-Distill-Llama-8B from Llama-3.1-8B-Base. That covers the weights;
it does not release you from the training corpus's terms, which prohibit
attempting to re-identify individuals and use for harassment or profiling. See
LICENSE.
Made with ❤️ by the Cakradana Team · cakradana.faizath.com
</div>