claim_drafter

Claim Drafter

Fine-tuning an open model to draft US patent claims from a plain-English invention disclosure, trained on Tinker.

The interesting parts are not the training loop — they are in the data:

• Targets are as-GRANTED claims, fetched per patent number, not the as-filed claims that ship with HUPD. Across 1,596 measured patents, 62% of as-filed claim 1s were substantially amended during prosecution, so training on them teaches the model to draft claims that draw rejections.
• Prompts don't leak the answer. Patent summaries restate the claims — a median 28% of claim 1's 8-word phrases appear verbatim. Dropping the summary cut measured leakage from 46% to 4%, so the model must draft rather than reformat.
• Preference pairs come from real examiners. For 5,614 applications we hold both the refused (as-filed) and allowed (as-granted) claims. That is a labelled preference with no LLM judge and no manual annotation.
• The RL reward is a program, not a reward model. Patent claims have formal properties — numbering, dependency validity, single-sentence form — that can be checked exactly. Calibrated at 0.991 on real granted claims.

---

Quick start

git clone <your-repo-url> && cd claim-drafter
make setup                      # .venv on Python 3.12 + dependencies
cp .env.example .env            # then add your TINKER_API_KEY
make test                       # offline tests, no key or data needed
make preflight                  # verifies everything incl. the API
make all                        # SFT -> DPO -> GRPO -> graphs

make all hands each stage's checkpoint to the next automatically, and prints a live progress line per stage:

sft  [############................]  43.2%  1037/2400  nll 0.8421  2.9s/it  50m12s<1h06m  eval in 8m21s

Stages can still be run one at a time (make sft, make dpo, make rl), and make graphs redraws runs/graphs/ from whatever logs exist. See the GitHub repository for the eval-cadence reasoning.

make help lists every target. The GitHub repository walks through every step in order and what to watch while it runs.

Requirements

• Python 3.11+ — the tinker SDK requires it.
• A Tinker API key from https://tinker-console.thinkingmachines.ai.
• ~5 GB disk for dependencies, ~2 GB more if you rebuild the datasets.

macOS note. If pip fails with Symbol not found: _XML_SetAllocTrackerActivationThreshold, your Homebrew Python has a broken pyexpat link and cannot install anything — venv fails too, because ensurepip hits the same error. make setup sidesteps it by using uv, which fetches its own standalone Python. To fix the underlying install instead: brew reinstall expat [email protected].

---

Repository layout

claim_drafter/          importable library
  config.py             shared defaults, .env loading
  domains.py            IPC code -> one of six industry domains (99.96% coverage)
  filters.py            text cleaning, leakage, unlearnable-claim detection
  patents.py            Google Patents fetch/parse (granted B2 and as-filed A1)
  rewards.py            programmatic claim reward + production guardrail
  rl_env.py             RL environment (ProblemEnv subclass)
  evaluator.py          custom evaluator over the 11 validation slices

scripts/                data pipeline and checks
  select_targets.py     choose which patents to fetch, before making the calls
  fetch_claims.py       concurrent, resumable claim fetching
  augment_thin_domains.py  stream an extra HUPD year for under-represented domains
  build_sft_dataset.py  build the SFT dataset
  build_dpo_pairs.py    build examiner-labelled preference pairs
  validate_dataset.py   structural pre-flight on a built dataset
  estimate_cost.py      size the pipeline before running it
  preflight.py          verify the whole setup end to end

training/               train_sft.py -> train_dpo.py -> train_rl.py
deploy/                 export_model.py, example_client.py
tests/                  offline test suite (runs on a fresh clone)
manifests/              provenance for every example -- makes the data reproducible
samples/                a few rows of each dataset, so the format is visible
docs/                   runbook, experiment plan, dataset card (in the GitHub repo)
data/                   generated, gitignored

Why the data isn't committed

The built datasets are ~600 MB and several files exceed GitHub's 100 MB limit. Instead, manifests/ records the patent number, IPC class, domain, split and token counts for every example. Combined with the scripts and a fixed seed, that reproduces the datasets byte-identically:

make data        # SFT dataset (~1 hour, mostly network)
make dpo-data    # preference pairs

samples/ holds a handful of rows from each so you can see the format without rebuilding anything. The tests run against those, so a fresh clone is testable immediately.

---

The training pipeline

Three stages, each starting from the previous checkpoint. Full reasoning and hyperparameters are in the GitHub repository.

make sft         # stage 1
make graphs      # then LOOK at the results before starting stage 2
make dpo         # checkpoint resolves from runs/sft automatically
make rl          # checkpoint resolves from runs/dpo automatically

Stop after any stage and you still have a working model. Stopping is a real decision point, not ceremony — if SFT already scores ~0.99 on formal validity, stage 3 has little left to win.

On the vocabulary, since it trips everyone up: LoRA and PEFT are not alternatives to SFT, DPO or RLHF — they are how the weights get updated during any of them. DPO is a way of doing RLHF without a reward model. GRPO is an RL algorithm that works with any reward, including a program.

---

Deployment

The weights come out of Tinker, so inference runs entirely on your own hardware and no customer disclosure ever leaves it.

python3 deploy/export_model.py --checkpoint tinker://<run-id>/sampler_weights/final
vllm serve Qwen/Qwen3.5-9B --lora-modules claim-drafter=./export/peft_adapter
python3 deploy/example_client.py

Validate every generation before showing it to a user — the same function that provided the RL reward is the production guardrail:

from claim_drafter.rewards import claim_reward
if claim_reward(generated) < 0.9:
    ...  # numbering or dependency defect: regenerate or route to review

---

Honest limitations

• Not legal advice. The model drafts claim form. It cannot assess novelty, non-obviousness or patentability, and claim_reward checks formal validity only. Every output needs attorney review.
• Chemistry is weakest. 59% of biotech and 22% of pharma claim sets were dropped because they reference drawn structures ("a compound of formula (I)") or sequence listings that exist only as images. The model trains on the method and device claims that survive.
• No design or plant patents. HUPD contains utility applications only.
• Evaluation is in-distribution. Every prompt is patent-office prose; real users write rough disclosures. Hand-write a dozen and read the outputs before trusting it.
• Corpus is 2014 + Jan-2016 filings, so it reflects drafting practice from roughly a decade ago.

Data provenance and licensing

Patent text is US government work and not subject to copyright. Applications come from the Harvard USPTO Patent Dataset; granted and as-filed claims are fetched from Google Patents by the scripts here. No dataset text is redistributed in this repository beyond the small samples. The code is MIT licensed.

If you re-run the fetchers, keep --workers modest — it is a polite crawler with backoff, and the service is not yours.

---

Using this model from the Hugging Face Hub

This repository hosts the LoRA adapter (r=32, α=32) for Qwen/Qwen3.5-9B, alongside the full training/data pipeline. The datasets it was trained on are published separately as a dataset repo: vishwr/claim_drafter.

The adapter is provided at the repository root (adapter_config.json, adapter_model.safetensors) for one-line loading, and is also kept at export/peft_adapter/ so the paths in the docs and the vllm command below resolve unchanged.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3.5-9B"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, "vishwr/claim_drafter")  # adapter at repo root

Serve with vLLM:

vllm serve Qwen/Qwen3.5-9B --enable-lora \
  --lora-modules claim-drafter=vishwr/claim_drafter

Validate every generation with the same program that provided the RL reward (claim_drafter/rewards.py) before showing it to a user — it checks numbering, dependency validity and single-sentence form.