text2cypher_lora_v4_balanced

text2cypher_lora_v4_balanced

A Qwen3-4B-Instruct-2507 fine-tune (LoRA, merged 16-bit) that turns a natural-language question + a graph schema description into a Cypher query. Trained on a complexity-balanced 2,252-row subset of v4's dataset — see the training repo for the full pipeline and qa/v4_generation_tracking.md for exactly how this dataset was built.

This model requires a specific prompt format — it will NOT work with a bare question

This is the single most important thing to know before using it. The model was trained to expect the graph schema in the system prompt, not baked into the weights — that's what lets one model handle arbitrary domains/schemas it's never seen, rather than being locked to one. A generic chat message like {"role": "user", "content": "Who are you?"} (the default HF "Use this model" snippet above) will just get you a generic base-Qwen answer — the fine-tuning has nothing to activate on without a schema.

Correct usage:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("BeastxD/text2cypher_lora_v4_balanced")
model = AutoModelForCausalLM.from_pretrained("BeastxD/text2cypher_lora_v4_balanced", device_map="auto")

SYSTEM_PROMPT_TEMPLATE = (
    """You are a Cypher query generation assistant for a Neo4j graph database.

You are given a graph schema and a question in natural language. Use the
schema strictly - it is the only source of truth for what exists in the graph.

How to read the schema:
- 'Node properties' lists each node label together with its properties and
  their types (e.g. STRING, FLOAT, DATE, POINT). Some properties list
  example or available values - these show the kind of data to expect, not
  an exhaustive list to match against literally unless the question refers
  to one of them directly.
- 'The relationships' lists every valid pattern of how node labels connect,
  in the form (:LabelA)-[:REL_TYPE]->(:LabelB). This tells you both the
  relationship type name and its direction - respect the direction when you
  build your MATCH pattern.

How to map the question to the schema:
1. Find the node label(s) the question is really asking about (the subject
  and the target of the question).
2. Find the relationship path in the schema that connects those labels -
  questions often require traversing more than one relationship.
3. Identify any filters mentioned in the question (names, dates, categories,
  thresholds) and match them to the correct property on the correct label.
4. If the question asks for a count, total, average, minimum, maximum, or
  'top N', use the appropriate aggregation function and ORDER BY / LIMIT.

Rules:
- Use only labels, relationship types, and properties that literally appear
  in the schema below. Never invent one.
- Return ONLY the Cypher query - no explanation, no markdown fences, no
  comments.
- Return only the specific properties the question names. Return a whole
  node only when the question asks generally about an entity without naming
  particular attributes.
- When computing a single overall aggregate (an overall average, count, or
  sum), do not carry unrelated variables into the WITH that produces it -
  every non-aggregated variable in a WITH implicitly groups the aggregate by
  that variable, turning one intended overall result into one result per
  group.
- Before returning the query, check every relationship pattern you used against
  the schema's relationship list. Your arrow direction and label order must
  match one of the listed (:LabelA)-[:REL_TYPE]->(:LabelB) patterns exactly -
  if your pattern is the reverse of a listed one, you have the direction
  wrong and must flip it.
- For "highest", "lowest", "top N", "most/least" phrasing, select with
  ORDER BY <property> ASC|DESC LIMIT N rather than computing min()/max() and
  re-matching on equality - re-matching on equality returns every tied row
  instead of one deterministic answer.
- If a MATCH path can reach the same return value multiple times through
  multi-hop or branching traversal, use DISTINCT on it - unless the question
  specifically asks for a count or list per relationship/edge, in which case
  duplicates are the correct answer and DISTINCT must not be used.
- When the question asks about a status, state, count threshold, or yes/no
  condition ("accepted", "active", "at least one", "any", "some", "is X"),
  first check whether the relevant node has a property in the schema that
  directly represents that condition (a BOOLEAN, or a COUNT/INTEGER property
  already tracking it) and filter on it directly. Do not reconstruct the
  condition via a traversal or exists() check if a direct property already
  encodes it.
- If the property the question refers to (e.g. "type", "kind", "category")
  does not exist on the node you first match, do not traverse further away
  from it searching for a substitute property on a different node. Stay on
  the matched node and use its closest literal property (e.g. count distinct
  values of an existing identifying property on that same node) rather than
  inventing a multi-hop path to a loosely related property elsewhere.
- Return ONLY the Cypher query - no explanation, no markdown fences, no
  comments.\n\nSchema:\n{schema}"""
)

schema = """Nodes:
  Common properties:
    · id:STRING — Stable canonical entity identifier
    · name:STRING — Use FTS index (QUERY_FTS_INDEX) for fuzzy name lookups; CONTAINS as fallback
    · first_observed:DATE — Native DATE. Compare with DATE literals: WHERE n.first_observed >= DATE('2024-01-01')
    · last_observed:DATE — Native DATE. Use with first_observed for "active at date" checks
    · status:STRING — ACTIVE / ARCHIVED / UNCERTAIN

  Per-label descriptions and domain properties:
  (:Customer) — a customer who owns appliances and submits work orders
    · phone:STRING — primary contact phone number
    · preferred_contact_method:STRING — [Phone, Email, SMS]
  (:Appliance) — a specific appliance unit owned by a customer
    · appliance_type:STRING — [Refrigerator, Washer, Dryer, Dishwasher, Oven, HVAC]
    · brand:STRING — manufacturer brand name
    · model_number:STRING — manufacturer model number

Relationships:
  (:Customer)-[:OWNS]->(:Appliance) — customer owns the appliance"""

question = "What brand and model number does the appliance owned by customer 'Jane Doe' have?"

messages = [
    {"role": "system", "content": SYSTEM_PROMPT_TEMPLATE.format(schema=schema)},
    {"role": "user", "content": question},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=250, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
# -> MATCH (c:Customer {name: 'Jane Doe'})-[:OWNS]->(a:Appliance) RETURN a.brand, a.model_number

Dataset

• 2,252 rows, built by trimming the full 3,744-row v4 dataset (BeastxD/text2cypher_lora_v4_raw), not adding new rows — easy-bucket rows trimmed from 2,489 down to 997 (domain-proportional random sample, seed 42) to match the complex bucket's count (995), medium (260) left as-is.
• Complexity distribution: 44.3% easy / 11.5% medium / 44.2% complex — a deliberately more even split than the raw version's 66.5% / 6.9% / 26.6%, trading data volume for balance. Trimmed rows are not discarded — recorded in qa/v4/trimmed_easy_rows_dropped_for_balance.csv in the training repo.
• Every row passed deterministic schema-grounding and relationship-direction checks before being included — see common/validate_and_build.py in the training repo.

Eval results

Formal semantic-accuracy eval (via common/semantic_rescore.py, same methodology as v2/v3) has not been run against this checkpoint yet — check the training repo's v4/evals_balanced/ for results once available. This model exists specifically to test whether complexity balance (at the cost of ~40% less training data than the raw version) helps or hurts real accuracy — that comparison isn't settled until both get evaluated.

Training details

• Base: unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit, 4-bit + rank-16 LoRA, targeting all attention + MLP projections.
• Trained on RunPod (RTX 5090), schema-grouped 80/10/10 train/val/heldout split.
• Same recipe as v3 (v3/code/runpod/docuprism_lora_training_runpod.ipynb), just repointed at the balanced v4 dataset — see v4/code/runpod/docuprism_lora_training_runpod_balanced.ipynb.