messy2json-qwen-0.5b

Messy2JSON - Qwen2.5-0.5B

A 0.5B parameter model fine-tuned with QLoRA to convert messy text into structured JSON.

Training

• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Method: QLoRA (4-bit NF4 + LoRA r=16, alpha=32)
• Dataset: stindardlogic/data-extraction-sft-100k (39k JSON-filtered samples, 9k used for training)
• Epochs: 2
• Final validation loss: 0.0045
• Token accuracy: 99.82%

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "aradmanamnaoon/messy2json-qwen-0.5b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)

messages = [
    {"role": "system", "content": "You extract structured data from messy text and output ONLY valid JSON."},
    {"role": "user", "content": "Best regards, Jane Doe, VP Engineering at Acme, [email protected]"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limitations

• Field names are not fixed. The model improvises JSON keys based on input. Two similar inputs may produce different schemas.
• Occasional nesting quirks. Fields may be nested under semantically close parents.
• Domain sensitivity. Best on invoices, email signatures, compliance excerpts, meeting notes, and log lines.
• No schema enforcement. The model infers structure from the text.
• 0.5B parameter model. Limited reasoning.
• English only.

Recommended Use

• Rapid prototyping of text-to-JSON pipelines
• Pre-processing messy documents before downstream parsing
• Edge deployment where a 1 GB model is preferred over an API call

Not Recommended For

• Production systems requiring guaranteed schema compliance
• Non-English text
• Inputs requiring multi-document reasoning
• Safety-critical extraction without human review

Training Details

Dataset

Fine-tuned on stindardlogic/data-extraction-sft-100k, filtered to JSON-only outputs.

Ethical Considerations

• Privacy: Do not feed PII, credentials, or confidential data into a public Space demo.
• Hallucination: The model may invent fields or values not present in the input.
• No provenance tracking. Output JSON does not cite which part of the input each field came from.

Citation

@misc{messy2json-qwen-0.5b,
  title  = {Messy2JSON: Qwen2.5-0.5B fine-tuned for structured extraction},
  author = {aradmanamnaoon},
  year   = {2025},
  url    = {https://huggingface.co/aradmanamnaoon/messy2json-qwen-0.5b}
}

Contact

Open a discussion on the model page.

Acknowledgments

• Base model by Qwen team
• Training framework: Hugging Face TRL, PEFT, Transformers
• Dataset: stindardlogic/data-extraction-sft-100k
• Compute: Google Colab free T4 GPU

License

Apache 2.0 - same as the base model. See LICENSE for details.

You are free to use, modify, and redistribute this model, including for commercial purposes, provided you retain the license notice.

Model Card Version

• v1.0 - initial release