Lumma-0.6B-Extract

Lumma-0.6B-Extract

Based on Lumma-0.6B-Base, Lumma-0.6B-Extract is a lightweight, single-turn specialized model designed to accurately extract structured information from text in one step, enabling efficient and reliable information extraction.

Benchmark results

<img src="benchmark_results.jpg" width="1000"/>

We benchmarked Lumma-0.6B-Extract on FrontiersMind's internal structured benchmark, measuring model's performances on ~500 documents of diverse types including invoices,travel itenaries, OCR Extracted text & Emails etc. We plan to open-source this benchmark in the coming weeks, along with a extensive leaderboard including most popular open-weight and closed-sourced APIs and a Python library allowing to easily measure model performances on structured extraction.

Use cases

• Information extraction: Extract structured information such as names, dates, organizations, locations, products, attributes, and other fields from unstructured text.
• Document processing: Convert unstructured text from documents, emails, reports, and messages into structured JSON.
• Data processing pipelines: Transform unstructured text into structured data for downstream applications, databases, search systems, and analytics.
• Real-time AI applications: Power applications that require fast and reliable information extraction with low latency.
• Resource-constrained environments: Support efficient extraction on mobile devices, edge devices, embedded systems, and hardware with limited compute and memory resources.

[!NOTE] 🚧 Coming Soon: We will soon be bringing multilingual Indic language support to Lumma-0.6B-Extract.

Structured Extraction

Lumma-0.6B-Extract uses a JSON template to define the information that should be extracted from the input text.

The extraction process consists of two main steps:

1. Extraction template: Provide a JSON template describing the fields and information to extract.
2. Information extraction: Provide the input text, and the model returns the requested information in the specified JSON structure.

This approach allows users to define custom extraction schemas depending on their application.

Example

Template:

{
    "name": "string",
    "company": "string",
    "job_title": "string"
}

Input text:

John Smith joined Acme Corporation as a Senior Software Engineer.

Expected output:

{
    "name": "John Smith",
    "company": "Acme Corporation",
    "job_title": "Senior Software Engineer"
}

Usage

import torch
import json
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "FrontiersMind/Lumma-0.6B-Extract"

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_PATH,
    trust_remote_code=True
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_PATH,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

model.eval()


input_text = "John Smith joined Acme Corporation as a Senior Software Engineer."

template = {
    "name": "string",
    "company": "string",
    "job_title": "string",
}

prompt = tokenizer.apply_chat_template(
input_text=input_text,
template=template,
tokenize=False,
add_generation_prompt=True
)


inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)

with torch.inference_mode():
    out = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id,
    )

raw = tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False)
pred = raw.split("<|endoftext|>")[0].strip()
print(json.dumps(json.loads(pred), indent=4))

License

This model is released under the Apache License 2.0.

Feedback & Suggestions

We’d love to hear your thoughts, feedback, and ideas!

• Discord: https://discord.gg/ZGdjCdRt
• Email: [email protected]
• Official Website: https://www.frontiersmind.ai/
• LinkedIn: https://www.linkedin.com/company/frontiersmind/
• X (Twitter): https://x.com/FrontiersMind