lemoe-query-distiller

Model Details

Overview

This model is a fine-tuned version of microsoft/mdeberta-v3-base adapted for the Query Distillation task in Spanish. It operates via Token Classification, analyzing natural language queries and labeling each word to decide whether it should be kept as a key search term or discarded as syntactic noise.

It is specifically designed as an ultra-lightweight and fast filter to be integrated into RAG pipelines and document search systems (like Paperless-ngx) orchestrated through LEMoE.

• Developed by: lemoe.link
• Model type: Encoder-based Transformer (Token Classification)
• Language(s): Spanish (es)
• License: AGPL
• Base model: microsoft/mdeberta-v3-base

Architecture and Operation

Since it is an Encoder model, it does not generate new text (it is not autoregressive). Instead, it assigns a binary weight to each input token:

• LABEL_1 (Keep): Entities, keywords, subjects, and services.
• LABEL_0 (Drop): Prepositions, generic action verbs, polite phrases, and connectors.

Intended Uses and Limitations

Direct Use

• Keyword Extraction: Converting long queries ("búscame la factura de Iberdrola por favor") into optimized search strings ("factura Iberdrola").
• Conversational Interfaces: Acting as a silent middleware layer between natural language user requests and strict lexical search engines.

Recommended Execution Environments

• CPU Inference: Being a base-sized Encoder model, it is optimized to run with millisecond latencies (typically < 100ms) on standard x86 processors, ideal for HomeLab clusters or low-power nodes.
• Microservices: Perfect for being served via local APIs in Docker containers, without requiring GPU acceleration for personal or small-network traffic volumes.

Limitations

• The model does not rewrite words (it does not correct spelling mistakes by generating new tokens, although its embeddings tolerate certain typos when evaluating the token's root).
• It is optimized exclusively for the Spanish language.

How to Get Started with the Model

Below is the Python code to instantiate the model and use it as a filter.

from transformers import pipeline

# Load the token classification pipeline
# Use aggregation_strategy="simple" to merge subwords
distiller = pipeline(
    "token-classification", 
    model="your-username/mdeberta-v3-base-lemoe-query-distiller",
    aggregation_strategy="simple"
)

def clean_search_query(user_input: str) -> str:
    # Get model predictions
    predictions = distiller(user_input)
    
    # Filter and concatenate tokens labeled as LABEL_1
    search_terms = [
        entity['word'] for entity in predictions 
        if entity['entity_group'] == 'LABEL_1'
    ]
    
    # Return the cleaned query
    return " ".join(search_terms).strip()

# Usage example
query = "busca la factura de iberdrola de la luz de este mes"
result = clean_search_query(query)

print(f"Original query: {query}")
print(f"Distilled search: {result}")