Downloads · 30 days
13
32% of all-time downloads
julioc-p/occiglot-7b-eu5-instruct-txt-sparql-en-Instruct-txt-sparql_4bit
occiglot-7b-eu5-instruct-txt-sparql-en-Instruct-txt-sparql_4bit is a machine learning model from julioc-p. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
This model is a fine-tuned version of occiglot/occiglot-7b-eu5-instruct for generating SPARQL queries from English natural language questions, specifically targeting the Wikidata knowledge graph.
Downloads · 30 days
13
32% of all-time downloads
All-time downloads
41
Public
Repo size
2.4 GB
Likes
0
Public
Click a slice to open those files.
.safetensors1.2 GB · 100%
From the Hugging Face model README
This model is a fine-tuned version of occiglot/occiglot-7b-eu5-instruct for generating SPARQL queries from English natural language questions, specifically targeting the Wikidata knowledge graph.
It was fine-tuned using QLoRA. It takes an English natural language question as input and aims to produce a corresponding SPARQL query that can be executed against the Wikidata knowledge graph. It is part of a series of experiments to investigate the impact of continual multilingual pre-training on cross-lingual transferability and task-specific performance. Uses 4-bit quantization.
occiglot/occiglot-7b-eu5-instructThe following Python script provides an example of how to load the model and tokenizer using the Hugging Face Transformers and PEFT libraries to generate a SPARQL query.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import re
model_id = "julioc-p/occiglot-7b-eu5-instruct-txt-sparql-4bit" #
base_model_for_tokenizer = "occiglot/occiglot-7b-eu5-instruct" #
# Configuration for 4-bit quantization
bnb_config = BitsAndBytesConfig(
load_in_4bit=True, #
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=False,
)
# Load the model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=bnb_config,
device_map="auto" # Automatically uses GPU if available
)
tokenizer = AutoTokenizer.from_pretrained(base_model_for_tokenizer)
tokenizer.pad_token = tokenizer.eos_token
def extract_sparql(text): #
match = re.search(
r"(SELECT|ASK|CONSTRUCT|DESCRIBE).*?\}", text, re.DOTALL | re.IGNORECASE #
)
if match:
return match.group(0).strip() #
return ""
# --- Example usage ---
question = "What is the boiling point of water?"
knowledge_graph_target = "Wikidata"
system_prompt_content = "You are an expert in SPARQL query generation. Generate the SPARQL query that answers the user's question."
chat_template = [
{"role": "system", "content": system_prompt_content},
{"role": "user", "content": question},
]
inputs = tokenizer.apply_chat_template(
chat_template,
tokenize=True,
add_generation_prompt=True, # This adds the assistant's turn start
return_tensors="pt"
).to(model.device)
# Generate the output
with torch.no_grad():
outputs = model.generate(inputs, max_new_tokens=256, pad_token_id=tokenizer.eos_token_id)
# Decode and extract the SPARQL query
# Decode only the generated part (assistant's response)
generated_text_assistant_part = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
cleaned_sparql = extract_sparql(generated_text_assistant_part)
print(f"Question: {question}")
print(f"Generated SPARQL: {cleaned_sparql}")
print(f"Raw generated text (assistant part): {generated_text_assistant_part}")
The model was fine-tuned on a subset of the julioc-p/Question-Sparql dataset. Specifically, a 35,000-sample English subset filtered to include only Wikidata-related queries was used.
The following hyperparameters were used for the v1.1 Occiglot English fine-tuning:
occiglot/occiglot-7b-eu5-instructr (LoRA rank): 64lora_alpha: 16lora_dropout: 0.1bias: "none"task_type: "CAUSAL_LM"target_modules: "q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj", "lm_head"num_train_epochs: 5per_device_train_batch_size: 1gradient_accumulation_steps: 8 (Effective batch size of 8)gradient_checkpointing: Trueoptim: "paged_adamw_32bit"learning_rate: 1e-5weight_decay: 0.05bf16: False (NVIDIA V100 does not support bfloat16)fp16: Truemax_grad_norm: 1.0warmup_ratio: 0.01lr_scheduler_type: "cosine"group_by_length: Truepacking: False (for v1 models)load_in_4bit: Truebnb_4bit_quant_type: "nf4"bnb_4bit_compute_dtype: torch.float16bnb_4bit_use_double_quant: Falsejulioc-p/Question-Sparql dataset.
-7b-eu5-instruct fine-tuned with QLoRA (v1.1).The primary evaluation metrics used were the QALD standard macro-averaged F1-score, Precision, and Recall. Non-executable queries resulted in P, R, F1 = 0. The percentage of Executable Queries was also tracked. Correctness was further broken down into "Correct (Exact Match)" and "Correct (Both Empty)".
On QALD-10 (English, N=391):
v1 Test Set (English, N=3500):
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
transformers, peft (0.13.2), bitsandbytes, trl, PyTorch.4.39.3)0.43.0)0.8.6)torch==2.1.0)