meta-sdk-baseline

Meta SDK Baseline — LoRA Adapter

LoRA adapter for Qwen/Qwen3.5-4B fine-tuned on 1,209 chunk-based training samples generated by Meta's Synthetic Data Kit from the same financial and medical source documents.

This is the Meta SDK baseline model from the paper "Knowledge Graph-Guided Fine-Tuning Data Generation: A Rigorous Benchmark" — the industry-standard chunk-based approach used as the comparison point against KnowledgeMesh.

Benchmark Results

Evaluated by Gemini 2.5 Flash pointwise judge (1–5 scale, 4 dimensions):

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen3.5-4B"
adapter_id = "likhithv/meta-sdk-baseline"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)

messages = [{"role": "user", "content": "What were Apple's total net sales in 2023?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs.to(model.device), max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Training Details

Eval Datasets

• likhithv/knowledgemesh-benchmark-eval

Compared Models

• likhithv/km-full-model — KnowledgeMesh, 4,361 KG-guided samples (+0.72 on independent eval)

Citation

@misc{knowledgemesh2026,
  title={Knowledge Graph-Guided Fine-Tuning Data Generation: A Rigorous Benchmark},
  author={Likhith V},
  year={2026},
  howpublished={https://huggingface.co/likhithv/meta-sdk-baseline}
}