endpoint-qwen3.5-4b-lora

Clinical Trial Endpoint Classifier — 4B (Qwen3.5-4B LoRA) — Best

The best performing endpoint classification model. A fine-tuned LoRA adapter on Qwen3.5-4B for extracting and classifying clinical trial endpoints from outcome text. Returns structured JSON with standardized endpoint names, measurement types, methods, and more.

Output Format

{
  "endpoints": [
    {
      "endpoint_name_standardized": "Objective Response Rate",
      "measurement_of": "tumor response",
      "measurement_type": "binary",
      "metric_type": "proportion",
      "timeframe": "Week 24",
      "measurement_method": "RECIST v1.1",
      "evaluation_criteria": "CR or PR",
      "unit": "%",
      "population": null,
      "is_composite": false,
      "components": []
    }
  ]
}

Field Definitions

Supports multiple endpoints from a single text (e.g., safety texts with 10+ sub-endpoints).

Training Details

Hyperparameters

Method: LoRA (bf16, NOT 4-bit)
LoRA rank: 16, alpha: 16
Learning rate: 2e-4 (cosine)
Batch size: 2 (gradient accumulation 8, effective 16)
Epochs: 3
Optimizer: adamw_8bit
Sequence length: 2048
Gradient checkpointing: unsloth

Usage

import json
from unsloth import FastLanguageModel
from transformers import AutoTokenizer
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Shubh-0789/endpoint-qwen3.5-4b-lora",
    max_seq_length=2048,
    load_in_4bit=False,
    load_in_16bit=True,
    dtype=torch.bfloat16,
)
text_tokenizer = AutoTokenizer.from_pretrained("Shubh-0789/endpoint-qwen3.5-4b-lora")
FastLanguageModel.for_inference(model)
model.generation_config.pad_token_id = text_tokenizer.pad_token_id

clinical_text = "Primary endpoints are ORR and progression-free survival (PFS) assessed by RECIST v1.1 | [Time Frame: Up to 24 months]"

messages = [
    {"role": "user", "content": f"Extract and classify the clinical trial endpoint from the following text. Return ONLY a JSON.\nText: {clinical_text}"}
]

inputs = text_tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_tensors="pt", return_dict=True,
).to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.1, do_sample=True)

result = text_tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
endpoints = json.loads(result)
print(json.dumps(endpoints, indent=2))

Output:

{
  "endpoints": [
    {
      "endpoint_name_standardized": "Objective Response Rate",
      "measurement_of": "tumor response",
      "measurement_type": "binary",
      "metric_type": "proportion",
      "timeframe": "Up to 24 months",
      "measurement_method": "RECIST v1.1",
      "evaluation_criteria": null,
      "unit": "%",
      "population": null,
      "is_composite": false,
      "components": []
    },
    {
      "endpoint_name_standardized": "Progression-Free Survival",
      "measurement_of": "disease progression or death",
      "measurement_type": "time-to-event",
      "metric_type": "hazard ratio",
      "timeframe": "Up to 24 months",
      "measurement_method": "RECIST v1.1",
      "evaluation_criteria": null,
      "unit": null,
      "population": null,
      "is_composite": false,
      "components": []
    }
  ]
}

With PEFT/Transformers

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base_model, "Shubh-0789/endpoint-qwen3.5-4b-lora")
tokenizer = AutoTokenizer.from_pretrained("Shubh-0789/endpoint-qwen3.5-4b-lora")

Model Comparison

Limitations

• Trained on English clinical trial text only
• Complex composite endpoints may need verification
• Minimum inference: any GPU with 10GB+ VRAM

Citation

@misc{endpoint-qwen3.5-4b-lora,
  author = {Shubh-0789},
  title = {Clinical Trial Endpoint Classifier — 4B (Qwen3.5-4B LoRA)},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Shubh-0789/endpoint-qwen3.5-4b-lora}
}