Qwen2.5-7B-Prompt-Perfector-Full

Qwen2.5-7B-Prompt-Perfector-Full

This repository contains the fully merged, standalone weights for Qwen2.5-7B-Prompt-Perfector. The model is a specialized, domain-specific instruction fine-tune engineered to transform short, casual user concepts ("lazy prompts") into highly detailed, expressive, and production-ready text-to-image prompt sequences optimized for Stable Diffusion (v1.5/XL/Cascade), Midjourney, and Dall-E 3.

🚀 Core Technical Achievements

• End-to-End Pipeline: Developed a complete synthetic data generation pipeline to extract, parse, and validate high-density visual descriptive tags.
• Quantized Low-Rank Adaptation (QLoRA): Fine-tuned the attention projections and MLP layers using 4-bit NormalFloat (NF4) double quantization to maximize parameters-per-VRAM efficiency.
• Inference Optimization: Successfully diagnosed and resolved Catastrophic Repetition Degeneration loops at runtime by implementing tailored repetition penalties ($1.25$) and hard sequence-stopping tokens (<|im_end|>).
• Architectural Merge: Combined the lightweight trained adapter layers directly back into the 7.6B base model tensor configurations to output a zero-dependency, plug-and-play inference artifact.

---

📊 Dataset & Training Specifications

• Training Dataset: Ilyankhan69/Prompt-Perfector-500 (Custom curated pairs mapping brief concepts to rich rendering tokens).
• Base Architecture: Qwen/Qwen2.5-7B-Instruct
• Fine-Tuning Framework: LLaMA-Factory
• Hardware Precision: Trained on dual-GPU infrastructure using bfloat16 compute with structural SDPA (Scaled Dot-Product Attention) acceleration flags.
• Target Modules: Complete linear parameter targeting (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).

---

💻 Programmatic Usage (Python / Transformers)

Because this repository contains the fully compiled weights, it requires no adapter stitching or external configuration files. It can be loaded directly using standard transformers code:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Ilyankhan69/Qwen2.5-7B-Prompt-Perfector-Full"

# 1. Load Tokenizer and Standalone Weights
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# 2. Format using the ChatML template structural format
lazy_input = "Futuristic car factory"
formatted_prompt = (
    "<|im_start|>system\n"
    "You are an expert prompt engineer. Turn casual language into highly detailed, comma-separated art generation prompts.<|im_end|>\n"
    f"<|im_start|>user\n{lazy_input}<|im_end|>\n"
    "<|im_start|>assistant\n"
)

# 3. Compute Inference
inputs = tokenizer(formatted_prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=120,
        temperature=0.7,
        repetition_penalty=1.25,
        stop_sequences=["<|im_end|>", "<|endoftext|>"]
    )

# 4. Extract Perfected Prompt String
generated_tokens = outputs[0][inputs.input_ids.shape[1]:]
perfected_output = tokenizer.decode(generated_tokens, skip_special_tokens=True)
print(perfected_output.strip())```


## 🎯 Prompt Transformation Example

* **Input Concept:** `Futuristic car factory`
* **Model Perfection Output:** `a concept art of the inside of a futuristic car manufacturing plant, trending on artstation and cgsociety`