Qwen2.5-3B-Solidity-V2

Qwen 2.5 3B Solidity Architecture Transformer

A specialized, fine-tuned large language model pipeline designed to securely translate structured legal contract clauses into modern, secure smart contracts conforming strictly to the Solidity 0.8.20 standard.

---

Project Pipeline & Team Roles

We structured our development team like a real-world production machine learning pipeline, dividing core engineering responsibilities across 4 roles:

• Abhishek (Researcher): Gathered standard legal clauses, audited smart contract benchmarks, and mapped out technical translation requirements.
• Ayush (Data Engineer): Handled parsing, data tokenization, and cleaning the training corpora into our optimized 2-key JSON schema.
• Meghansh (Fine-Tuning Engineer): Configured the LoRA parameters, managed the Unsloth training loops, resolved environment serialization errors, and quantized the final weights.
• Himanshu (Security & QA): Formulated adversarial test scripts and verified model outputs against version-specific compilation checks and logical vulnerabilities (e.g., reentrancy, access control).

---

Technical Stack & Architecture

1. Base Model & Training Framework

• Base Model: Qwen/Qwen2.5-3B-Instruct (Chosen for its lightweight footprint and strong base capability in structural formatting and reasoning).
• Fine-Tuning Framework: Unsloth (Utilized to accelerate training speed and heavily reduce VRAM consumption during computation via optimized gradient checkpointing).
• PEFT Technique: LoRA (Low-Rank Adaptation) with a rank dimension ($r = 16$) and targeting critical linear projections (q_proj, v_proj, k_proj, etc.) to isolate the translation layers.

2. Dataset Strategy & Data Engineering

The dataset uses a specialized 2-key structural schema to map conversational constraints cleanly into localized Qwen token blocks.

• Structure: {"instruction": "...", "output": "..."}
• Alignment Mapping:

  <|im_start|>system
  [Custom System Alignment Rules]<|im_end|>
  <|im_start|>user
  [Legal Clause Input]<|im_end|>
  <|im_start|>assistant
  [Architectural Plan + Secure Solidity Code]<|im_end|>
  

3. Execution Interface & Local UI

• Quantization Format: Q4_K_M GGUF (Compiles the 3B layers into a highly optimized ~2GB package, allowing smooth execution on standard consumer hardware).
• User Interface (UI): LM Studio (Serving as our secure, local desktop testing environment to run inferencing completely offline without dependency on external third-party API costs).

---

Step-by-Step Deployment Guide

Step 1: Model Training (Cloud Pipeline)

1. Ingest your training file (solidity_dataset.json) into the Google Colab workspace.
2. Run the fine_tune_qwen.py script to initiate the TRL SFTTrainer.
3. Note on Troubleshooting: Standard TrainingArguments can cause serialization crashes (PicklingError) at step checkpointing on newer versions of trl. This is bypassed by utilizing the native SFTConfig block and setting save_strategy = "no".
4. Upon loop completion, mount Google Drive and call model.save_pretrained_gguf to natively compile the layers into a quantized GGUF format.

Step 2: Setting up the Local Desktop UI (LM Studio)

1. Download the generated .gguf weights from your drive.
2. Place the file inside your local LM Studio models repository directory (e.g., C:\Users\<User>\.cache\lm-studio\models\local\qwen-solidity\).
3. Open LM Studio and switch to the Chat View using the speech bubble icon on the left panel.
4. Click the model dropdown menu at the very top and select your newly loaded local model file.

Step 3: Enforcing Runtime System Alignment

Because a 3B parameter model is highly compact, it can occasionally slip back into older, pre-2020 compiler quirks (like declaring constructor() public or forgetting state mutations). To solve this, you must apply explicit behavioral boundaries in the UI:

1. Expand the Right-Hand Configuration Sidebar in LM Studio.
2. Scroll to the "System Prompt" card and toggle it to ON.
3. Paste the following instructions into the prompt window and click Apply Changes:

You are an expert Solidity compiler assistant specializing in modern, secure smart contracts conforming strictly to version 0.8.20.

OUTPUT FORMAT RULES:
1. Always start your response with a clean "ARCHITECTURE PLAN" broken down into "State Variables:" and "Functions:".
2. Follow the plan with a clean markdown code block starting with exact keyword "solidity".
3. NEVER add visibility modifiers (like 'public') to the 'constructor'. Write it as: constructor() { ... }
4. If a function is restricted to the owner, apply the `onlyOwner` modifier to its definition.

Verification & Testing
Once the UI is configured, test the alignment with a prompt containing a real-world legal constraint:

Plaintext
Convert the following legal clause into a secure Solidity 0.8.20 smart contract.
[CLAUSE]
1. The contract must track a public timestamp marking a release deadline.
2. The deployment address is designated as the contract Owner.
3. The Owner can extend this release deadline to a new timestamp in the future.
[/CLAUSE]
Expected Quality Control Metrics:
Structural Parsing: The model outputs an unstructured text ARCHITECTURE PLAN detailing functions and states prior to any code generation.

Syntax Integrity: The generated smart contract omits constructor visibility, applies the custom onlyOwner access modifier, and cleanly validates structural states.