schema-forg-ai

Schema Forg AI - LoRA

Overview

Schema Forg AI is a LoRA adapter trained on top of Microsoft Phi-3 Mini 4K Instruct to specialize the model in API schema generation and conversion tasks.

Rather than creating a new general-purpose language model, this LoRA teaches the base model how to generate structured API specifications with a strong focus on developer workflows.

Capabilities

After loading this adapter, the model becomes significantly better at tasks such as:

• Generate Zod schemas
• Generate OpenAPI JSON specifications
• Generate OpenAPI YAML specifications
• Convert JSON structures into API schemas
• Produce structured API documentation
• Generate consistent backend-oriented outputs

Dataset

The adapter was trained using the ERP Schema Forg Dataset, a synthetic dataset composed of thousands of examples covering schema generation and conversion tasks, including:

• JSON → Zod
• JSON → OpenAPI JSON
• JSON → OpenAPI YAML
• API documentation generation
• Nested objects
• Arrays
• Validation rules
• Endpoint specifications

The dataset was created specifically to improve the model's understanding of structured API development.

Base Model

• Microsoft Phi-3 Mini 4K Instruct

Training

This adapter was trained using Supervised Fine-Tuning (SFT) with the TRL library.

Training focused exclusively on schema-related tasks while preserving the reasoning capabilities of the original Phi-3 Mini model.

Usage

This repository contains only the LoRA adapter.

To use it, load the adapter together with the base model:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct"
)

model = PeftModel.from_pretrained(
    base_model,
    "edsoncarvalhointuria/schema-forg-ai"
)

tokenizer = AutoTokenizer.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct"
)

If you prefer a standalone model without requiring PEFT, a merged version is also available on the Hugging Face Hub.

Intended Use

This LoRA is intended for:

• API development
• Backend engineering
• OpenAPI generation
• Zod schema generation
• Developer tooling
• Educational projects
• Research on LLM specialization

Limitations

This adapter specializes the base model for schema-related tasks.

While the underlying Phi-3 model retains its general capabilities, this LoRA performs best when used for:

• OpenAPI
• Zod
• JSON Schema
• Structured JSON
• API documentation

It is not intended to replace a general-purpose instruction model.

Framework Versions

• TRL: 1.8.0
• Transformers: 5.14.1
• PyTorch: 2.5.1 + CUDA 12.1
• Datasets: 5.0.0
• Tokenizers: 0.22.2

License

Apache 2.0

Author

Developed by Edson Carvalho Inturia as an educational and research project exploring:

• LoRA fine-tuning
• Synthetic dataset generation
• Large Language Models
• API schema generation
• OpenAPI tooling
• Zod schema generation

The goal of this project is to investigate how domain-specific LoRA adapters can significantly improve structured code generation while remaining lightweight and easy to deploy.

Cite TRL as:

@software{vonwerra2020trl,
  title   = {{TRL: Transformers Reinforcement Learning}},
  author  = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
  license = {Apache-2.0},
  url     = {https://github.com/huggingface/trl},
  year    = {2020}
}