GLM-4.5-Air-HS-LoRA-CurriculumLearning

AdityaNarayan/GLM-4.5-Air-HS-LoRA-CurriculumLearning

GLM-4.5-Air-HS-LoRA-CurriculumLearning is a text generation model from AdityaNarayan. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.

A LoRA fine-tuned version of GLM-4.5-Air (108B MoE) trained on the Hyperswitch codebase using Phased Curriculum Learning.

Downloads · 30 days

30% of all-time downloads

All-time downloads

Public

Repo size

1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors507 MB · 96%

At a glance

Task: Text Generation
Library: peft
License: apache-2.0
Access: Public
Created: Dec 13, 2025
Updated: Dec 16, 2025
SHA: fec3222a

Try a prompt

Base models

zai-org/GLM-4.5-Air

Datasets

AdityaNarayan/HS-Repo-Curriculum-Learning

Model card

From the Hugging Face model README

GLM-4.5-Air-HS-LoRA-CurriculumLearning

A LoRA fine-tuned version of GLM-4.5-Air (108B MoE) trained on the Hyperswitch codebase using Phased Curriculum Learning.

Model Description

This model is specifically trained to understand and assist with the Hyperswitch payment orchestration codebase. It was trained using a 3-phase curriculum learning approach on multi-node H200 GPUs with PyTorch FSDP.

Key Features

🎯 Domain-Specific: Trained exclusively on Hyperswitch Rust codebase
📚 Curriculum Learning: 3-phase progressive training (Foundation → Evolution → PR Mastery)

Training Details

Hardware Configuration

Component	Specification
GPUs	16× NVIDIA H200 (144GB each)
Nodes	2 nodes × 8 GPUs
Distributed Strategy	PyTorch FSDP (Full Shard)
Precision	BF16 Mixed Precision

LoRA Configuration

Parameter	Value
LoRA Rank (r)	128
LoRA Alpha	256
LoRA Dropout	0.05
Target Modules	q_proj, k_proj, v_proj, o_proj
Trainable Parameters	368 tensors

Training Hyperparameters

Parameter	Value
Effective Batch Size	32 (1 × 2 grad_accum × 16 GPUs)
Sequence Length	16,384 tokens
Chunk Overlap	2,048 tokens
LR Scheduler	Cosine
Weight Decay	0.01
Max Grad Norm	1.0
Precision	BF16

Curriculum Learning Phases

The model was trained using a 3-phase curriculum learning approach, where each phase builds upon the previous:

Phase 1: Foundation (2 epochs)

Metric	Value
Dataset	Codebase structure and file patterns
Samples	9,293 train / 512 eval
Learning Rate	2.5e-5
Warmup Ratio	0.15
Training Time	12.7 hours
Final Eval Loss	0.365
Final Eval Accuracy	88.8%

Phase 2: Evolution (2 epochs)

Metric	Value
Dataset	Commit patterns and code changes
Samples	16,622 train / 1,545 eval
Learning Rate	2.0e-5
Warmup Ratio	0.10
Training Time	24.7 hours
Final Eval Loss	2.55
Final Eval Accuracy	40.8%

Note: Higher loss in Phase 2 is expected due to the complexity of diff/commit patterns.

Phase 3: PR Mastery (1 epoch)

Metric	Value
Dataset	Pull request and review patterns
Samples	9,797 train / 509 eval
Learning Rate	1.5e-5
Warmup Ratio	0.05
Training Time	6.9 hours
Final Eval Loss	0.501
Final Eval Accuracy	90.2%

Training Summary

Metric	Value
Total Training Time	44.9 hours
Total Steps	1,926
Total Epochs	5 (2 + 2 + 1)
Initial Train Loss	0.592
Final Train Loss	0.495
Final Perplexity	1.65

Citation

If you use this model, please cite:

@misc{glm45air-hs-lora-curriculum,
  title = {GLM-4.5-Air-HS-LoRA-CurriculumLearning},
  author = {Aditya Narayan},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/AdityaNarayan/GLM-4.5-Air-HS-LoRA-CurriculumLearning}
}

Acknowledgments

Base model: GLM-4.5-Air by Zhipu AI
Training framework: PyTorch FSDP + PEFT
Dataset: Hyperswitch open-source repository by Juspay

Task: Text Generation
Library: peft
License: apache-2.0
Languages: en
Created: Dec 13, 2025
Updated: Dec 16, 2025