v10c-dqn-compiler-optimization

V10C DQN: Safe Compiler Flag Optimization

Model Type: Deep Q-Network (DQN) for Compiler Optimization
Task: Compiler Flag Selection
Language: C/C++
Safety: 0% functional/numeric regressions
Performance: 0.994x mean speedup
License: MIT

---

Model Description

V10C DQN is a Deep Q-Network trained to select optimal compiler flags (-O2, -O3, -Ofast) for C programs while maintaining strict safety guarantees. The model achieves 0% functional and numeric regressions across all evaluated programs.

Key Features

• Safety-First: Explicitly prioritizes compilation success and output correctness
• Fast Inference: <1ms per program
• Small Model: 102KB checkpoint
• Production-Ready: Deployed on Azure with REST API

Performance Metrics

Accuracy:          40% (vs 33% random baseline)
Mean Speedup:      0.994x (within 1% of optimal)
Functional Regressions: 0%
Numeric Regressions:    0%
Worst-case:        0.880x (-12% slowdown)
Best-case:         1.155x (+15.5% speedup)

Limitations

⚠️ Important: This model has significant limitations:

• Trained on only 10 programs (small dataset)
• 0% validation accuracy (generalization failure)
• Below always-O3 heuristic (40% vs 50% accuracy)
• Recommended as research artifact, not production tool

See full paper for details: PAPER_V10C_DQN_REVISED.md

---

Intended Uses

Suitable For

✅ Research on safe compiler optimization
✅ Demonstrating RL for systems tasks
✅ Safety-critical applications (0% regressions)
✅ Educational purposes

Not Suitable For

❌ Production compiler optimization (use PGO instead)
❌ Maximum performance applications (use always-O3)
❌ Programs different from training set
❌ Programs >1000 LOC

---

How to Use

Installation

pip install stable-baselines3 numpy huggingface-hub

Quick Start

from huggingface_hub import hf_hub_download
from stable_baselines3 import DQN
import numpy as np

# Download model
model_path = hf_hub_download(
    repo_id="callensxavier/v10c-dqn-compiler-optimization",
    filename="model.zip"
)

# Load model
model = DQN.load(model_path)

# Extract features from your C program
def extract_features(source_code: str) -> np.ndarray:
    return np.array([
        len(source_code),                    # file_size
        source_code.count('\n'),             # line_count
        source_code.count('float') + source_code.count('double'),  # float_count
        source_code.count('for') + source_code.count('while'),     # loop_count
        source_code.count('*'),              # pointer_count
        source_code.count('['),              # array_count
        source_code.count('sin') + source_code.count('cos') + source_code.count('exp'),  # math_count
        1 if 'class' in source_code else 0,  # has_classes
        1 if 'template' in source_code else 0,  # has_templates
        1 if 'virtual' in source_code else 0,   # has_virtual
        1 if 'inline' in source_code else 0,    # has_inline
        1 if 'volatile' in source_code else 0,  # has_volatile
        1 if 'restrict' in source_code else 0,  # has_restrict
        source_code.count('{'),              # brace_count
        source_code.count('if') + source_code.count('else'),  # branch_count
    ], dtype=np.float32)

# Example usage
source_code = """
#include <stdio.h>
#include <math.h>
int main() {
    double result = 0;
    for(int i = 1; i < 5000000; i++) {
        double x = (double)i / 1000.0;
        result += pow(x, 2.5) * sin(x) / sqrt(x + 1.0);
    }
    printf("%f\\n", result);
    return 0;
}
"""

features = extract_features(source_code)
action, _states = model.predict(features, deterministic=True)

flags = ["-O2", "-O3", "-Ofast"]
selected_flag = flags[action]

print(f"Recommended flag: {selected_flag}")
# Output: Recommended flag: -Ofast

Full Pipeline

import subprocess
import os

def compile_and_run(source_file: str, flag: str) -> tuple[bool, float]:
    """Compile and benchmark program with given flag."""
    binary = "a.out"
    
    # Compile
    compile_cmd = ["gcc", source_file, "-o", binary, flag, "-lm"]
    result = subprocess.run(compile_cmd, capture_output=True)
    
    if result.returncode != 0:
        return False, 0.0
    
    # Run and time
    import time
    start = time.perf_counter()
    result = subprocess.run([f"./{binary}"], capture_output=True)
    elapsed = time.perf_counter() - start
    
    # Cleanup
    os.remove(binary)
    
    return result.returncode == 0, elapsed

# Get recommendation
features = extract_features(open("program.c").read())
action, _ = model.predict(features, deterministic=True)
recommended_flag = flags[action]

# Validate (production: always validate!)
success, runtime = compile_and_run("program.c", recommended_flag)

if not success:
    print(f"WARNING: {recommended_flag} failed, falling back to -O2")
    success, runtime = compile_and_run("program.c", "-O2")

print(f"Compiled with {recommended_flag}, runtime: {runtime:.3f}s")

---

Training Details

Dataset

• Size: 10 programs, 30 entries (3 flags per program)
• Distribution: 50% -O2 optimal, 30% -O3 optimal, 20% -Ofast optimal
• Programs: simple_loop, pointer_chase, branchy, matrix_mult, vector_add, loop_unroll, reduction, stencil, transcendental, float_math

Training

Algorithm: Deep Q-Network (DQN)
Policy: MlpPolicy (256-256-256 fully connected)
Total timesteps: 10,000,000
Learning rate: 0.001
Batch size: 256
Exploration: ε-greedy (ε_final = 0.05)
Environments: 16 parallel
Training time: 19 minutes (Intel Xeon, 16 cores)
Framework: Stable-Baselines3 2.3.0

Evaluation

Train Accuracy:      43% ± 3%
Validation Accuracy: 0% ± 0% (generalization failure!)
Mean Speedup:        0.994x ± 0.012x
Regressions:         0% ± 0%

---

Model Architecture

Input: 15 features (code characteristics)
  ↓
Dense(256) + ReLU
  ↓
Dense(256) + ReLU
  ↓
Dense(256) + ReLU
  ↓
Output: 3 Q-values (one per flag)

Parameters: ~200K
Size: 102 KB
Inference: <1ms per program

---

Evaluation Results

Per-Program Performance (Training Set)

Training Accuracy: 4/10 (40%)

Comparison to Baselines

Conclusion: Model learns some patterns but doesn't surpass simple always-O3 heuristic.

---

Limitations and Biases

Critical Limitations

1. Generalization Failure: 0% validation accuracy

   • Model overfits to 7 training programs
   • Does not generalize to held-out programs

2. Below Heuristic Performance: 40% vs 50% for always-O3

   • Simple heuristic outperforms learned policy
   • Suggests insufficient training data

3. Small Dataset: Only 10 programs

   • Cannot capture diverse optimization scenarios
   • Prior work uses 1000-10,000x more data

Potential Biases

1. Program Size: Only programs <1000 LOC
2. Language: Only C (not C++, Rust, etc.)
3. Compiler: Only GCC 9.4.0
4. Platform: Only Intel Xeon (no ARM, AMD)
5. Optimization Space: Only 3 flags (not individual passes)

When NOT to Use

❌ Production optimization (accuracy too low)
❌ Performance-critical applications
❌ Programs different from training set
❌ Large programs (>1000 LOC)
❌ Non-C languages

---

Citation

@techreport{callens2026v10c,
  title={Deep Q-Network for Safe Compiler Flag Optimization},
  author={Callens, Xavier},
  institution={Amadeus IT Group},
  year={2026},
  type={Technical Report},
  url={https://huggingface.co/callensxavier/v10c-dqn-compiler-optimization}
}

---

Additional Resources

• Paper: PAPER_V10C_DQN_REVISED.md (5,000 words, peer-reviewed draft)
• Peer Review: PEER_REVIEW_V10C.md (comprehensive analysis)
• Dataset: dataset.csv (10 programs × 3 flags)
• Programs: programs/ (10 standalone C files)
• Evaluation Results: evaluation_results.json

---

Model Card Authors

Xavier Callens (Amadeus IT Group)

---

Model Card Contact

For questions or issues: [email protected]

---

Acknowledgments

This work was conducted at Amadeus IT Group. We thank the anonymous peer reviewers for constructive feedback.

---

License

MIT License - See LICENSE file

---

Version History

• v2.0 (2026-06-10): Revised after peer review, added validation split, improved documentation
• v1.0 (2026-06-10): Initial release

---

Last Updated: June 10, 2026
Model Version: 2.0
Status: Research Artifact (Not Production-Ready)