Synth-Eye-GAN

Synth.Eye GAN

<p align="center"> <img src="images/Logo_White.png" alt="Synth.Eye GAN" width="480"/> </p>

Synth.Eye GAN is a data-driven extension of the Synth.Eye platform. It replaces physics-based Blender rendering with StyleGAN2-ADA to generate synthetic training images of fingerprint residue on industrial parts, which are used to train YOLO defect detection models for real-time inspection.

---

Models

---

Workflow

1. Synthetic data generation — GAN models generate front-side and fingerprint images. A compositing pipeline blends them with pressure simulation, motion blur, and alpha effects to produce labeled training images.
2. YOLO training — Both YOLO models are trained entirely on the synthetic composite dataset.
3. Real-time inspection — A PyQt5 desktop application runs both YOLO models live on a Basler camera feed for industrial surface inspection.

---

Visual Examples

GAN Output: Real vs. Synthetic

Left: real training photos captured at INTEMAC Research Center. Right: GAN-generated synthetic images from front.pkl, back.pkl, and fingerprint.pkl.

<p align="center"> <img src="images/real-vs-syth.png" alt="Real vs Synthetic GAN output" width="340"/> </p>

YOLO Inference

Dual-model inference on a real industrial part — blue frame: yolov8m_object_detection (part orientation), orange box: yolov8m_defect_detection (fingerprint residue defect).

<p align="center"> <img src="images/Yolo.png" alt="YOLOv8 dual-model detection" width="480"/> </p>

---

Training Data

---

Architecture

StyleGAN2-ADA generators use a custom fork at LukasMoravansky/stylegan2-ada-pytorch, based on the original NVlabs/stylegan2-ada-pytorch.

Fork additions (changelog):

• PyTorch 2.x compatibility fix (custom bilinear interpolation for R1 regularization)
• Windows support via STYLEGAN2_FORCE_REF_IMPL environment variable
• Training launcher with named configuration presets

YOLO models use Ultralytics YOLOv8 Medium (≥ 8.4.48).

---

Usage

GAN inference (requires the StyleGAN2-ADA fork)

import pickle
import torch

with open("front.pkl", "rb") as f:
    G = pickle.load(f)["G_ema"].cuda()

z = torch.randn(1, G.z_dim).cuda()
c = torch.zeros(1, G.c_dim).cuda()
img = G(z, c)  # (1, 3, 256, 256), range [-1, 1]

YOLO inference

from ultralytics import YOLO

model = YOLO("yolov8m_object_detection")
results = model("image.jpg", imgsz=640)

---

Links

---

⚠️ Limitations

• Small training set: Front/Back GANs were trained on ~130 real images per side. Output diversity is limited accordingly.
• Domain-specific: All models are tuned to a single industrial part type from INTEMAC Research Center. They are not general-purpose generators or detectors.
• Fingerprint domain gap: The fingerprint GAN was trained on scanned ink fingerprints (SOCOFing), which differ from optical-camera fingerprint residue on metal surfaces. Expect some visual mismatch relative to the target deployment domain.

---

License

MIT