cspws-unet-attn

Model Card / Research Report: csPWS Nuclei Segmentation (U-Net + Attention, ONNX)

---

Research Significance

🔬 Scientific Impact

1. Early Cancer Detection

   • Chromatin packing alterations are early biomarkers of carcinogenesis.
   • csPWS microscopy uniquely measures nanoscale chromatin architecture that conventional light microscopy cannot resolve.
   • Automated segmentation makes it feasible to process large patient datasets and detect subtle structural shifts before visible histological changes.

2. Quantitative Chromatin Metrics

   • Segmentation enables extraction of chromatin-sensitive statistics (packing scaling, spatial variance, entropy of nuclear textures).
   • These metrics can stratify normal vs. pre-cancerous vs. malignant nuclei.

3. Clinical Translation

   • Using ONNX + quantized models, the pipeline runs on CPU and CoreML (Mac/edge devices), lowering the barrier for hospitals with limited GPU resources.
   • This accelerates translation from research to bedside diagnostics.

🌍 Significance

• Scale: Instead of manually analyzing thousands of nuclei, automated segmentation can handle tens of thousands per patient sample, yielding statistically robust insights.
• Accessibility: Open-sourcing the pipeline via Hugging Face democratizes access for labs globally, especially in regions with fewer resources.
• Research Reproducibility: Publishing checkpoints, ONNX exports, and quantized models ensures other researchers can validate and extend the work.
• Interdisciplinary Value: Bridges computational optics, AI/ML segmentation, and cancer biology.

📊 Statistics / Metrics Covered

This pipeline already computes Dice and IoU (from your eval run):

• Example:
  • Dice ≈ 0.886
  • IoU ≈ 0.870 Other statistics that become available once segmentation is reliable:

1. Nuclear Morphometrics

   • Area, perimeter, circularity, eccentricity.
   • Detects abnormal nuclear shapes (common in cancer).

2. Chromatin Packing Scaling (D)

   • From csPWS signal variance: higher fractal dimension D = more disordered packing.
   • Has been correlated with cancer aggressiveness.

3. Population-Level Distributions

   • Aggregated histograms of chromatin variance across >10,000 nuclei per slide.
   • Can give p-values, effect sizes, and confidence intervals between normal vs diseased groups.

4. Clinical Performance Metrics

   • ROC-AUC, sensitivity, specificity in detecting precancerous lesions.
   • Statistical power increases by 1–2 orders of magnitude compared to manual biopsy scoring.

✅ In short: This research is impactful because it makes nanoscale chromatin disorder quantifiable at scale, provides automated reproducible segmentation, and has direct significance in early cancer diagnostics, risk stratification, and treatment planning.

---

Key Terminology Legend: csPWS Nuclei Segmentation Model

Core Technology & Imaging

• csPWS (Chromatin-sensitive Partial Wave Spectroscopic microscopy) - Advanced microscopy technique that analyzes how light interacts with chromatin (DNA packaging) in cell nuclei. Sensitive to nanoscale changes in chromatin organization, making it useful for early cancer detection.
• Chromatin Packing Heterogeneity - Variations in how DNA is packaged within cell nuclei. Irregular packing patterns can indicate cancerous changes, serving as a biomarker for cellular health.
• Σ/BF/SW (Sigma/Brightfield/Shortwave) - Three imaging modalities combined into synthetic data: Sigma (composite signal), BF (standard transmitted light microscopy), and SW (specific wavelength imaging).

Machine Learning Architecture

• U-Net (U-shaped Network) - Convolutional neural network with encoder-decoder structure shaped like the letter "U". Particularly effective for medical image segmentation tasks where precise localization is required.
• Attention Gates - Neural network components that help the model focus on relevant features by selectively emphasizing important spatial regions while suppressing irrelevant background information.
• Encoder-Decoder - Two-part neural network structure where the encoder progressively reduces spatial resolution while increasing feature depth, and the decoder reconstructs full-resolution output using encoded features.
• Conv2d (2D Convolution) - Convolutional layer that applies filters to extract spatial features from images through sliding window operations.
• MaxPool (Max Pooling) - Downsampling operation that retains the maximum value in each region, effectively reducing spatial dimensions while preserving important features.
• ReLU (Rectified Linear Unit) - Activation function that outputs zero for negative inputs and leaves positive values unchanged, introducing non-linearity to the network.
• Sigmoid - Activation function that squashes output values to the range 0-1, commonly used for binary classification tasks.

Model Optimization & Deployment

• ONNX (Open Neural Network Exchange) - Standard format for representing machine learning models, enabling portability across different frameworks and platforms.
• INT8 (8-bit Integer Quantization) - Technique that converts 32-bit floating-point weights to 8-bit integers, reducing memory usage by approximately 75% with minimal accuracy loss.
• QDQ (Quantize-Dequantize) - Specific quantization format that maintains model structure while reducing numerical precision for efficiency gains.
• Per-Channel Quantization - More precise quantization method that applies different quantization parameters to each channel or filter, better preserving model accuracy compared to global quantization.
• Opset (Operation Set) - ONNX version specification that defines the available operations for model representation (example: opset=17).

Training & Evaluation Metrics

• BCE (Binary Cross-Entropy) - Loss function for binary classification that measures pixel-wise accuracy between predicted and true labels, penalizing incorrect classifications.
• Dice Coefficient - Overlap-based similarity metric ranging from 0 (no overlap) to 1 (perfect overlap), measuring how well predicted and true segmentation masks align.
• IoU (Intersection over Union) - Ratio of overlapping area to total area covered by both predicted and true masks, serving as a standard evaluation metric for segmentation tasks.
• Combined Loss - Final loss function calculated as BCE + (1 - Dice), balancing pixel-wise accuracy and region-based overlap measures.

Data & Processing

• Synthetic Data - Artificially generated training images created by fusing Sigma/BF/SW channels into RGB composites, used due to limited availability of labeled real csPWS data.
• Tiled Inference - Processing technique that divides large images into smaller tiles, enabling high-resolution image processing within memory constraints by processing tiles separately and stitching results.
• Data Augmentation - Techniques to artificially expand the training dataset through transformations like random flips, rotations, and intensity shifts, improving model generalization and reducing overfitting.
• RGB (Red-Green-Blue) - Color model representing images using three color channels, used for displaying the fused microscopy data.
• PNG (Portable Network Graphics) - Lossless image format used for storing processed microscopy images without quality degradation.

Technical Infrastructure

• Hydra - Configuration management framework that enables flexible experiment setup and hyperparameter tuning without modifying code directly.
• MLflow (Machine Learning Flow) - Platform for managing the complete machine learning lifecycle, including experiment tracking, metric logging, and model version management.
• Gradio - Framework for creating web-based machine learning interfaces, providing user-friendly demonstrations and interaction capabilities for trained models.
• CLI (Command Line Interface) - Text-based interface for executing scripts and commands, enabling automated workflows and batch processing.
• CI/CD (Continuous Integration/Continuous Deployment) - Automated pipeline for testing, building, and deploying software, ensuring consistent and reliable model deployment.

Performance & Hardware

• MPS (Metal Performance Shaders) - Apple's GPU acceleration framework optimized for Apple Silicon processors (M1/M2/M3), providing faster training and inference on Mac systems.
• CPU (Central Processing Unit) - General-purpose processor for standard computing tasks, used for model inference when specialized accelerators are unavailable.
• GPU (Graphics Processing Unit) - Specialized processor optimized for parallel computations, significantly accelerating machine learning training and inference operations.
• TPU (Tensor Processing Unit) - Google's custom AI accelerator chips designed specifically for machine learning workloads, offering high performance for tensor operations.
• CoreML - Apple's machine learning framework optimized for on-device inference on Apple platforms, enabling efficient mobile deployment.

Cloud & Storage

• Azure Blob (Azure Binary Large Object) - Microsoft's cloud storage service for storing large amounts of unstructured data, used for model artifacts and dataset storage.
• HF (Hugging Face) - Popular platform for sharing machine learning models, datasets, and creating interactive demonstrations, facilitating model distribution and collaboration.
• API (Application Programming Interface) - Set of protocols and tools for building software applications, enabling programmatic access to models and services.

Research Methodology

• Internal Validity - Quality of experimental design and data collection. Primary concern: synthetic morphology may not accurately represent real nuclei characteristics.
• External Validity - Generalizability of results to other contexts. Limitation: benchmarks limited to specific hardware configurations, unknown performance on other systems.
• Construct Validity - Accuracy of measurements in representing intended concepts. Issue: binary masks may ignore important intra-nuclear chromatin domain information.
• Statistical Validity - Reliability of statistical analysis and conclusions. Concerns include limited validation set size and lack of cross-seed analysis for robustness testing.
• LLD (Low-Level Design) - Detailed technical specification of system architecture, implementation details, and component interactions.

File Formats & Extensions

• .pt (PyTorch) - PyTorch model checkpoint format containing trained weights, model architecture, and training state information.
• .onnx (ONNX Model) - ONNX format model file designed for cross-platform deployment and framework interoperability.
• .json (JavaScript Object Notation) - Human-readable text format for storing structured data, commonly used for configuration files and results storage.
• .yaml/.yml (YAML Ain't Markup Language) - Human-readable data serialization format used for configuration files, preferred for its readability and hierarchical structure.

---

🎯 Objective

To segment nuclei from chromatin-sensitive Partial Wave Spectroscopic (csPWS) microscopy images in order to quantify chromatin packing heterogeneity — a biomarker for early cancer detection.

---

❓ Research Questions

1. Can synthetic Σ/BF/SW fused composites provide a strong baseline for nuclei segmentation?
2. Does a U-Net with attention achieve robust segmentation accuracy?
3. Can ONNX export + INT8 quantization provide portable, efficient deployment without accuracy loss?
4. What are the limitations and risks of training on synthetic data alone?

---

🔧 Low-Level Design (LLD) — Architecture & Training

1. Data Pipeline

• Input: Synthetic fused PNGs (Σ/BF/SW → RGB, size 256×256).
• Preprocessing: Normalize to [0,1]; random flips/rotations/intensity shifts.
• Dataset Class: NucleiDataset loads (image, mask) pairs.
• DataLoader: Batches of 4, shuffle on training, deterministic on validation.

2. Model Architecture (U-Net + Attention)

Encoder (Downsampling Path)

• Conv2d(3,64,3x3) → ReLU → Conv2d(64,64,3x3) → ReLU → MaxPool(2x2)
• Conv2d(64,128,3x3) → ReLU → Conv2d(128,128,3x3) → ReLU → MaxPool(2x2)
• Conv2d(128,256,3x3) → ReLU → Conv2d(256,256,3x3) → ReLU → MaxPool(2x2)
• Conv2d(256,512,3x3) → ReLU → Conv2d(512,512,3x3) → ReLU → MaxPool(2x2)
• Bottleneck: Conv2d(512,1024,3x3) → ReLU → Conv2d(1024,1024,3x3) → ReLU

Decoder (Upsampling Path with Attention Gates)

• Upconv(1024→512) + AttentionGate(skip=512, dec=512) → Conv(1024→512→512)
• Upconv(512→256) + AttentionGate(skip=256, dec=256) → Conv(512→256→256)
• Upconv(256→128) + AttentionGate(skip=128, dec=128) → Conv(256→128→128)
• Upconv(128→64) + AttentionGate(skip=64, dec=64) → Conv(128→64→64)
• Output: Conv2d(64,1,1x1) → Sigmoid activation

3. Loss Computation

• Binary Cross-Entropy (BCE): per-pixel classification.
• Dice Loss: overlap-based similarity.
• Final Loss: Loss = BCE + (1 - Dice)

4. Backpropagation

• Compute forward pass → loss.
• loss.backward() computes gradients for all conv/attn weights.
• Gradients propagate decoder → encoder.
• Optimizer: Adam (lr=1e-3).
• Step: optimizer.step(), then optimizer.zero_grad().

5. Training Loop

• Repeat: forward → loss → backward → update weights.
• Validate after each epoch (Dice, IoU metrics).
• Save best checkpoint best.pt.

6. Export & Quantization

• Export PyTorch → ONNX (opset=17).
• Simplify with onnxsim.
• Quantize with per-channel QDQ (INT8).
• Performance: PyTorch (MPS) ≈0.13s/img, ONNX (CPU) ≈0.17s/img, INT8 CoreML ≈0.12s/img.

7. Deployment

• Hugging Face Model Repo: ONNX + quantized weights.
• Hugging Face Space: Gradio inference UI.
• Azure Blob Storage: Artifact storage for CI/CD.

---

📊 Results (Synthetic Benchmark)

---

🔍 Output Interpretation

• Output: Binary mask (white = nucleus, black = background).
• Meaning: White regions = nuclear boundaries for chromatin packing analysis.
• Significance: Early cancer biomarker detection via chromatin heterogeneity.

---

⚠️ Threats to Validity

• Internal: Synthetic morphology may not match real nuclei.
• External: Limited to Apple M3 benchmarks, unknown on GPUs/TPUs.
• Construct: Binary masks ignore intra-nuclear chromatin domains.
• Statistical: Limited validation set, no cross-seed analysis.

---

🛠️ Setup & CLI

Install

git clone https://github.com/jahidularafat/cspws_v3.git
cd cspws_v3
python -m venv .venv && source .venv/bin/activate
pip install -U pip
pip install -e .

Generate Data

python tools/make_synthetic_dataset_adv.py --out data/synth_adv --n 120 --channels 3

Train

python scripts/train_hydra.py --config-path cfg --config-name cspws_unet_attn train.epochs=5

Evaluate

python scripts/eval.py --ckpt runs/cspws_unet_attn/best.pt --data-root data/synth_adv

Export ONNX

python scripts/export_onnx.py --weights runs/cspws_unet_attn/best.pt --channels 3 --img-size 256 --out exports/model.onnx

Quantize

python -m onnxsim exports/model.onnx exports/model.sim.onnx
python scripts/quantize_static.py --model exports/model.sim.onnx --data data/synth_adv/train/images --out exports/model.int8.onnx --img-size 256 --channels 3 --per-channel --format qdq

Benchmark

python scripts/bench.py --data-root data/synth_adv --ckpt runs/cspws_unet_attn/best.pt --onnx exports/model.int8.onnx --channels 3 --img-size 256 --thresholds 0.3,0.5,0.7 --repeats 5 --out benchmark_results.json --mlflow

Inference

python scripts/infer_tiled.py --ckpt runs/cspws_unet_attn/best.pt --in data/synth_adv/test/images --out runs/preds_test --tile-size 256 --tile-overlap 32

Compute Metrix

python scripts/compute_metrics.py --images data/synth_adv/test/images --masks runs/preds_test --out runs/metrics.csv

Gradio App

python scripts/gradio_app.py --onnx exports/model.int8.onnx --img-size 256 --channels 3

Publish to Hugging Face

HF_TOKEN=xxx ./setup_hf.sh --user <hf-username> --model-repo cspws-unet-attn --space-repo cspws-space --onnx exports/model.sim.onnx --quant exports/model.int8.onnx --ckpt runs/cspws_unet_attn/best.pt

Publish to Azure

export AZURE_STORAGE_CONNECTION_STRING="..."
export AZURE_CONTAINER="cspws-artifacts"
az storage blob upload-batch -d $AZURE_CONTAINER -s exports/

---

✅ Summary

• Architecture (LLD): U-Net + Attention with explicit layers, loss, backprop, quantization.
• Results: Dice=0.89, IoU=0.87 on synthetic.
• Outputs: Nuclei masks, interpretable in biological context.
• Deployment: Hugging Face + Azure ready.

---

Metric.csv data interpretation

This appears to be evaluation metrics for an image segmentation or object detection model, showing performance results for 18 different images (IDs 00102-00116).
Here's what each metric means:
Dice Score (0-1, higher is better)

Measures overlap between predicted and ground truth segmentations
Also called F1-score for segmentation
Your results: 0.90-0.95, which is quite good

IoU (Intersection over Union, 0-1, higher is better)

Measures the overlap area divided by the union area
Standard metric for object detection/segmentation
Your results: 0.82-0.90, indicating strong performance

Precision (0-1, higher is better)

Of all pixels predicted as positive, how many were actually positive
Your results: 0.96-0.99, very high - the model rarely makes false positive predictions

Recall (0-1, higher is better)

Of all actual positive pixels, how many were correctly identified
Your results: 0.83-0.91, good but lower than precision

Key Observations:

The model has very high precision (96-99%) but relatively lower recall (83-91%)
This suggests the model is conservative - it's very accurate when it makes predictions but might miss some positive cases
Overall performance is strong across all images with consistent results
The trade-off between precision and recall indicates the model prioritizes avoiding false positives

This pattern is common in medical imaging or other applications where false positives are costly.

📚 Citation

@article{csPWS2025,
  title={Nuclei segmentation using chromatin-sensitive partial wave spectroscopic microscopy},
  author={Arafat, J. and collaborators},
  year={2025}
}