Downloads · 30 days
0
KakumuraArimura/DDPM_LDM_DDPM_VARIANCE
DDPM_LDM_DDPM_VARIANCE is a machine learning model from KakumuraArimura. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for diffusers. The card lists the license as apache-2.0.
An Undergraduate Thesis Project on Ultrasound Image Analysis with DDPM, DDPM-variance, and Latent Diffusion Models
Downloads · 30 days
0
Access
Public
Updated Apr 24, 2026
Repo size
10.8 GB
Likes
2
Public
Click a slice to open those files.
.safetensors6.4 GB · 60%
From the Hugging Face model README
An Undergraduate Thesis Project on Ultrasound Image Analysis with DDPM, DDPM-variance, and Latent Diffusion Models
This project implements and compares three generations of diffusion probabilistic models for medical image synthesis and data augmentation: Standard DDPM, DDPM with learned variance prediction, and Latent Diffusion Models (LDM). The primary objective is to evaluate the effectiveness of synthetic data augmentation for improving classification performance on a 4-class udder ultrasound image dataset. Comprehensive experiments demonstrate that data augmentation with diffusion models significantly enhances downstream classifier accuracy across multiple backbone architectures. Additionally, we explore mixed-data augmentation strategies that combine samples from different generative models to leverage complementary strengths.
Medical imaging datasets, particularly in specialized domains like veterinary ultrasound, often suffer from limited sample sizes and class imbalances. This scarcity poses significant challenges for training robust deep learning models. Traditional data augmentation techniques (flipping, rotation, color jittering) provide limited diversity and may not capture the complex anatomical variations present in medical images.
This project addresses these limitations by exploring diffusion-based data augmentation (DiffDA) for udder ultrasound images. We implement and compare three state-of-the-art diffusion models:
The generated synthetic images are used to augment training data for downstream classification tasks, with systematic evaluation across four classifier architectures: ResNet-18, Swin-T, ViT-Tiny, and ConvNeXt-Tiny.
Key Contributions:
DDPMs [1] are generative models that learn data distributions by reversing a gradual noising process. The forward process adds Gaussian noise over T timesteps:
$$ q(\mathbf{x}t|\mathbf{x}{t-1}) = \mathcal{N}(\mathbf{x}t; \sqrt{1-\beta_t}\mathbf{x}{t-1}, \beta_t\mathbf{I}) $$
The reverse process is learned by a neural network:
$$ p_\theta(\mathbf{x}{t-1}|\mathbf{x}t) = \mathcal{N}(\mathbf{x}{t-1}; \mu\theta(\mathbf{x}_t, t), \sigma_t^2\mathbf{I}) $$
Training uses a simplified objective predicting the added noise:
$$ \mathcal{L}{\text{simple}} = \mathbb{E}{t, \mathbf{x}0, \boldsymbol{\epsilon}} \left[ | \boldsymbol{\epsilon} - \boldsymbol{\epsilon}\theta(\mathbf{x}_t, t) |^2 \right] $$
Nichol & Dhariwal [2] extended DDPMs with several improvements:
These improvements yield better log-likelihoods and enable sampling with fewer steps while maintaining quality.
LDMs [3] operate in a compressed latent space using a pretrained autoencoder, dramatically reducing computational requirements:
$$ \mathcal{E}(\mathbf{x}) = \mathbf{z} \quad \text{(Encoder)}, \quad \mathcal{D}(\mathbf{z}) = \tilde{\mathbf{x}} \quad \text{(Decoder)} $$
The diffusion process occurs in the latent space $\mathbf{z} \in \mathbb{R}^{h \times w \times c}$, where $h \times w$ is 64× smaller than the original image. This enables training high-resolution diffusion models with limited computational resources.
Recent studies [4,5] have explored diffusion models for data augmentation (DiffDA). Key findings include:
train.py)UNet2DModel from HuggingFace Diffusersnum_class_embeds parametertrain_ddpm_variance.py)UNet2DConditionModel with cross-attention conditioningLabelEmbedding with 256-dim embeddings, 8 tokenstrain_ldm.py)AutoencoderKL (stabilityai/sd-vae-ft-mse)All models implement class-conditional generation via label embeddings:
# DDPM-variance: Cross-attention conditioning
class LabelEmbedding(nn.Module):
def __init__(self, num_classes=5, embedding_dim=256, num_tokens=8):
self.embedding = nn.Embedding(num_classes, embedding_dim)
self.mlp = nn.Sequential(
nn.Linear(embedding_dim, embedding_dim * 2),
nn.SiLU(),
nn.Linear(embedding_dim * 2, embedding_dim * num_tokens)
)
# LDM: Similar architecture with 512-dim embeddings
Conditioning is implemented with classifier-free guidance (CFG) during sampling, enabling trade-off between fidelity and diversity.
The complete augmentation workflow:
The project uses a udder ultrasound image dataset with 4 classes representing different anatomical or pathological conditions. The original dataset structure:
datasets/
├── train/
│ ├── 1/ # Class 1 images
│ ├── 2/ # Class 2 images
│ ├── 3/ # Class 3 images
│ └── 4/ # Class 4 images
└── test/ # Test set (not used for diffusion model training)
Key Statistics:
compare_4_model.py):Four augmentation scales were tested to study the effect of synthetic data quantity:
| Experiment | Target Images/Class | Model Used | Purpose |
|---|---|---|---|
| Base-500 | 500 | DDPM (Model 1) | Initial feasibility test |
| Base-1000 | 1000 | DDPM (Model 1) | Scaling effect analysis |
| Base-2000 | 2000 | DDPM (Model 1) | Intermediate scaling |
| Base-5000 | 5000 | DDPM (Model 1) | Full augmentation |
| DDPM-variance-5000 | 5000 | DDPM-variance (Model 2) | Improved model comparison |
| LDM-5000 | 5000 | LDM (Model 3) | Latent space efficiency |
| Mixed-LDM-0.2 | 5000 | LDM (20%) + DDPM-variance (80%) | Complementary effects study |
| Mixed-LDM-0.5 | 5000 | LDM (50%) + DDPM-variance (50%) | Balanced fusion analysis |
| Mixed-LDM-0.8 | 5000 | LDM (80%) + DDPM-variance (20%) | LDM-dominant optimization |
Generative Quality Assessment:
compute_fid.py)LPIPS.py)Classification Performance:
Ablation Studies:
FID_lpips.txt):ddpm_augmented_v1/trainldm_augmented_v2/trainLower FID indicates better distribution matching with real images. LDM achieves the best FID score (54.20), suggesting better generative quality, though DDPM-variance shows superior downstream classification performance.
FID_lpips.txt):LPIPS measures perceptual diversity - higher scores indicate more diverse generated images. Both DDPM and LDM show good diversity (>0.64), with DDPM showing slightly higher overall diversity.
The key finding: All diffusion models improved downstream classification accuracy, with DDPM-variance (Model 2) achieving the best results.
Classification_Experiments/Final_Analysis_Report_ddpm_variance_V2/Performance_Table.csv - Model 2 Results):| Model | Baseline Accuracy | Augmented Accuracy | Improvement |
|---|---|---|---|
| ResNet-18 | 68.99% | 92.31% | +23.32% |
| Swin-T | 70.89% | 94.51% | +23.62% |
| ViT-Tiny | 76.58% | 90.11% | +13.53% |
| ConvNeXt-Tiny | 70.89% | 95.05% | +24.16% |
Key Observations:
Note: These results represent the performance with DDPM-variance (Model 2) augmentation, which significantly outperforms the standard DDPM (Model 1) results shown in earlier experiments.
Figure: Confusion matrix for ResNet-18 trained on DDPM-variance (Model 2) augmented data. Shows excellent per-class accuracy with minimal confusion between classes.
Figure: Performance comparison of ResNet-18 on baseline vs. DDPM-variance augmented datasets. Demonstrates dramatic accuracy improvement from 68.99% to 92.31% (+23.32%).
Figure: Confusion matrix for ConvNeXt-Tiny trained on DDPM-variance augmented data. Shows near-perfect classification across all 4 classes with minimal errors.
Figure: ConvNeXt-Tiny performance comparison showing dramatic improvement from 70.89% to 95.05% (+24.16%) - the highest overall accuracy achieved in this study.
Figure: Comprehensive bar chart comparing all four classifiers' performance on baseline vs. DDPM-variance augmented datasets. All models achieve over 90% accuracy, with ConvNeXt-Tiny reaching 95.05% - demonstrating the exceptional effectiveness of DDPM-variance for medical image data augmentation.
Additional detailed confusion matrices and comparison charts for Swin-T and ViT-Tiny trained on DDPM-variance augmented data are available in the Final_Analysis_Report_ddpm_variance_V2/ folder.
The table below shows the performance improvement comparison across all three diffusion models for ResNet-18:
| Model | Baseline | Augmented | Improvement | Relative Advantage |
|---|---|---|---|---|
| DDPM (Model 1) | 68.99% | 83.54% | +14.55% | Baseline |
| DDPM-variance (Model 2) | 68.99% | 92.31% | +23.32% | +8.77% better than Model 1 |
| LDM (Model 3) | 68.99% | 92.41% | +23.42% | +8.87% better than Model 1 |
Analysis: Both DDPM-variance (Model 2) and LDM (Model 3) show dramatically better performance than standard DDPM (Model 1), with improvements around +23.3% vs +14.55%. While LDM shows slightly better ResNet-18 performance (+23.42% vs +23.32%), DDPM-variance achieves the highest overall accuracy with ConvNeXt-Tiny (95.05%) and best balance across all metrics (see Section 5.3).
Complete results for all models:
Final_Analysis_Report_4_2/Final_Analysis_Report_ddpm_variance_V2/Final_Analysis_Report_vdm_2/Analysis from extra_images5000_added_ddpm_vdm_variance/DDPM_VARIACNE_LDM_Data_Augmentation_Comparison_Summary.csv:
| Metric | DDPM (Model 1) | DDPM-variance (Model 2) | LDM (Model 3) |
|---|---|---|---|
| Generation Speed | Medium | Fast (fewer steps) | Slow (VAE encode/decode) |
| Memory Usage | High | Medium | Low (latent space) |
| Image Quality | Good | Excellent (edge preservation) | Very Good |
| Classification Gain | Good | Best | Good |
DDPM-variance (Model 2) emerges as the optimal choice for this application, balancing quality, speed, and downstream performance.
Figure: Comprehensive bar chart comparing all three diffusion models (DDPM, DDPM-variance, LDM) across multiple metrics including FID scores, generation speed, memory usage, and downstream classification improvement. DDPM-variance shows the best balance of quality and efficiency.
Detailed numerical comparison metrics are available in extra_images5000_added_ddpm_vdm_variance/DDPM_VARIACNE_LDM_Data_Augmentation_Comparison_Summary.csv.
Experiments with 500, 1000, 2000, and 5000 synthetic images per class reveal:
The project includes comprehensive analysis reports for each diffusion model in separate folders:
Final_Analysis_Report_4_2/: Complete analysis for DDPM (Model 1) augmentation resultsFinal_Analysis_Report_ddpm_variance_V2/: Complete analysis for DDPM-variance (Model 2) augmentation resultsFinal_Analysis_Report_vdm_2/: Complete analysis for LDM (Model 3) augmentation resultsEach folder contains confusion matrices, performance comparison charts, and detailed metrics for all four classifier backbones trained on that model's augmented dataset.
The label_test/ directory contains comprehensive visualizations of label embeddings and cross-attention mechanisms used in Models 2 & 3 (DDPM-variance and LDM). These visualizations demonstrate the effectiveness of the cross-attention conditioning mechanism:
Figure: 2D PCA projection of label embeddings showing clear separation of classes 0-4 in embedding space. Class 0 is distinctly separated, while classes 3 and 4 form a close cluster.
Figure: t-SNE visualization of label embeddings (seed 1). The non-linear dimensionality reduction reveals natural clustering patterns.
Figure: t-SNE visualization of label embeddings (seed 2). Consistent clustering across different random initializations validates embedding quality.
Figure: Mean t-SNE visualization showing stable embedding structure across multiple runs.
Figure: 5×5 cosine similarity matrix heatmap. Diagonal = 1.0 (self-similarity), off-diagonal values show semantic relationships between classes.
Figure: Euclidean distance matrix between class embeddings, quantifying separation in embedding space.
Figure: Statistical distribution of attention token norms, validating proper initialization and training of cross-attention parameters.
These visualizations confirm that the learned label embeddings form a semantically meaningful space where similar classes are closer together, enabling effective cross-attention conditioning during image generation. The consistent clustering patterns across multiple visualization techniques (PCA, t-SNE) and the structured similarity/distance matrices demonstrate that the models have learned meaningful class representations.
Recognizing the complementary strengths of different diffusion models, we conducted mixed-data augmentation experiments to investigate whether combining samples from multiple generative models could yield superior classification performance. The experimental design addresses:
Three mixed-data groups were created with different LDM-to-DDPM-Variance ratios (where a = LDM proportion):
| Group | Ratio (LDM:DDPM-Var) | Description | Target Total |
|---|---|---|---|
| A (a=0.2) | 20:80 | DDPM-Variance dominant, LDM for FID correction | 5000/class |
| B (a=0.5) | 50:50 | Balanced fusion for feature coverage | 5000/class |
| C (a=0.8) | 80:20 | LDM dominant, DDPM-Variance for diversity boost | 5000/class |
Methodology:
Results from mixed_data_Comparison_Summary.csv show mixed-data augmentation outperforms single-model approaches:
| Model | DDPM-Var Only | LDM Only | a=0.2 | a=0.5 | a=0.8 |
|---|---|---|---|---|---|
| ConvNeXt-Tiny | 95.05% | 91.77% | 95.05% | 96.15% | 97.80% |
| ResNet-18 | 92.31% | 92.41% | 93.41% | 95.05% | 95.05% |
| Swin-T | 94.51% | 90.51% | 95.60% | 93.96% | 94.51% |
| ViT-Tiny | 90.11% | 87.97% | 92.86% | 90.66% | 91.76% |
Figure: Comprehensive bar chart comparing all augmentation strategies including mixed-data experiments. Shows performance advantages of mixed-data augmentation over single-model approaches.
Figure: Multi-series line chart showing classification accuracy as a function of LDM mixing ratio (Alpha). Demonstrates how different classifier architectures respond to varying proportions of LDM and DDPM-Variance samples, with ConvNeXt-Tiny achieving peak performance at α=0.8 (80% LDM + 20% DDPM-Var).
The project follows a well-organized directory structure designed for reproducibility and clarity:
ddpm/
├── Core Model Training Scripts
│ ├── train.py # Model 1: Standard DDPM training
│ ├── train_ddpm_variance.py # Model 2: DDPM with learned variance
│ └── train_ldm.py # Model 3: Latent Diffusion Model
│
├── Data Augmentation Generators
│ ├── add_p.py # Generate images using DDPM (Model 1)
│ ├── ddpm_variance_add.py # Generate images using DDPM-variance (Model 2)
│ └── vdm_add.py # Generate images using LDM (Model 3)
│
├── Evaluation & Metrics
│ ├── FID.py # FID calculation for diffusion models
│ ├── compute_fid.py # FID/IS for augmented datasets
│ ├── LPIPS.py # LPIPS diversity score calculation
│ └── all_experiments_log.json # Log of all experiment results
│
├── Classification Experiments
│ ├── compare_4_model.py # Train 4 classifiers with advanced techniques
│ ├── jieguo.py # Generate analysis reports and visualizations
│ ├── comparison.py # Multi-experiment comparison
│ └── Classification_Experiments/ # All experiment outputs
│ ├── Augmented_Dataset_4_2/ # Best results with Model 1 augmentation
│ │ ├── models/ # Trained classifier weights
│ │ └── results/ # Performance metrics
│ ├── Augmented_Dataset_ddpm_variance_V2/ # Best results with Model 2 augmentation
│ ├── Augmented_Dataset_vdm_2/ # Best results with Model 3 augmentation
│ ├── Final_Analysis_Report_4_2/ # Final analysis for Model 1
│ ├── Final_Analysis_Report_ddpm_variance_V2/ # Final analysis for Model 2
│ ├── Final_Analysis_Report_vdm_2/ # Final analysis for Model 3
│ ├── label_test/ # Label embedding visualizations
│ ├── mixed_data/ # Mixed-data experiment results (LDM + DDPM-Var)
│ └── extra_images5000_added_ddpm_vdm_variance/ # Final comparison results
│
├── Datasets
│ ├── datasets/ # Original dataset (train/test split)
│ ├── Base_datasets/ # Original base datasets
│ ├── Base_datasets_augmented/ # DDPM-augmented datasets (500, 1000, 2000, 5000/class)
│ ├── ddpm_augmented_v1/ # Model 1 augmented dataset (5000/class)
│ ├── ddpm_variance_augmented_v2/ # Model 2 augmented dataset (5000/class)
│ └── ldm_augmented_v2/ # Model 3 augmented dataset (5000/class)
│
├── Model Checkpoints
│ ├── ddpm-udder-results*/ # Model 1 training checkpoints
│ ├── ddpm_variance_*/ # Model 2 training checkpoints
│ └── ldm_udder_v*/ # Model 3 training checkpoints
│
└── Utility Scripts
├── split.py # Dataset splitting utility
├── test_*.py # Various testing scripts
└── prompt.txt # Project notes and prompts
Base_datasets_augmented/: Traditional DDPM augmentation to 500 images/classBase_datasets_augmented_2/: Augmentation to 1000 images/classBase_datasets_augmented_3/: Augmentation to 2000 images/classBase_datasets_augmented_4/: Augmentation to 5000 images/classddpm_variance_v*/: Intermediate weights for Model 2 trainingddpm_udder_v*/: Intermediate weights for Model 1 trainingldm_udder_v*/: Intermediate weights for Model 3 trainingddpm_variance_22) contains the final trained modelClassification_Experiments/Augmented_Dataset_4_2/: Best classifier results with Model 1 augmentationClassification_Experiments/Augmented_Dataset_ddpm_variance_V2/: Best results with Model 2 augmentationClassification_Experiments/Augmented_Dataset_vdm_2/: Best results with Model 3 augmentationClassification_Experiments/Final_Analysis_Report_*/: Complete analysis reports for each modelClassification_Experiments/label_test/: Visualization of attention mechanisms and label embeddings in Models 2 & 3Classification_Experiments/mixed_data/: Mixed-data experiment results (LDM + DDPM-Var combinations)Classification_Experiments/extra_images5000_added_ddpm_vdm_variance/: Final experimental results comparison# Clone the repository
git clone [repository_url]
cd ddpm
# Create and activate conda environment (recommended)
conda create -n ddpm python=3.9
conda activate ddpm
# Install PyTorch (CUDA 11.8 example)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
# Install other dependencies
pip install diffusers accelerate transformers datasets
pip install lpips torchmetrics matplotlib scikit-learn
pip install timm # for classifier backbones
python train.py
Configuration: Edit Config class in train.py for data paths, batch size, etc.
python train_ddpm_variance.py
Configuration: Modify Config class in train_ddpm_variance.py. Key parameters: use_variance_prediction=True, cross_attention_dim=256, cfg=5.
python train_ldm.py
Note: Requires pretrained VAE from stabilityai/sd-vae-ft-mse. Set vae_model path in config.
After training, generate synthetic images to reach 5000 images per class:
python add_p.py
Configure: Set target_count=5000, model_path to trained checkpoint in add_p.py.
python ddpm_variance_add.py
Configure: Update model_path and output_dir in ddpm_variance_add.py.
python vdm_add.py
Configure: Set model_path, vae_path, and output_dir in vdm_add.py.
# Compute FID and Inception Score
python compute_fid.py
# Calculate LPIPS diversity scores
python LPIPS.py
Configuration: Update REAL_DIR and FAKE_DIR in compute_fid.py to point to real and generated datasets.
# Train all 4 classifiers on augmented dataset
python compare_4_model.py
# Generate comprehensive analysis reports
python jieguo.py
# Compare multiple experiments
python comparison.py
Configuration: Modify dataset paths and training parameters in compare_4_model.py.
To reproduce the key experiments from this study:
Baseline Augmentation (Model 1):
# Use checkpoints from ddpm-udder-results6/ or ddpm-udder-results7/
python add_p.py # Set model_path accordingly
DDPM-variance Augmentation (Model 2):
# Use checkpoints from ddpm_variance_22/
python ddpm_variance_add.py
LDM Augmentation (Model 3):
# Use checkpoints from ldm_udder_v22/
python vdm_add.py
python test_label.py generates t-SNE and cosine similarity plotstest_ddpm_variance.py, test_ldm.py produce sample imageslabel_test/ directory for cross-attention visualizationsIf you use this codebase or build upon this work, please cite:
@article{your_thesis_2026,
title={RESEARCH ON DIFFUSION-BASED IMAGE GENERATION ALGORITHMS FOR BOVINE UDDER MASTITIS},
author={Junze Ye},
journal={Undergraduate Thesis},
year={2026},
publisher={NJAU}
}
[1] Ho, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems.
[2] Nichol, A. Q., & Dhariwal, P. (2021). Improved Denoising Diffusion Probabilistic Models. International Conference on Machine Learning.
[3] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. IEEE/CVF Conference on Computer Vision and Pattern Recognition.
[4] Groh, M., et al. (2023). Evaluating the Performance of StyleGAN2-ADA on Medical Images. International Conference on Medical Image Computing and Computer-Assisted Intervention.
[5] Diffusion Models for Data Augmentation Survey (2023). arXiv preprint arXiv:2308.12453.
This work was conducted as part of an undergraduate thesis project at [NJAU]. Special thanks to my supervisor, Professor Zhai, for guidance and support throughout this project. Also thanks to the open-source community for the Diffusers library that made this research possible.
This project is available for academic research purposes. For commercial use, please contact the author.
Last Updated: April 2026
Project Status: Completed Research