flash-attn-windows-wheels

Flash Attention Prebuilt Wheels for Windows

Prebuilt flash-attn wheels for Windows, focused on combinations that aren't published anywhere else (newer PyTorch versions, CUDA 13, Python 3.12+, consumer Blackwell support).

All wheels are built from the official Dao-AILab/flash-attention source and bundle kernels for multiple GPU architectures, so a single wheel works across most NVIDIA cards from Ampere through consumer Blackwell.

Available Wheels

Want a combination that isn't here? Open a discussion on the Community tab.

Install

Pick the wheel matching your environment and run:

pip install https://huggingface.co/Sumitc13/flash-attn-windows-wheels/resolve/main/<WHEEL_FILENAME>

Example:

pip install https://huggingface.co/Sumitc13/flash-attn-windows-wheels/resolve/main/flash_attn-2.8.3+cu130torch2.11.0cxx11abiTRUE-cp312-cp312-win_amd64.whl

Picking the Right Wheel

The filename encodes everything you need to match:

flash_attn-{VERSION}+cu{CUDA}torch{TORCH}cxx11abi{ABI}-cp{PY}-cp{PY}-win_amd64.whl
            └─2.8.3─┘ └─130─┘   └─2.11.0─┘     └─TRUE─┘   └─312─┘

Verify your environment first:

python -c "import torch; print(torch.__version__, torch.version.cuda)"
# e.g. "2.11.0+cu130 13.0" → use cu130 + torch2.11.0 wheel

GPU Support

Wheels are built with TORCH_CUDA_ARCH_LIST=8.0;8.6;8.9;9.0;12.0 unless noted otherwise, covering:

Verify Installation

import torch
from flash_attn import flash_attn_func

q = torch.randn(1, 8, 128, 64, dtype=torch.float16, device='cuda')
print(flash_attn_func(q, q, q).shape)
# Expected: torch.Size([1, 8, 128, 64])

Build Environment

• OS: Windows 11 (64-bit)
• Compiler: MSVC 14.44 (Visual Studio 2022 Community)
• CUDA Toolkit: matches the cu* tag in each wheel filename
• Source: official Dao-AILab/flash-attention release tag matching the wheel version
• No source patches — these are stock builds with the documented CUDA 13 + MSVC preprocessor flag (NVCC_FLAGS=--compiler-options /Zc:preprocessor).

License

BSD-3-Clause, matching upstream Flash Attention.

Disclaimer

Unofficial community builds provided as-is with no warranty. Not affiliated with Dao-AILab or NVIDIA.