faster-whisper-small-int8

faster-whisper-small-int8

INT8-quantized CTranslate2 export of openai/whisper-small, built for faster-whisper and tuned to actually run on low-end hardware — tested down to 2 CPU cores / 4GB RAM, no GPU required.

This was built out of a real need: cloud speech-to-text pricing was too high for an ongoing AI video generation project (text-based model, no diffusion), so this model exists to make good speech-to-text usable on modest, everyday machines instead.

Why this exists

• Cloud STT APIs get expensive fast at any real volume.
• The original openai/whisper-small checkpoint is heavier than most laptops or low-end VPS boxes can comfortably run.
• INT8 quantization via CTranslate2 cuts the memory/size footprint by roughly 75% with minimal accuracy loss, and this repo ships that export pre-built and ready to load.

Model details

Usage

from faster_whisper import WhisperModel

model = WhisperModel(
    "devxyasir/faster-whisper-small-int8",
    device="cpu",
    compute_type="int8",
    cpu_threads=2,      # match your machine's core count
    num_workers=1,      # keep memory predictable on low-RAM boxes
)

segments, info = model.transcribe(
    "audio.mp3",
    beam_size=5,        # default quality; try 3 or 2 first if you need more speed
    vad_filter=True,    # skip silent stretches
)

print(f"Detected language: {info.language} ({info.language_probability:.2f})")
for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

If you need more speed than beam_size=5 gives you

Don't jump straight to beam_size=1 — try 3 first, it keeps most of the accuracy while recovering a good chunk of the speed. Also worth adding: condition_on_previous_text=False (faster, more stable on long/noisy audio) and a shorter chunk_length (e.g. 20) if memory is tighter than 4GB.

Need to go below INT8?

CTranslate2 doesn't support sub-8-bit kernels on CPU. If you need to go lower than this repo, look at a whisper.cpp GGML export (Q5_0/Q4_0) of the same base checkpoint — different runtime, genuinely lower bit-width, with a real accuracy trade-off at Q4.

Limitations

• Quantization is applied uniformly; extremely quiet, overlapping-speaker, or heavily accented audio may see slightly more degradation than clean studio audio.
• Tested primarily on CPU inference; GPU users are usually better served by the original unquantized checkpoint or int8_float16 on CUDA.
• Inherits any limitations and biases of the base openai/whisper-small Whisper checkpoint.

Credits

This is a community-quantized export, not an official OpenAI or SYSTRAN release. If this model is useful in your project, a credit/link back and a follow are genuinely appreciated — it's built and maintained by one person, not a team.

---

About the developer

Muhammad Yasir — Senior AI Engineer (full-stack: web, mobile, desktop, AI systems)

• 🤗 Hugging Face: huggingface.co/devxyasir
• 💻 GitHub: github.com/devxyasir
• 🐦 X / Twitter: @devxyasir
• 📸 Instagram: @devxyasir
• 💼 LinkedIn: linkedin.com/in/devxyasir
• 📧 Email: [email protected]
• 💬 WhatsApp only: +923156808967

Found this useful? A credit when you use it, and a follow on Hugging Face, genuinely helps a solo builder keep shipping open models like this one. Open to freelance/contract AI engineering work and full-time AI/agentic systems roles — reach out on WhatsApp or email above.