README

Insanely Fast Whisper

An opinionated CLI to transcribe Audio files w/ Whisper on-device! Powered by 🤗 Transformers, Optimum & flash-attn

TL;DR - Transcribe 150 minutes (2.5 hours) of audio in less than 98 seconds - with OpenAI's Whisper Large v3. Blazingly fast transcription is now a reality!⚡️


pipx install insanely-fast-whisper==0.0.15 --force






Not convinced? Here are some benchmarks we ran on a Nvidia A100 - 80GB 👇

| Optimisation type    | Time to Transcribe (150 mins of Audio) |
|------------------|------------------|
| large-v3 (Transformers) (fp32)             | ~31 (31 min 1 sec)             |
| large-v3 (Transformers) (fp16 + batching [24] + bettertransformer) | ~5 (5 min 2 sec)            |
| large-v3 (Transformers) (fp16 + batching [24] + Flash Attention 2) | ~2 (1 min 38 sec)            |
| distil-large-v2 (Transformers) (fp16 + batching [24] + bettertransformer) | ~3 (3 min 16 sec)            |
| distil-large-v2 (Transformers) (fp16 + batching [24] + Flash Attention 2) | ~1 (1 min 18 sec)           |
| large-v2 (Faster Whisper) (fp16 + beam_size [1]) | ~9.23 (9 min 23 sec)            |
| large-v2 (Faster Whisper) (8-bit + beam_size [1]) | ~8 (8 min 15 sec)            |

P.S. We also ran the benchmarks on a Google Colab T4 GPU instance too!

P.P.S. This project originally started as a way to showcase benchmarks for Transformers, but has since evolved into a lightweight CLI for people to use. This is purely community driven. We add whatever community seems to have a strong demand for! 

 🆕 Blazingly fast transcriptions via your terminal! ⚡️

We've added a CLI to enable fast transcriptions. Here's how you can use it:

Install insanely-fast-whisper with pipx (pip install pipx or brew install pipx):

bash
pipx install insanely-fast-whisper


⚠️ If you have python 3.11.XX installed, pipx may parse the version incorrectly and install a very old version of insanely-fast-whisper without telling you (version 0.0.8, which won't work anymore with the current BetterTransformers). In that case, you can install the latest version by passing --ignore-requires-python to pip:

bash
pipx install insanely-fast-whisper --force --pip-args="--ignore-requires-python"


If you're installing with pip, you can pass the argument directly: pip install insanely-fast-whisper --ignore-requires-python.


Run inference from any path on your computer:

bash
insanely-fast-whisper --file-name 

Note: if you are running on macOS, you also need to add --device-id mps flag.

🔥 You can run Whisper-large-v3 w/ Flash Attention 2 from this CLI too:

bash
insanely-fast-whisper --file-name  --flash True 


🌟 You can run distil-whisper directly from this CLI too:

bash
insanely-fast-whisper --model-name distil-whisper/large-v2 --file-name  


Don't want to install insanely-fast-whisper? Just use pipx run:

bash
pipx run insanely-fast-whisper --file-name 


 [!NOTE]
 The CLI is highly opinionated and only works on NVIDIA GPUs & Mac. Make sure to check out the defaults and