Faster whisper

One feature of Whisper I think people underuse is the ability to prompt the model to influence the output tokens. Some examples from my terminal history:, faster whisper. Although I seem to have trouble to get the context to persist across hundreds faster whisper tokens. Tokens that are corrected may revert back to the model's underlying tokens if they weren't repeated enough.

For reference, here's the time and memory usage that are required to transcribe 13 minutes of audio using different implementations:. Unlike openai-whisper, FFmpeg does not need to be installed on the system. There are multiple ways to install these libraries. The recommended way is described in the official NVIDIA documentation, but we also suggest other installation methods below. On Linux these libraries can be installed with pip. Decompress the archive and place the libraries in a directory included in the PATH. The module can be installed from PyPI :.

Faster whisper

Faster-whisper is a reimplementation of OpenAI's Whisper model using CTranslate2, which is a fast inference engine for Transformer models. This container provides a Wyoming protocol server for faster-whisper. We utilise the docker manifest for multi-platform awareness. More information is available from docker here and our announcement here. Simply pulling lscr. This image provides various versions that are available via tags. Please read the descriptions carefully and exercise caution when using unstable or development tags. When using the gpu tag with Nvidia GPUs, make sure you set the container to use the nvidia runtime and that you have the Nvidia Container Toolkit installed on the host and that you run the container with the correct GPU s exposed. See the Nvidia Container Toolkit docs for more details. For more information see the faster-whisper docs ,. To help you get started creating a container from this image you can either use docker-compose or the docker cli. Containers are configured using parameters passed at runtime such as those above. For example, -p would expose port 80 from inside the container to be accessible from the host's IP on port outside the container.

Simply pulling lscr.

.

Released: Mar 1, View statistics for this project via Libraries. Tags openai, whisper, speech, ctranslate2, inference, quantization, transformer. For reference, here's the time and memory usage that are required to transcribe 13 minutes of audio using different implementations:. Unlike openai-whisper, FFmpeg does not need to be installed on the system. There are multiple ways to install these libraries.

Faster whisper

The Whisper models from OpenAI are best-in-class in the field of automatic speech recognition in terms of quality. However, transcribing audio with these models still takes time. Is there a way to reduce the required transcription time? Of course, it is always possible to upgrade hardware.

Sony dsc h300

Tip We recommend Diun for update notifications. Here is a non exhaustive list of open-source projects using faster-whisper. Flagged, fork of project that launched last week that did this and had its own HN story. You would only need one model, like we have today. I've seen this happen where a blog or site is mentioned and the author shows up. VAD filter. When using the gpu tag with Nvidia GPUs, make sure you set the container to use the nvidia runtime and that you have the Nvidia Container Toolkit installed on the host and that you run the container with the correct GPU s exposed. So it theoretically influences the whole thing, just diluted and indirectly. MIT license. Custom properties. Ensure any volume directories on the host are owned by the same user you specify and any permissions issues will vanish like magic.

The best graphics cards aren't just for gaming, especially not when AI-based algorithms are all the rage. The last one is our subject today, and it can provide substantially faster than real-time transcription of audio via your GPU, with the entire process running locally for free. You can also run it on your CPU, though the speed drops precipitously.

I'd be interested in running this over a home camera system, but it would need to handle not talking well. It's also much easier to just rattle off a list of potential words that you know are going to be in the transcription that are difficult or spelled differently. TOMDM 3 months ago parent next [—]. Could be useful in a home automation context - give it a bit of clues about the environment and tech. Comparing performance against other implementations. Other tools that automatically update containers unattended are not recommended or supported. Related, does it handle whitenoise ie not talking well? We've changed the URL now. You signed in with another tab or window. But it will influence the initial text generated, which influences the subsequent text as well. See the conversion API.