Basic Installation and Processing

[tmyfrn]

I've been struggling with this for several weeks, sadly and with lots of online searching and ChaTGPT help. I can't get it working.

I have installed in python environments two versions of whisper. One has openai-whisper and the other has whisper-cli. I set these both to use the large-V2 model. Other options are minimal as I'm still learning to use them.

With a simple bash script I test both of these with a 10 minute audio file.

Openai-whisper takes 30 minutes to process, but gives excellent results.
Whisper-cli takes about 2 minutes or less and give horrible results.

I can't find a way to make Openai-whisper go faster (I'm using Apple M1 Max with 32g ram), and I can't find a way to make Whisper-cli improve accuracy.

As an addition: I downloaded a whisper app from the Apple Store and tested it. I had it use the large-V2 as well. It took about 2 minutes to process the 10 minute audio and gave excellent results. So it seems there is a way to achieve what I want. I do not want to use the app from the Apple Store as I need to imbed this into a script to automate the process, as well as filter the results for other purposes.

I would so appreciate any help on this. My technical level is moderate these days.

Thanks,

~ tommy

[glangford]

• Try the turbo model in your bash script and evaluate the results (--model turbo). This model often works well for transcription, but do not use it for translation
• See https://github.com/ggerganov/whisper.cpp (this is likely what your app store app is using under the covers)

[tmyfrn]

Good to know about the turbo model, I will test it right away. Will let you know the results.

I assumed that the App Store app was using the whisper.cpp (which is the whisper-cli, right) or the open-whisper. Whichever, it delivers fast and great results. I can get fast, bad results, or slow, great results. I just can't get both like that app does. I wrote the creator of that app to get some info but there has been no response.

Just to clarify: with open-whisper I get slow, great results, with the whisper-cli I get fast, bad results. But with the App Store app, I get fast and great results. That is what I need to replicate from either open-whisper or whisper-cli.

[tmyfrn]

I did some tests with the turbo model on open-whisper. The results were much better. Accuracy was very good and the speed was better, but still what I feel is very slow. A 10 minute audio file took 7 minutes, 30 seconds. That is slow, but possibly doable.

As for speed, I'm using a Mac M1 Max with 32g memory. I understand that this model can't use GPU, which they say would make it faster.

I guess I'm still wondering how this App Store app can do good accuracy and still be so fast compared to the openai-whisper that I am running.

[glangford]

I guess I'm still wondering how this App Store app can do good accuracy and still be so fast compared to the openai-whisper that I am running.

If you go through the project page for whisper-cpp, there is lots of information describing the port and the performance improvements. In particular,

"On Apple Silicon, the inference runs fully on the GPU via Metal"
...
"On Apple Silicon devices, the Encoder inference can be executed on the Apple Neural Engine (ANE) via Core ML. This can result in significant speed-up - more than x3 faster compared with CPU-only execution. "

[tmyfrn]

I did discover that the App Store app is indeed using GPU. I am still unable to get my version of open-whisper to use the GPU. I'm working on it with the help of ChatGPT, but having real problem.

[tmyfrn]

I'm pretty much tested out! I cannot get open-whisper to run with GPU on my Mac M1 Max. Been debugging and trying everything possible all day, and yesterday and a bit the day before.

I am working in a python environment.

I don't really expect anyone here to debug this for me, but if anyone happens to see this error code and knows right away what the problem is, please let me know. From my limited knowledge, the script I am using checks for the device being use, cpu or mps. It detects mps. But it can't run any model, small, base, large, turbo, with that, so crashes out or defaults back to cpu.

I mentioned before that I have an app which came from the Apple Store, which runs whisper. When I use it, the GPU usage hits the top, so it seems whisper with this Mac M1 Max can use the GPU.

python transcribe.py
Using device: mps
Traceback (most recent call last):
File "/Users/tommyfarnsworth/whisper-torch/transcribe.py", line 9, in
model = whisper.load_model("turbo").to(device)
File "/Users/tommyfarnsworth/whisper-torch/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1343, in to
return self._apply(convert)
~~~~~~~~~~~^^^^^^^^^
File "/Users/tommyfarnsworth/whisper-torch/lib/python3.13/site-packages/torch/nn/modules/module.py", line 991, in _apply
self._buffers[key] = fn(buf)
~~^^^^^
File "/Users/tommyfarnsworth/whisper-torch/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1329, in convert
return t.to(
~~~~^
device,
^^^^^^^
dtype if t.is_floating_point() or t.is_complex() else None,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
non_blocking,
^^^^^^^^^^^^^
)
^