Are there models I'm not aware of that are trained for this task? Taking audio in an unknown language, and rather than identifying the language, just transcribing the sounds to IPA? That would not be useful to most people, but it would be a Godsend to many, many linguists working with minority languages around the world.
Will definitely check this out since it sounds like it eases through the pain of dealing with these.
After seeing so many *subscription based* transcription apps all wrapping *open source models*, finding Handy was a real delight and I'm happy to see the author keep on building!
Noob question: How do you think about funding from a foundation(i have no clue if you need it or not, I do hope you have a way to get paid one way or another because handy is amazing) for maintenance of this? if you did or were going to get paid by asking for maintaining such a project what might be the kind of organizations you would look for to get supported and how would you do it?
I want to open an office document, for example, and start talking. And I want the software to continuously type what I am saying at the cursor with minimal latency. The continuous part is crucial. Many software will paste whatever I said after I have stopped recording, but that is not useful.
It's a nice reminder: You can use AI to slop cannon at maximum speed, or you can use it to scale your ambitions and build something more rigorous and lasting than ever before.
I'd build Transcribe.cpp into the apps I maintain, but I feel like this functionality should (generally) be integrated into the OS or "everywhere" via an app like Handy.
Nice. Here's the Python one: https://github.com/handy-computer/transcribe.cpp/tree/main/b... - looks like it's not yet available as a binary wheel on PyPI with the dependency included (the library on PyPI right now uses ctypes to call a separately installed library) but that's planned for a future release.
> I think as we look forward to the future, more inference will start happening locally for one reason or the other. This brings the distribution story front and center. In order to have more applications running inference locally, we need to make running inference easier.
This makes these projects so much more trustworthy and easier to approach:
> Were any of the words here written using AI? Nope. They came from my mouth or my fingers.
Certainly I am very lucky that quite a few people donate to Handy, and also some people and organizations who sponsor the work I do
To be honest I just love contributing to open source and wish to continue to do so. So anyone who supports this is good to me. Organizations which believe in OSS and push it forward are typically most aligned with me
Of course you can always email me (contact@handy.computer) and we can discuss in more detail
It really depends on how one uses transcription.
For example, I really value being able to open different windows, and look at graphs, or scroll some data while I'm dictating, because it can help me with providing some support information for what I'm saying.
Some apps can even take into account things you copy or look at as part of the transcription's context to improve the results [0].
[0]: https://superwhisper.com/docs/common-issues/context#types-of...
However the accuracy of the real time models is poor, so I did a second pass with a higher accuracy model before committing the text.
I often prefer to dictate everything in my head about a particular thing for 5–10 minutes and then go through it afterward. I find that much more useful because it doesn’t break my thought process the way continuous transcription does.
I agree this should be everywhere and I hope to distribute libtranscribe some day properly so it is more a system library! It will take time to stabilize but I think we can get there
I maintain this list of all the best open-source ones in this awesome-style GitHub repo. People looking for open-source dictation tools, hope you find something that works for you here:
I’m planning on having it as a first class feature of the app too just too many other issues to work on first
But the answer largely depends on you, the languages you speak, and personal preference. Whisper is still excellent and supported in transcribe.cpp
Cohere Transcribe is also excellent, but many of the new models are as well
I did this investigation before creating transcribe.cpp it would have been much more convenient and save me literal months of work. Happy to share the repo and binaries produced as well, but it was mostly throw away work to profile how to ship accelerated ONNX in Handy.
>> Were any of the words here written using AI? Nope. They came from my mouth or my fingers.
I have to push back on this a bit, as I believe (quite strongly) that we're shaped by the tools we use; text-to-speech LLMs are still LLMs, and generally their mistakes are shaped by the expectations inherent in their training. This, in turn, shapes the words that appear on the screen. For those who regularly use them, you then learn which word sequences are likely to be accurately transcribed, and this definitively becomes part of your thinking process. Over time, the LLM becomes tangled into your thinking; the use of AI, even in this way, very much can and often does shape the resulting words.
Once things are more or less ironed out on MacOS and Windows a lot of attention will be turned towards Linux
I know a lot of Linux PRs are open it just takes me so long to get around and test them. And often multiple different implementations trying to fix similar issues which is a lot of overhead sometimes
A while ago, I auditioned about 10 different STT apps on my Mac, with this realtime/streaming transcription as a goal. I failed to find that feature in an app I was happy with, but settled on Handy as the best option otherwise. So if Handy adds this, it will be perfect!
So in general this definitely works. Handy is just missing the feature to insert these streamed words into the app where the cursor is.
Looks like it's using IBM's Granite-Speech-4.1-2B-Plus https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plu... and/or MOSS-Transcribe-Diarize https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize
1. https://www.poetryloverspage.com/poets/tyutchev/silentium/li...
I’m not sure how many specific models will be supported as the library is more focused on transcription specifically. But the models which support diarization natively must be supported I think. And parakeet multitalker was the primary driving force for this change
In fact, it cleans up the entire paragraph that you just said, and even if you have meandering thoughts, it cleans those up too.
Actually, this above statement was fully dictated with iOS and it added all the punctuation automatically, so I think that iOS is also doing some of this natively. In fact, I’m on the iOS 27 beta and it seems to be doing an even better job of correcting itself and correcting earlier words and adding punctuation too.
It may have been an early issue but with no obvious way to interact and report the issue and, eh, Google’s general attitude around customer satisfaction, I just gave up and deleted it again.
Same with punctuation, you could determine that a comma belonged in a certain place once you have enough words.
I'm super excited to share transcribe.cpp today.
transcribe.cpp is a ggml based transcription library which supports all the latest transcription models. Every model published under the handy-computer HF org has been numerically validated and WER tested to match the reference implementation. It's accelerated everywhere.
I'm the author and maintainer of Handy. This library grew from the pains of distributing a cross-platform speech-to-text application to many people.
This is a v0.1.0 library which means that there are some rough edges which I cannot discover alone! Please report them, and let's fix them together!
Let me say this. I think distributing a cross-platform application with the current ASR inference stack is terrible.
You've basically got whisper.cpp and ONNX. That's it. You could roll MLX in for Apple devices, but now you've to support two different engines and port models to each. I've been a fan of ONNX for getting model support into Handy quickly, but so much performance is left on the table with CPU only.
There are a few random libraries out there which claim to support a lot of models, but they have unknown authors, and unknown testing, as far as I've seen. They leave me with more questions than answers.
When will they stop maintaining this library? Has the creator thought about bindings so you can actually use it in a real desktop or mobile app? Is this effectively demo code? Have they benchmarked it? Is it faster than ONNX?
And this is what led to transcribe.cpp. As Handy's maintainer I needed a library I could trust. Where I could download a file and run inference on it. Where I can know that the inference coming from the model in the engine is as good as the reference implementation. The inference should run on the GPU for the best performance. It should be trivially embeddable in Handy, it cannot be a huge pytorch lib. It must be something that works on Mac, Windows, and Linux. And ggml seemed like by far the best way forward. It has a strong community, and a great distribution story.
You get a fast and accurate inference engine with wide ranging model support.
We intend to support as many state-of-the-art transcription models as possible. As of today, we support most of the modern transcription models that are publicly available. There are a few missing still, but they will be added soon.
One of my top goals was to run any ASR model I wanted on Vulkan. In my opinion this is the floor for any application shipping local inference. For every model we support, there is a corresponding benchmark run from a Ryzen 4750U (CPU + Vulkan) on Fedora as well as on my M4 Max.
I also wanted to make sure that inference in transcribe.cpp is accurate and as close to the reference implementation as possible. This largely came from a huge degree of uncertainty of inference accuracy when using .onnx models I found on Hugging Face. In order to ensure the inference we do is correct we numerically validate every model versus the reference. On top of numerical validation, we run full WER sweeps to make sure that whatever the reference is outputting, we output the same thing. That means every model has run through thousands of utterances and is very close or same as the reference. And the results of this data are published in the transcribe.cpp repo as well as with each model on Hugging Face.
transcribe.cpp is more or less a drop in support for whisper.cpp. The main reason for this is: Handy used whisper.cpp and I needed to ship an update with transcribe.cpp which would replace it. I needed to keep some compatibility with the very popular .bin files which run in whisper.cpp and shipped with Handy. transcribe.cpp can run them. There are some flags and features in whisper.cpp which we do not support yet. But I think for the vast majority of use cases our whisper implementation is solid and can replace whisper.cpp while having about equal performance.
Language bindings were on my mind to begin with. While this library is written in C/C++, I needed bindings in Rust. And I also knew that in order for us to distribute local transcription as widely as possible, it requires at minimum decent first-party support of bindings. I've chosen 4 languages that I think are fairly representative of where people will use the library. I welcome others to contribute bindings directly to the project as well, assuming that they are willing to take on the maintenance burden of doing so.
And of course, at the end of the day, a lot of the decisions were driven by Handy. As a result of Handy being popular, I intend to maintain this library, just as I've done my best to maintain Handy. I intend to be someone who continues to maintain open source projects and contribute to the ecosystem where I can.
This library never would have existed without Handy because I wouldn't have had the problem of trying to support a bunch of different ASR models. I would have never learned all the use cases that people have for ASR. I've done my best to cover the ones that I hear about the most. Certainly, there are cases in the library that are not currently handled. If there are things that I missed, you are free to contribute to the library!
transcribe.cpp is aimed squarely at making locally run ASR easier. We know that transcription can run extremely accurately on most devices, and there should be no need to send your voice to a cloud service. An RK3566 can run models via transcribe.cpp faster than real time on its anemic CPU. Faster than real time transcription with SOTA models runs in a handful of watts. It's not a hope or a dream, it's a fact.
I think as we look forward to the future, more inference will start happening locally for one reason or the other. This brings the distribution story front and center. In order to have more applications running inference locally, we need to make running inference easier. Certainly transcribe.cpp does not solve this on the whole, and there is a long way to go, but I hope it's a small step forward. I've certainly learned a lot.
I am extremely thankful for all the folks who have supported this project.
First and foremost is to Mozilla AI, their BiR program, and Davide from Mozilla AI. This project was largely a problem in my head that I came to them with, and they decided to support me in solving the problem. At the time transcribe.cpp wasn't even a concrete idea, I was just exploring how to solve accelerated distribution in Handy. So a huge thanks to them, their support, and helping to bring this project into existence.
ggml. This project wouldn't be possible without ggml and all of the contributors to it. Thank you all so much for the work you've done. I think ggml really does amazing work in helping to make distributing local inference applications easy and possible.
Modal has also been a critical help for me. I reached out to them, and they gave me credits. These credits are put towards doing the WER testing and ensuring the library works well on CUDA. It is an immense help being able to verify the correctness of the work.
Blacksmith helps to power some of the CI/CD for transcribe.cpp. Again I reached out to them and they immediately responded with credits. Of course CI/CD is critical for making sure everything put out has been tested to at least some degree.
Hugging Face both for being a pillar in the local AI community, as well as providing the handy-computer org private storage, so I could upload models at my own will.
Yes absolutely. I don't think it's possible for a single individual to write an engine from scratch of this size using ggml in a handful of months without outside assistance. Were any of the words here written using AI? Nope. They came from my mouth or my fingers.