What the author built
Singh says he didn't need faster dictation. He wanted something that could take his thinking-out-loud ramble and pull out the things he actually had to do.
The flow he describes has two stages:
- Whisper transcribes a recorded voice note locally.
- Qwen, running through llama.cpp, reads that transcript. It works out which tasks he meant and returns them as structured data.
His example note was "order snacks for the weekend party, thoroughly clean the house, and clean the air fryer". It came back as three separate items, each with a time attached and the list sorted chronologically.
Hardware and model choices (the author's own setup)
Singh reports using Qwen 3.5-9B with Q4_K_M quantization on a 12GB GPU. He says Whisper plus the model use about 7GB of VRAM on his RTX 4070 Ti. Treat that as one data point from one machine, not a requirement.
The model itself checks out. Qwen's repository news lists Qwen3.5-9B among the models released on 2 March 2026, alongside the 4B, 2B and 0.8B sizes. The same repository says llama.cpp supports Qwen3.5. That confirms the model exists and runs in this runtime. It doesn't confirm Singh's quantization choice or his VRAM figure.
Why the date logic lives in code
The most transferable design decision is that the LLM doesn't do calendar arithmetic. Singh says the model extracts the task and the intended timing, and then ordinary code computes the actual dates.
This is a sensible split. A language model is good at spotting that "tomorrow" or "this weekend" appears in a sentence. Resolving that phrase against today's date, then sorting, is a job for deterministic code.
The account doesn't say how his code handles time zones, ambiguous deadlines or locale. This split narrows where errors can occur, but it doesn't remove ambiguity. "Sometime next week" still needs a rule.
Structured output: a likely fit, not something he reports using
Singh doesn't say how he gets structured data out of the model. llama.cpp does have a mechanism for this, though. Its documentation says the server accepts a json_schema field for grammar-based sampling, and the chat endpoint's response_format parameter supports schema-constrained JSON.
The llama.cpp grammar guide adds a caveat. The schema only constrains the output and isn't injected into the prompt, so the model can't see it. If you want the model to understand the expected structure, describe it in the prompt.
There are also known rough edges. One GitHub issue reports that, in a March 2026 build, a schema using $ref and $defs made the grammar parser fail silently and fall back to unconstrained JSON. For anyone building something similar, the lesson is to validate the model's output in code and not trust it blindly. Whether Singh used constrained generation is unknown.
From three windows to one app
Singh's first workflow was clumsy. It meant a Windows sound recorder, a terminal running the llama server, and a script pointed at the saved clip. He compares it to a car with no seats.
He fixed that by wrapping everything in a single Python file with a desktop GUI. The app:
- starts the model server and connects Whisper
- records audio and transcribes it
- converts the transcript into tasks
- shows the tasks in batches of six, like a draggable sticky note
He says Claude Code helped him iterate on the code. That included adding input-device selection and a toggle that unloads both models to free the GPU. Pressing record again restarts the server and models.
Why Whisper ran on the CPU
Singh says faster-whisper couldn't find his audio libraries or his NVIDIA math libraries, however many times he downloaded them. He moved transcription to the CPU. For 20–30-second notes, he says it finishes in seconds, so he didn't try again on the GPU.
The faster-whisper documentation makes this plausible. It says GPU execution requires cuBLAS for CUDA 12 and cuDNN 9 for CUDA 12. It also notes that recent ctranslate2 versions only support CUDA 12 with cuDNN 9, and describes workarounds for older CUDA combinations. A mismatch between those pieces is a common reason for this kind of failure. Singh doesn't say which one bit him.
The project also documents CPU inference with int8 quantization. For short voice notes, falling back to the CPU is a reasonable call, and it leaves the GPU free for the LLM.
Local is not the same as audited
Singh's reason for keeping it local is simple: his thoughts shouldn't leave his PC just to become a to-do list. He says there's no cloud API in the middle and no subscription.
That is a design intent, not a security audit. His account doesn't cover:
- where audio or transcripts are stored, or whether they're ever deleted
- whether any dependency makes network calls
- encryption or access controls
Anyone copying the idea should check those points themselves. One thing is worth knowing: the faster-whisper docs say that loading a model by size name automatically downloads the matching CTranslate2 model from the Hugging Face Hub. So the first run may involve a download even if later runs don't.
Limitations the author admits
- Names are sometimes misheard or mangled.
- There is a short warm-up pause when the GPU models are brought back online.
A mangled name is a transcription failure. A wrong task is an extraction failure. Splitting the pipeline helps you tell them apart, but it doesn't prevent either. Singh calls the trade-offs acceptable. He also says single-model audio systems may eventually simplify the setup, which is a prediction and not something he demonstrates.
Takeaways for Windows users
- Split the jobs. Use a speech model for audio, an LLM for meaning, and plain code for dates and sorting.
- Package it. Singh says the workflow only stuck once it was one app. Three windows meant he wouldn't use it past day one.
- Don't assume the GPU is the answer for everything. CPU transcription of short clips can be good enough and avoids CUDA library headaches.
- Validate the output. Consider constrained JSON output and code-side checks, since model output can drift from the schema.
- Be precise about "local". No cloud API is not the same as a verified private system.
The account has no code, repository, prompt, schema or dependency versions, so it can't be reproduced as written. It works best as a design pattern, and it shows a local 9B-class model doing a useful job on a mid-range gaming GPU.