What the setup is
Hein's hardware is an AMD Ryzen 5 mini PC with 32GB of RAM and no powerful graphics card. He says he picked Qwen2.5:7B, a 4.7GB download, because it ran acceptably on that machine. Treat this as his configuration, not a minimum requirement.
Ollama's own materials back up the basic workflow. Ollama's desktop app for macOS and Windows launched on July 30, 2025, and lets you download and chat with models. It supports file drag and drop for working with text or PDFs. That matches Hein's description. He drags a PDF into the conversation or uses the attachment button, then asks questions in plain language. He can ask follow-ups in the same conversation without re-uploading.
The model size also checks out. Ollama's library lists the Qwen2.5 7B instruct build at 4.7GB with Q4_K_M quantization. The same tag listing shows other 7B variants at very different sizes, including a 15GB FP16 build. So "Qwen2.5:7B is 4.7GB" describes a particular quantized build. It doesn't describe every 7B variant.
The privacy claim, and its boundary
The privacy argument is the point of the exercise, so it needs care. Ollama's privacy policy (last updated March 2026) says that when it runs locally, it doesn't collect, store, transmit or access prompts, responses or other content you process locally. It does say it may collect limited metadata, such as app version and request counts, and that this metadata excludes prompt and response content.
That is Ollama's own assurance, not an independent audit. It also depends on which model you use. The same policy says that with cloud-hosted models, Ollama processes your prompts and responses transiently to provide the service. Running Ollama doesn't by itself mean the processing is local. The privacy benefit applies when you're using a model downloaded to your PC.
Local inference also doesn't secure the PC itself. Documents, chat history and downloaded models still live on your machine. Disk encryption, account hygiene and normal endpoint security still matter.
What Hein tested
Hein built eight sample documents: invoices, bank statements, service contracts and product manuals. He attached them one at a time and started with simple questions before moving to harder ones.
- Contract exclusions: Qwen picked out hardware replacement, ransomware recovery and after-hours onsite work as exclusions in a maintenance contract.
- Hypothetical reasoning: Asked about a Saturday network outage, it connected several provisions. It explained that onsite support wasn't included and would need separate approval and extra charges.
- Bank statement: Asked to find recurring charges and total them, it first caught a streaming subscription and a mobile plan but missed a gym membership. After he asked it to review again, it found all three and got the total right.
- Invoices and healthcare paperwork: Hein describes using it to find billing dates, explain charges and summarize amounts and due dates. He says this could help with medical bills. He doesn't report a specific accuracy test on medical documents.
The missed gym charge is the most useful detail in the piece. A model that confidently returns an incomplete total is a real risk when the subject is your money. This isn't a flaw unique to local models. Hein's advice applies everywhere: check extracted figures against the source document.
The 24MB PDF problem
Hein hit a wall with a 24MB PDF. Nothing seemed to happen at first. A second attempt produced a "File too large" message. He opened the file in Microsoft Edge, used the print feature to split it into smaller files, and attached the sections one by one.
Ollama's announcement doesn't publish a 24MB cap, so this is his observed experience, not a documented limit. Splitting has a cost. The model sees each part separately, so questions that depend on information spread across sections may get weaker answers. Keep the sections in page order and say which part you've attached.
Context length matters for long documents
Long documents also run into the model's context window, which is a separate issue from file size. Ollama says that for large documents, you can raise the context length in the app's settings. It notes this takes more memory.
Third-party guides add some useful cautions, which are not Ollama's official statements:
- One guide says Ollama's default context was 4,096 tokens as of its v0.32 line, and that content beyond the window may be dropped without any warning. Defaults have changed across versions, so check your own install.
- The Qwen2.5 library page advertises support for up to 128K tokens. The tag list shows 32K for the 7B builds. What the model supports isn't the same as what Ollama has configured, so a long document can be silently truncated.
- One guide says a more memory-friendly approach for long files is splitting the document and combining the summaries. That is consistent with Hein's Edge workaround.
If answers about a long document seem to ignore its later pages, a too-small context window is a likely cause. It's worth checking before you blame the model.
Practical checklist for Windows users
- Install the Windows app from Ollama's site. Ollama's Windows documentation says the installer doesn't need administrator rights and installs in your home directory by default.
- Plan disk space. The same documentation says the installer needs at least 4GB. Models take more space, ranging from a few GB to hundreds of GB depending on the model.
- Choose a local model that fits your RAM. Hein used a 4.7GB quantized 7B model.
- Attach one document at a time. Start with simple lookups, then move to comparisons and hypotheticals.
- Raise the context length in the app's settings if the document is long, and keep an eye on memory use.
- Split oversized PDFs if the app rejects them. Keep the pages in order.
- Verify anything that matters, such as totals, dates and contract terms, against the original.
How far to trust it
Hein's tests used sample documents he created, not real financial or medical records. That's sensible for a test, but it means the results say little about messy scans, odd layouts or documents full of jargon. A 7B model on modest hardware is also a long way from the largest cloud models, and Hein himself says he isn't giving up ChatGPT or Claude.
For summaries, lookups and plain-English explanations of paperwork, a local model is a reasonable privacy-minded tool. For legal, medical or tax decisions, it isn't a substitute for a professional or for reading the original. The takeaway from Hein's experience is a modest one. You can get useful document help without uploading your files, if you verify the output and stay within what the hardware can handle.
References
- I stopped uploading private documents to AI, this free app handles them on my PC How-To Geek · 2026-10-25T14:31:17+00:00
- Ollama's new app · Ollama Blog ollama.com