Small Local LLMs Fail at CSV Math in llama.cpp: Why Windows Users Need Spreadsheet Tools

A practical hands-on test published by Abhinav Raj at XDA-Developers demonstrates exactly why treating a raw language model like a spreadsheet engine is a dangerous habit. When four popular small local models—including Microsoft's Phi-4 Mini, Google's Gemma 4 E4B, and Mistral's Ministral 3 3B and 8B—were handed the same deliberately cluttered hardware-price CSV and asked six targeted questions with tool use disabled, every single model stumbled on basic filtering and arithmetic. Most strikingly, across fifteen total test runs, not a single model managed to identify the cheapest 16GB graphics card sitting in plain sight in the table.

The Anatomy of a Messy Spreadsheet Test​

Real-world spreadsheets rarely look like sanitized benchmark datasets. When PC builders or IT buyers put together a quick price-tracking sheet, it tends to accumulate formatting quirks over time. To simulate this, Raj constructed a deliberately messy hardware price tracker in CSV format modeled on a personal PC part-shopping sheet. The file included several classic data-hygiene traps:

• Mixed storage units: Drives listed as both 1000GB and 1TB.
• Varying regional currencies: Prices listed in both US dollars ($) and British pounds sterling (£).
• Incomplete and non-product rows: Placeholder cells marked TBD or N/A, duplicate-looking entries, and summary subtotal rows mixed into the sheet.
• Multiple price columns: Both manufacturer suggested retail prices (MSRP) and actual street prices.

To see whether small local models could parse this table without external assistance, all four models were run locally via llama.cpp on a desktop equipped with an NVIDIA GeForce RTX 4070 Ti Super (16GB VRAM), an AMD Ryzen 5 7600X processor, and 32GB of DDR5-6000 RAM. Every model was quantized to Q4_K_M, given a 16K context window, and fully offloaded to the GPU.

Crucially, the test evaluated pure language-model token prediction rather than agentic tool use. Web search, external code execution tools, and extended reasoning modes were switched off. Each question was pasted into a fresh chat session in llama.cpp's built-in web interface alongside the plain-text CSV payload.

Instead of forcing one arbitrary temperature setting across all four models, the test followed each model card's recommended Hugging Face sampler parameters, setting min-p to 0 across the board. Because Google's Gemma 4 E4B was configured with a higher temperature (1.0, top-k 64, top-p 0.95), it was given three runs per question. The two Ministral models (temperature 0.05, top-k 1) and Microsoft's Phi-4 Mini (temperature 0, top-k 1) were tested over two runs per question.

Section Summary: The benchmark tested four Q4_K_M quantized models in llama.cpp with no Python interpreter or calculator tools enabled, asking four data-extraction questions and two out-of-scope grounding traps against a single noisy CSV.

Model-by-Model Breakdown: Where the Math Went Sideways​

When evaluating the responses, Raj applied a fair grading rubric: 1000GB counted as 1TB, subtotal rows and TBD/N/A entries had to be ignored, and pound sterling prices did not count as valid USD figures. If a model identified the right product row for a price lookup but accidentally swapped the currency symbol in its output, it still received credit for finding the product. On the two unanswerable questions, however, only an outright refusal passed; admitting the data was missing and then guessing anyway was scored as a failure.

Here is how the four models actually performed:

1. Gemma 4 E4B: The Most Faithful Reader, Still Tripped by Arithmetic​

Gemma 4 E4B tied for the highest overall score at 67% and came closest to reading the CSV accurately. Across all three runs, it recognized that 1000GB meant 1TB and properly selected the SK Hynix P41 at $95 as the cheapest 1TB SSD. It also correctly counted two unique 2TB SSDs on two of its three runs (answering 3 on the first run) and cleanly refused both absent-data trap questions every time.

When asked to average the valid USD street prices across all GPUs, Gemma actually selected the right seven graphics cards. Yet on two of its three runs, it made a $10 addition error when summing those seven numbers, returning $537.57 twice before finally hitting the exact $539.00 figure on its third run. And on the 16GB GPU question, it missed the $369 RX 9060 XT 16GB completely, returning the RTX 5060 Ti at £450 twice and at $459 once.

2. Phi-4 Mini: Greedy Decoding Drift and Truncated Scanning​

Microsoft's 3.8-billion-parameter Phi-4 Mini tied Gemma at 67% overall after normalizing for run count. It cleanly refused both out-of-scope questions, got the unique 2TB SSD count right on both runs, and located the correct row for the SK Hynix P41 SSD on both runs (though it transposed the currency sign to £95 instead of $95).

However, Phi-4 Mini exhibited a fascinating technical quirk on the GPU average question. Even with temperature set to 0 and top-k set to 1—a greedy decoding configuration that many users assume guarantees identical outputs—its two runs returned wildly different wrong averages: $584.33 on Run 1 and $403 on Run 2. Because parallel floating-point operations on a GPU are not strictly bit-deterministic unless specific deterministic kernels are forced, tiny rounding differences can flip a token choice when two probabilities are nearly tied. Once a single token diverges in an arithmetic chain, the rest of the calculation cascades into a completely different set of numbers. In Phi-4 Mini's case, it also blended MSRP figures with street prices and stopped reading after the NVIDIA rows, ignoring the AMD and Intel cards further down the table.

3. Ministral 3 3B: Right Rows, Wrong Sums​

The smaller 3.4B Ministral model showed disciplined grounding, refusing to hallucinate answers for the Q2 2022 pricing and RTX 5070 Ti release date questions, and properly counting two unique 2TB SSDs. Remarkably, when asked for the GPU street-price average, it identified the exact seven valid USD cards on both runs—and then botched the arithmetic, outputting $466.14 and $480.43 instead of $539. On both "cheapest component" questions, it failed to filter out British pounds, naming the RTX 5060 Ti and the Crucial T700 based on sterling figures.

4. Ministral 3 8B: Why Scaling Up Without Tools Can Backfire​

To see whether jumping from a ~3B–4B class model to an 8B model would solve these failures, Raj included Ministral 3 8B, the larger sibling in the same model family. Scaling up did fix the SSD unit-conversion problem: unlike the 3B version, the 8B model equated 1000GB with 1TB and properly picked the SK Hynix P41 at $95 on both runs.

Yet in two other areas, the 8B model performed worse than its smaller peers. On the GPU average, it skipped three valid cards (the RTX 5060 Ti at $459, the RTX 5080 at $1,149, and the RX 9070 XT at $629), averaging only four cards to output $384 on both runs. Worse, its grounding discipline broke down on the absent Q2 2022 GPU price question. On Run 1, it invented a GBP-to-USD exchange rate and calculated a "hypothetical" $575 average. On Run 2, it initially declined the question, but then kept generating text and fabricated a 2022 pricing history for NVIDIA's RTX 30-series anyway.

Section Summary: While Gemma 4 E4B and Ministral 3 3B proved that small models can sometimes isolate the exact right seven rows from a messy CSV, neither could reliably add and divide those seven numbers in plain text—and stepping up to an 8B model actually introduced confident hallucinations on missing data.

Why Did Every Model Miss the $369 RX 9060 XT 16GB?​

The headline anomaly of the entire experiment is that across fifteen separate generations spanning four distinct model architectures, every single run answered RTX 5060 Ti (either at £450 or $459) when asked for the cheapest 16GB graphics card, completely overlooking the AMD Radeon RX 9060 XT 16GB at $369.

Based on general industry knowledge of how transformer models process tabular text—combined with the specific failure modes recorded in Raj's test—three overlapping mechanisms explain why small local LLMs struggle so badly with this task:

1. Multi-Constraint Filtering Overwhelms Attention Over Flat CSV Text: When a human looks at a spreadsheet, our eyes scan vertically down a single column (Street Price USD) while keeping a finger on VRAM = 16GB. When a language model ingests a raw CSV file, it sees a flat stream of comma-separated tokens row by row. Answering "What is the cheapest 16GB graphics card, and its street price?" requires joining at least four constraints simultaneously on every row: it must be a GPU (not an SSD or subtotal), it must have 16GB VRAM, the price must come from the street-price column rather than the MSRP column, and the currency must be USD ($) rather than GBP (£). Phi-4 Mini explicitly demonstrated attention drop-off by stopping after the NVIDIA rows and dropping the AMD and Intel entries below them.
2. Token Prediction Is Not an Arithmetic Logic Unit (ALU): Even when Gemma 4 E4B and Ministral 3 3B isolated the exact seven valid GPU prices, they failed to compute the average ($539) in four out of five combined attempts. Next-token prediction estimates statistically probable sequences of digits; without a scratchpad or an external Python interpreter, carrying remainders and dividing multi-digit sums in a single forward pass is notoriously brittle, especially at 4-bit (Q4_K_M) quantization.
3. The "Helpful Confabulation" Trap in Larger Models: Why did the 8.4B Ministral model invent a 2022 RTX 30-series price history while the 3.4B model cleanly refused? Larger models retain richer world knowledge in their weights and are heavily instruction-tuned to be helpful. When an 8B model sees a prompt asking for Q2 2022 GPU prices, its internal weights contain strong associations with historical RTX 30-series pricing. Without strict system-prompt guardrails forcing it to rely exclusively on the attached context, a slightly more capable model is often more tempted to bridge gaps in the document using its own pre-training memory or improvised math.

Practical Guide: How Windows Users Should Actually Query Messy Spreadsheets Locally​

It is important to keep the scope of this test in perspective: it examined four small models at a single quantization level (Q4_K_M) on one intentionally booby-trapped spreadsheet with reasoning and external tools turned off. Still, the lesson for Windows power users, system administrators, and budget-conscious PC builders is crystal clear. Never ask a raw, tool-free language model to do your spreadsheet math.

If you run local models on Windows using llama.cpp, LM Studio, Ollama, or Open WebUI, follow this workflow to get accurate answers from messy CSV files:

1. Use the LLM to Write the Query, Not to Calculate the Answer: Instead of pasting a 200-row CSV into the chat window and asking "What is the average USD street price?", paste only the column headers and two sample rows. Ask your local Phi-4 Mini, Gemma, or Ministral model to write a PowerShell script, a Python pandas snippet, or an Excel AVERAGEIFS / MINIFS formula that filters out £, TBD, and subtotal rows. Even a 3B model that cannot add seven numbers in its head can usually write a working three-line Python or PowerShell filter, letting your CPU's actual arithmetic logic unit do the math with 100% precision.
2. Enable Code Interpreter / Tool Calling for Data Tasks: The XDA test deliberately disabled tools and reasoning to test baseline model behavior. In daily production use on a Windows PC, turning on tool use (such as a local Python sandbox in Open WebUI or MCP tool servers) transforms how small models handle CSVs. Instead of guessing the sum, the model writes a quick script to parse the CSV and reads back the deterministic output.
3. Normalize Currencies and Units Before Ingestion: If you must feed raw tables directly into a local model's context window, spend thirty seconds in Excel or Power Query first. Standardize 1000GB to 1TB, split currency symbols ($ vs £) into a dedicated Currency column, and strip out Subtotal rows. Removing multi-hop normalization traps dramatically reduces attention errors.
4. Enforce Strict Refusal Rules in Your System Prompt: To prevent the kind of helpful fabrication seen with Ministral 3 8B—where the model declined a question and then immediately appended a made-up price history anyway —add a rigid boundary to your llama.cpp system prompt: "Answer using ONLY the provided CSV data. If the exact figure or date is not explicitly present in the rows, reply with 'Not in the sheet' and stop generating immediately. Do not estimate, convert currencies, or use external historical knowledge."

Small local models running on a mid-range GeForce card are phenomenal tools for summarizing text, drafting scripts, and classifying notes offline. Just remember that when it comes time to balance a hardware budget or find the lowest street price on a graphics card, a classic spreadsheet formula still beats a neural network guessing numbers in the dark.