We gave Claude a 2,000-row sales export with 10 planted problems, and all 4 runs found all 10. The one real mistake came from the prompt: with a one-line request, both runs put a 400-unit typo into the headline total, a row worth €11,600 that made revenue 0.6% too high, and flagged it only in a note after the tables. A prompt that sets the rules and asks for typos to be shown separately fixed it in both models.
What was wrong with the file
The file covers January to September 2026: 1,950 real orders, 12 test orders and 38 exact duplicate rows, for 2,000 rows in total. We planted the kind of mess a real CRM or shop export produces. A correct answer had to fix all of it before adding anything up.
| Planted problem | Rows affected | Found by |
|---|---|---|
| Three date formats (2026-04-16, 16/02/2026, 24 Aug 2026) | 1,003 not in ISO format | 4 of 4 runs |
| Region spelling variants ("NORTH", "Nrth", "Wset", trailing spaces) | 220 | 4 of 4 runs |
| Units stored as text ("3 pcs") | 97 | 4 of 4 runs |
| Prices with a decimal comma ("140,22") | 126 | 4 of 4 runs |
| Blank unit price | 60 | 4 of 4 runs |
| Discount written as "10%" instead of 0.1 | 172 | 4 of 4 runs |
| Status in mixed capitals ("COMPLETED", "completed") | 115 | 4 of 4 runs |
| Test orders ("TEST ACCOUNT") | 12 | 4 of 4 runs |
| Exact duplicate rows | 38 | 4 of 4 runs |
| One typo: 400 units where every other order is 25 or fewer | 1 | 4 of 4 runs |
Two more things are business rules that a reader has to know. 183 rows are priced in US dollars, and cancelled and refunded orders are not revenue. Both one-line runs excluded the 164 cancelled and 62 refunded orders without being told; the rules prompt spelled this out.
Where the one-line prompt went wrong
The one-line prompt read: "Can you clean it up and tell me the total revenue in euros, revenue by region, our top 5 products by revenue and our best month?" Both models cleaned the file correctly and got the region order, the top 5 products and the best month (March 2026) right. Neither invented a number: every figure reconciles with our answer key once you allow for the typo and their slightly different exchange rate.
The problem was one decision. Order SO-100092 lists 400 desk mats at €29, a row worth €11,600. Both runs spotted that it was out of line, said it might be a typo, and then left it in the headline ("I kept it", "I left it in"). Against our answer key, where the order is 4 units, their totals were about €11,500 too high. The warning came in a closing note after the tables, which is exactly the part a busy reader skips on the way to the board slide.
| Run | Headline total | Typo in the headline? | Time |
|---|---|---|---|
| Opus 5.5, one-line prompt | €1,808,507.23 | Yes, flagged at the end | 60 s |
| Sonnet 5.5, one-line prompt | €1,808,507 | Yes, flagged at the end | 43 s |
| Opus 5.5, rules prompt | €1,796,912.90 | No, shown with and without | 85 s |
| Sonnet 5.5, rules prompt | €1,796,912.90 | No, shown with and without | 72 s |
The one-line prompt also gave no exchange rate. Both runs worked one out from the file (1.087 dollars per euro, which matches our 0.92 euros per dollar), said so, and asked for confirmation. That is good behaviour, but it means the number depends on a guess unless you check that line.
The prompt that fixed it
The rules prompt asks the same question, says the numbers are for a board report, and adds four rules and three checks. It states the definition of revenue, the exchange rate and what to exclude, and it asks for the checking work to be shown before the answer. These are its rules and checks, ready to copy:
- Revenue = units x unit price x (1 - discount). Report in euros. Convert USD rows at 1 USD = 0.92 EUR.
- Count only completed orders. Leave out cancelled and refunded orders, test orders and exact duplicate rows.
- Dates written with slashes are day/month/year.
- Before you calculate anything, list every data problem you find, with how many rows it affects and what you did about it.
- Flag any value that looks like a typo instead of silently using or deleting it, and show me the totals with and without it.
- End with a check: rows in, rows removed by reason, rows used.
With these lines, both models kept the typo out of the headline, showed a with-and-without table and asked the user to confirm the order. Both also labelled the 60 filled-in prices as an assumption and gave the total without them (€1,749,412.44). Both ended with a row check that adds up: 2,000 rows in, 276 removed, 1,724 used. The Opus run went one step further and gave the total if the order should be 4 units, €1,797,028.90, which is exactly our answer key. The rules runs took 85 and 72 seconds, against 60 and 43 for the one-line runs.
The date rule matters more than it looks. 262 of the 743 slash dates in the file, such as 05/03/2026, could be read either way. Both one-line runs read them day first, and the Opus run gave its reason: many other slash dates start with a number above 12. That was right for this file, but it is still a guess.
What this test does not show
Each combination ran once, so a second run could differ. The rules prompt changed several things at once, including the line saying the numbers were for a board report, so we cannot say which single line made the difference. The file is synthetic, built to be messy in known ways, and real exports can fail in ways we did not plant, such as columns that change meaning halfway through. We tested Claude in Claude Code. The claude.ai upload screen uses the same models with code execution, but it is a different surface and we did not test it. Claude's pricing page lists Sonnet and Haiku on the Free plan and Opus only on Pro and above, so a Free user can choose Sonnet but not Opus.
We think the result is still the most useful finding a team can act on this week: on a file like this, the brief decided the headline more than the model did. A model that finds every problem can still hand you a headline you should not use, if you never told it what to do with doubtful values.
If you run sales, finance or marketing
- Sales operations: paste the rules lines above into your monthly pipeline or revenue prompt, with your own definition of a closed deal.
- Finance leads: always ask for the row check and the with-and-without table. If the rows do not add up to the file, do not use the total.
- Marketing: for campaign exports, write your attribution and exclusion rules into the prompt. A test lead or a duplicate click changes the result the same way a test order does.
- Founders: Sonnet 5.5 on the rules prompt gave exactly the same headline as Opus 5.5 in this test. Here, the prompt mattered more than the choice of model.
How to run this on your own export this week
Code execution and file creation are available on every Claude plan, including Free, on the web, desktop and mobile apps, and you can upload CSV files up to 30MB each. Check your company's policy before you upload real customer data; our guide to using AI at work safely covers what to ask.
- Write your definitions first: what counts as revenue, which orders to exclude, which currency and rate, and how dates are written.
- Ask for the problem list before the numbers, and for doubtful values to be shown with and without.
- Read the row check. Rows in minus rows removed must equal rows used.
- Spot-check two or three figures against your source system before anything goes in a report.
For more on the tool itself, see our Claude guide. For the wider method, read how to analyse data with AI, how to use AI for Excel and how to reduce AI hallucinations.