AI for Financial Report Analysis: Verification

Ask a language model for a company's revenue and it will give you a number. That is the problem. The number arrives with the same fluent confidence whether it is this year or last year, whether the header said "in thousands" or "in millions," whether it is the GAAP figure or the adjusted one management prefers. A wrong number does not look wrong. It looks like a number.

The workflow that fixes this is the value of this page: never let the model be the source of a figure. Let it find, cite and summarize. Let arithmetic and a human confirm.

One boundary, and it holds throughout. This is a document-processing guide, not investment advice. Reading a 10-K is document work — locating disclosures, extracting figures, summarizing management's language. Nothing here recommends buying, selling or holding anything, and no workflow on this page produces a valuation opinion.

The Challenge

An annual report is not prose with tables in it. It is a structured legal artifact whose meaning is carried as much by layout as by sentences — and a language model receives none of that structure. It receives a string.

What actually goes wrong, none of it exotic:

• The wrong column. An income statement prints two or three fiscal years side by side. The model returns the prior year for the current year. Both numbers are real, both are in the document, and the error is invisible unless you already knew the answer.
• The wrong row. Tables nest: line items indent under subtotals, merged headers span columns. Flattened to text, "Total current assets" and the line above it become adjacent strings with no hierarchy between them.
• The units. A statement headed "(in thousands)" prints 4,737,251 and means $4.74 billion. A model that drops the header reports $4.7 million: a 1000x error, stated with total confidence.
• The subtotal folded into the total. Ask for a sum of segments and the model adds the subtotal row it already contains.
• GAAP mixed with non-GAAP. "Adjusted EBITDA" and "net income" sit paragraphs apart. Only one is audited. A model asked for "earnings" silently chooses.
• Restated versus as-originally-reported. The same fiscal year appears in three consecutive 10-Ks, and it is not always the same number.

Why this happens

A financial table's meaning lives in two dimensions. A cell means "FY2025 revenue" only because of the column above it and the row beside it. The units apply because of a header three rows up. The indentation says this line rolls into that one.

The text that reaches the model is one-dimensional. PDF extraction flattens the grid into a sequence of tokens: column boundaries become whitespace, merged headers collapse, and indentation — which was the hierarchy — becomes nothing at all. This is a hard, decades-old problem in data processing, not a defect of any particular model.

The model then reasons flawlessly over a mangled representation, returning a confident assertion built on a corrupted premise. It is hallucination in its most expensive costume, because the number is usually almost right — and it is why "use a better model" does not fix it.

What the research says

FinanceBench (Islam et al., 2023) tested 16 model configurations — GPT-4-Turbo, Llama 2, Claude 2, with vector stores and long-context prompts — on a 150-case sample, manually reviewing 2,400 answers. GPT-4-Turbo with a retrieval system incorrectly answered or refused to answer 81% of questions. All models examined, the paper concluded, "exhibit weaknesses, such as hallucinations, that limit their suitability for use by enterprises."

DocFinQA made the point about length: it put 7,437 FinQA questions back into their full source filings, pushing average context from under 700 words to 123,453 words. The paper's own summary of what that did to a frontier model is blunt — GPT-4's error rate roughly doubled. (For a model the paper reports end to end: GPT-3.5 scores about 67% when handed the gold context and 36–43% when made to find it inside the full document.) Retrieval did not rescue it either: a fine-tuned ColBERT retriever hit the right chunk first only 35% of the time. Finding the right page is most of the job; see information retrieval.

FinQA established that this is expert work, not a reading test: financial experts scored 91.16%, non-expert crowd workers 50.68%.

These are 2023–2024 evaluations and models have improved since. What has not changed is the shape of the failure: silent, plausible, numeric.

How AI Solves It

The most useful distinction on this page is between two operations that look similar and carry opposite risk.

Finding and summarizing is strong. Computing and asserting is dangerous.

Finding is recall-shaped. "Where is the lease-commitment schedule?" The model says page 87; you open page 87. The answer verifies itself in three seconds, and being wrong costs a page turn.

Asserting is generative. "What was total revenue?" The model says $4.1 billion. You cannot check that without going back to the document — which is what you were trying to avoid. Being wrong costs you a number in a model, a memo, or a client deck, and it propagates silently.

Everything below follows: use the model to locate, not to compute; force provenance on every figure; verify with arithmetic, not with the model. And before any of it — check whether you need to parse the document at all.

Do not parse the PDF if a structured version exists

Most articles on this topic never mention this, and it is the first thing a competent data engineer would say.

SEC filings are already machine-readable. Since the Inline XBRL mandate completed its phase-in — fiscal periods ending on or after June 15, 2021 for all remaining operating-company filers — the financial statements in every 10-K and 10-Q are tagged fact by fact. The SEC publishes those facts free at data.sec.gov: no API key, just a declared User-Agent header and a 10-request-per-second limit.

An actual response fragment from the companyconcept endpoint for Roku's revenue, fetched July 2026:

{
  "start": "2025-01-01",
  "end":   "2025-12-31",
  "val":   4737251000,
  "fy":    2025,
  "fp":    "FY",
  "form":  "10-K",
  "accn":  "0001628280-26-008114",
  "filed": "2026-02-13"
}

The value in dollars, unscaled — no "(in thousands)" header to miss. The exact period. The form. The accession number, a permanent pointer back to the filing. No column to misread, no unit to misinterpret, no subtotal to confuse with a total.

That is not an extraction. That is the number. No model touched it, so no model can have been wrong about it. The rule: if the figure exists as an XBRL fact, take the XBRL fact. Use AI for what is not tagged — narrative, footnotes, risk factors, segment commentary — which is where AI is good anyway.

Recommended Tools & Models

Start with the data source, not the model. In descending order of trustworthiness:

The bottom row is where every generic guide starts. It should be your last resort.

Any current frontier model reads a filing competently; the prompt matters far more than the choice.

Claude, ChatGPT and Google Gemini all read uploaded filings. NotebookLM earns a mention for one reason: it cites the passage each claim came from — the exact behaviour this workflow exists to enforce. Perplexity AI helps find which filing to read; n8n wires the pipeline together. For documents that cannot leave your machine, Ollama and LM Studio run open models locally — at a real accuracy cost, on the task where accuracy matters most.

Step-by-Step Implementation

1. Ask whether this number needs a model at all

• In the audited statements of a US-listed company, period ending mid-2021 or later? It is an XBRL fact. Fetch it. Parse nothing.
• In the narrative — MD&A, risk factors, footnotes? A job for a model, with citations.
• A non-GAAP measure, a private company, an old filing? You are parsing a document, and the rest of this guide applies in full.

Fetching a fact is one HTTP request with a User-Agent header identifying you:

https://data.sec.gov/api/xbrl/companyconcept/CIK0001428439/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json

Two caveats. Companies use extension tags — custom concepts outside the standard us-gaap taxonomy — so a missing concept does not mean a missing disclosure. And facts are versioned by filing: the same fiscal year appears in several 10-Ks, and differing values mean a restatement, which is information rather than an error.

2. Extract with forced provenance

The prompt has one job: make it impossible for the model to hand you a number without also handing you the means to check it.

{
  "item": "Total net revenue",
  "value": 4737251,
  "units_as_printed": "thousands",
  "value_in_dollars": 4737251000,
  "fiscal_period": "FY2025 (ended 2025-12-31)",
  "column_header_verbatim": "2025",
  "row_label_verbatim": "Total net revenue",
  "statement": "Consolidated Statements of Operations",
  "page": 54,
  "quoted_line": "Total net revenue $ 4,737,251 $ 4,112,898",
  "basis": "GAAP",
  "status": "found"
}

Three things there do the work. The refusal clause: models produce numbers because they are asked for numbers — give the model an honourable exit and it takes it. column_header_verbatim: to write down which column it read, the model has to look at the column. quoted_line with its neighbours: verification becomes "does this string appear in the source" rather than "re-read the filing."

3. Run the deterministic checks

This step involves no AI at all, and it is what makes the workflow defensible.

Cross-foot. Do the components add to the stated total? From Roku's FY2025 10-K: Platform $4,144,886 plus Devices $592,365 equals $4,737,251 — exactly the printed total. If your extracted segments do not sum to your extracted total, one of the three came from the wrong row, column or year. You do not yet need to know which; you need to know you cannot use them.

Tie out to the primary statement. A figure quoted in MD&A must equal the same figure in the audited statement. If they disagree, one of them is non-GAAP.

Units sanity check. A mid-cap streaming company has neither $4.7 million nor $4.7 trillion in annual revenue. One order-of-magnitude assertion against the prior year catches the 1000x error every time.

Prior-period delta. Pull last year from XBRL (free) and compute the change. A 0.0% change usually means the model read the same column twice; a 900% change usually means it read a different line item.

Run these as code, not as a prompt. A model asked to audit its own arithmetic tends to agree with itself — you have then spent tokens buying a second opinion from the same witness.

4. Run an adversarial verification pass

A second model pass is genuinely useful, but only framed as an audit against the source, and only in a fresh context.

Expect false alarms, and budget for them. FinVerBench, a 2026 preprint, tested 14 model runs on detecting injected errors in statements built from 10-K XBRL data and found that calibration, not detection, is the binding problem: nine of the fourteen runs flagged 95–100% of the clean, error-free statements as erroneous. GPT-4.1 posted a 95.3% false-positive rate — 41 of 43 clean statements misclassified. The cause was not bad arithmetic. The paper is precise about it: 95.1% of those false positives "flag real but benign numerical discrepancies — the model's arithmetic is correct, but it misinterprets missing line items as errors." It mistook a simplified presentation for an inconsistent one.

That does not make the audit pass worthless. It means the audit pass produces work, not verdicts.

5. Use AI where it is genuinely strong

Locating a disclosure in a 300-page filing. "Which note covers customer concentration?" This is retrieval-augmented generation doing what it is good at, and the answer verifies itself the moment you open the page.

Summarizing MD&A. Narrative, no arithmetic. The model summarizes management's stated explanation — a claim about what the document says, which is checkable.

Comparing language across years. A real analyst technique, and AI is unusually good at it. Feed it this year's risk factors and last year's and ask what changed: added, deleted, quietly reworded. Academic finance has documented that these changes carry information — Cohen, Malloy and Nguyen's "Lazy Prices" (Journal of Finance, 2020) found that year-over-year changes in 10-K language predict future earnings, profitability and even firm bankruptcies. Analysts have always redlined filings. The model does it in a minute instead of an afternoon, and it does not get bored on page 190. (Finding the change is document work. What it means for a security is not, and is outside this guide.)

Structured extraction from narrative, with function calling and a schema — subsidiaries from Exhibit 21, lease commitments from the footnotes. The citation rule still applies. And triage: "which of these forty filings mentions a going-concern qualification?" is a magnificent use of a cheap model, because a false positive costs you one document opened.

6. Automate last

With the prompts stable and the checks passing, n8n can watch for new filings, pull XBRL facts, run extraction and arithmetic, and route only failures to a human. Do not automate before the manual version works — you will produce wrong numbers faster, and at scale nobody reads them before they reach a spreadsheet. Automate the checks first and the extraction second.

Real-World Examples

Example 1: The 1000x error, with real numbers

The document. Roku's 10-K for fiscal 2025, filed 13 February 2026. The revenue table in MD&A is headed "(in thousands, except percentages)" and prints Total net revenue $ 4,737,251 $ 4,112,898.

Before. A model reads the flattened text. The units header, three rows up and in smaller type, did not survive the flattening:

"Roku's total net revenue for fiscal 2025 was $4,737,251."

That is not a hallucinated number. It is the right digits, faithfully copied, and it is wrong by a factor of a thousand. Roku's FY2025 revenue was $4.74 billion, not $4.7 million. Put that in a spreadsheet and you have a company with the revenue of a mid-sized restaurant group.

After. The prompt requires units_as_printed and value_in_dollars as separate mandatory fields, so the model must find the header before it can fill the form: "value": 4737251, "units_as_printed": "thousands", "value_in_dollars": 4737251000. The order-of-magnitude check would have caught it regardless — no model involved.

Verify independently. The XBRL fact for the same concept reads "val": 4737251000, unit USD. The structured record has no units problem, because the value is the value.

Example 2: The column that was one over

The same table. Two fiscal years, adjacent columns:

Before. Asked for FY2025 total net revenue, the model returns $4,112,898 thousand — $4.11 billion. It is off by $624 million.

Notice what makes this worse than an invented number. $4.11 billion is a real Roku revenue figure. It is in the document, correctly transcribed, and it would survive a spot-check by anyone who half-remembers that Roku does about four billion in revenue. It is simply the wrong year — and nothing in the output says which column was read, so there is nothing to catch.

After. column_header_verbatim is required. The model must write "2025" or "2024", and to write it, it must look. If it writes "2024" while claiming FY2025, the mismatch is mechanical and a script catches it without a human.

The same trap sits in Apple's income statement, which prints three years at once: total net sales of 391,035 / 383,285 / 394,328 (in millions) for FY2024, FY2023 and FY2022. FY2022 is larger than FY2023 — so a model grabbing the wrong column does not even produce a plausible growth story. It produces a plausible-looking number that quietly inverts the trend.

Example 3: Cross-footing catches what nobody was looking for

Before. An extraction pulls three figures from the Roku table: Platform $4,144,886, Devices $592,365, and — because the model drifted one row while reading a nested table — a total of $4,112,898. Nothing about that looks wrong. Three real numbers, correctly transcribed, from one document.

After. The cross-foot runs as three lines of code, not as a prompt:

4,144,886 + 592,365 = 4,737,251
4,737,251 != 4,112,898  ->  FAIL

The check does not need to know which number is wrong. It only needs to know the three cannot all be right, and that is enough to stop the figure reaching a spreadsheet. This is why the arithmetic lives in code: cheap, deterministic, never tired, no opinions.

Example 4: Where AI actually wins — the risk-factor redline

Before. An analyst covering forty companies wants to know which changed their risk-factor language this year. Item 1A runs 15 to 40 pages per filing. By hand this is a two-week project that nobody has ever finished.

After. Both years' Item 1A sections go into a long-context window model with a diffing prompt. Output, in about a minute per company:

Added: a new risk factor on supply concentration in a single manufacturing region. Deleted: last year's risk factor on a now-settled legal matter. Reworded: "we may experience" became "we have experienced" in the paragraph on component shortages — hypothetical to actual. Unchanged: the remaining 22 risk factors, verbatim.

That tense change is exactly what four hours of human concentration tends to miss.

What changed: nothing about the model. The task changed — from "assert a number" to "find a difference between two texts I can then read myself." The output is a pointer, and every pointer is checkable in seconds. No arithmetic is involved at all.

Industry-Specific Applications

Equity research. XBRL for the model inputs, AI for the narrative. Reading forty MD&A sections is where the model saves you days.

Credit and lending. Covenant compliance is both the killer application and the sharpest edge: a covenant test is arithmetic on figures pulled from a statement, and a units error propagates straight into a breach determination that is wrong in either direction. Tie every input to a primary statement or an XBRL fact — never to a model's summary of one.

Audit and accounting. The FinVerBench result lands here: models flagging clean statements as erroneous generate review work, not assurance. Use AI to triage, and keep the conclusion with the human who signs.

Private markets and non-US filers. The hardest case: no XBRL, often no HTML, sometimes a scan. Every warning here applies at full force and none of the escape hatches exist.

Best Practices

• If a structured version exists, use it. An XBRL fact is not an extraction with a confidence score; it is the number.
• If the model cannot cite the page, the number does not exist. Make not_found an honourable answer.
• Require the column header verbatim — the cheapest defence against the most common error.
• Require the units header before any figure, and the dollar value with the units applied.
• Cross-foot in code, not in the model. Deterministic checks catch non-deterministic errors. The reverse does not hold.
• Tie every quoted figure back to the primary statement. MD&A is commentary; the statement is the record.
• Audit in a fresh context. A model shown its own answer defends it.
• Label GAAP versus non-GAAP explicitly, every time. Only one of them is audited.
• Keep the boundary. Summarize what the document says. Do not ask a model whether the numbers are good news.
• Log every figure with its provenance. In six months, "where did this come from" must have an answer that is not "the AI said so."

Common Pitfalls

The number that is wrong in a way that looks right. The defining failure, and the reason every other rule exists. A prior-year figure, a subtotal, a non-GAAP measure — all real, all in the document, all wrong for the question asked.

The units error. A 1000x mistake delivered with total confidence, caused by a header the flattener dropped. The most catastrophic error here, and the most preventable.

Trusting the model's own verification. Ask a model to check its arithmetic and it will frequently confirm it, at length and persuasively. Self-consistency is not correctness.

The false-positive tax. FinVerBench found nine of fourteen model runs flagging 95–100% of clean statements as erroneous — GPT-4.1 at 95.3%. An audit pass that cries wolf on everything gets switched off within a week, and then you have no audit pass at all. Treat its output as a queue, not a verdict.

Long-context decay. DocFinQA is the honest picture of pasting in a whole filing and asking a precise question: GPT-4's error rate roughly doubled once the answer had to be found rather than handed over. A large context window means the model can see the document. It does not mean it will find the right row.

Restatements and fiscal calendars. The same year filed twice with different numbers is a restatement, not a bug. And Apple's "FY2024" ended 28 September 2024, not 31 December — a model comparing "2024" across companies with different year-ends is comparing different things.

Extension tags and scanned pages. A concept missing from the us-gaap taxonomy may just be a custom tag, so a missing XBRL fact is not a missing disclosure. And if a document is a scan, OCR runs first: a misread digit in a scanned table is invisible, silent, and upstream of every check on this page.

Scope creep into advice. The most seductive pitfall. The model will happily tell you whether the results look good, and it will sound authoritative. Extraction and summarization are document tasks with checkable answers. "Is this a good investment" is not, this workflow does not produce it, and nothing here should be read as though it did.

Measuring Success

• Citation rate. What fraction of extracted figures carry complete, verifiable provenance? Target 100%. Below that is a figure you cannot defend.
• Cross-foot pass rate. Your primary quality signal, and it costs nothing to compute.
• Tie-out failures caught, per filing. Should be non-zero. If your checks never fire, they are not running — nobody's extraction is perfect.
• Units errors reaching a human. Should be zero: the order-of-magnitude assertion catches them first.
• Audit-pass precision. Of the figures the adversarial pass flagged, how many were genuinely wrong? If that ratio is bad, the pass is generating busywork and you will stop running it.
• Zero uncited numbers published. Pass/fail, not a percentage. The one that matters.

Cost Analysis

The tokens are cheap. Measure them anyway, because everyone overestimates them. Two real filings, measured July 2026 by stripping the markup and counting:

(Tokens estimated as characters divided by four, the standard rough heuristic; dense numeric tables tokenize worse than prose, so treat these as a floor. Prices are input, per million tokens.)

Under sixty cents to read an entire annual report. Extraction plus the adversarial audit means reading it at least twice, so double it. Across a 40-company universe, four filings a year each, on Claude Sonnet 5, two passes: roughly $100 a year in tokens — a rounding error, not a budget line. XBRL costs nothing at all.

The real cost is human. One to two days to build the extraction schema and the deterministic checks, then 20–40 minutes per filing working the exception queue. That number does not go to zero, and any vendor telling you it does is selling you the 81% failure rate at a discount.

What you are buying is not "AI reads the filing for you." It is: retrieval that took an afternoon now takes a minute; narrative comparison that was never done at all now gets done; and the arithmetic you always should have been checking now gets checked automatically. What you are not buying is the right to publish a number nobody looked at.

Related Use Cases

• AI for Survey Analysis — the same discipline on qualitative data: fixed schema, classification pass, agreement check
• AI for Literature Review — citation verification as a first-class workflow step

See also our write-up of Mafin 2.5 and reasoning-based RAG on financial documents.

Related Tools

• Claude — long filings; follows the "refuse if you cannot cite it" rule
• ChatGPT — extraction, and code for the deterministic checks
• Google Gemini — large context for whole-filing and multi-year passes
• NotebookLM — cites the passage each answer came from
• Perplexity AI — finding which filing to read in the first place
• Microsoft Copilot — Excel and Word, where the outputs land
• n8n — wiring extraction, checks and the exception queue together
• Ollama and LM Studio — local models for confidential documents

Related Models

• Claude Sonnet 5 — the default; literal about instructions, which is what a citation rule needs
• Claude Opus 4.8 — multi-document reasoning across filings and years
• Claude Haiku 4.5 — cheap enough to triage a whole coverage universe
• GPT-5.5 — strong generalist, large context
• Gemini 3.5 — large context at low cost
• DeepSeek — cheapest capable option for bulk narrative work

New to this? Start with prompt engineering — the difference between a prompt that asks for a number and one that asks for a citable number is the difference between this workflow and the 81% failure rate. Then read hallucinations, because in financial documents they do not look like nonsense. They look like $4,737,251. For the wider picture, see AI in finance and data analysis.