--- source: 'https://howaiworks.ai/use-cases/ai-for-financial-report-analysis' section: use-cases title: 'AI for Financial Report Analysis: Verification' description: >- Pull figures from a 10-K without publishing a wrong one: citation-forced extraction, cross-footing, tie-out, and why XBRL beats parsing the PDF. category: Research & Analysis subcategory: Financial Reporting targetAudience: - Financial Analysts - Equity Research Associates - Accountants - Data Engineers toolsUsed: - Claude - ChatGPT - Google Gemini - NotebookLM - Perplexity AI - n8n modelTypes: - Claude Sonnet 5 - GPT-6 Astra - Gemini 3.8 Flash - Claude Haiku 4.5 difficulty: Advanced industries: - Finance - Investment Research - Accounting - Banking roles: - Financial Analyst - Equity Research Associate - Accountant - Data Engineer estimatedTimeToImplement: >- 1-2 days to build the extraction and verification pass, then 20-40 minutes per filing potentialROI: >- Reading a full 10-K costs $0.04-$1.14 in API tokens depending on the model (September 2026 pricing; arithmetic shown in Cost Analysis). The saving is analyst hours on retrieval, not on verification — that pass does not go away. tags: - financial analysis - SEC Filings - XBRL - Document Analysis - table extraction - hallucinations - RAG - Finance AI relatedUseCases: - ai-for-literature-review - ai-for-contract-review - ai-for-survey-analysis lastUpdated: '2026-09-06' readingTime: 12 minutes --- # AI for Financial Report Analysis: Verification > Pull figures from a 10-K without publishing a wrong one: citation-forced extraction, cross-footing, tie-out, and why XBRL beats parsing the PDF. Ask a language model for a company's revenue and it will give you a number. That is the problem. The number arrives with the same fluent confidence whether it is this year or last year, whether the header said "in thousands" or "in millions," whether it is the GAAP figure or the adjusted one management prefers. **A wrong number does not look wrong. It looks like a number.** The workflow that fixes this *is* the value of this page: **never let the model be the source of a figure.** Let it find, cite and summarize. Let arithmetic and a human confirm. One boundary, and it holds throughout. **This is a document-processing guide, not investment advice.** Reading a 10-K is document work — locating disclosures, extracting figures, summarizing management's language. Nothing here recommends buying, selling or holding anything, and no workflow on this page produces a valuation opinion. ## The Challenge An annual report is not prose with tables in it. It is a structured legal artifact whose meaning is carried as much by layout as by sentences — and a language model receives none of that structure. It receives a string. What actually goes wrong, none of it exotic: - **The wrong column.** An income statement prints two or three fiscal years side by side. The model returns the prior year for the current year. Both numbers are real, both are in the document, and the error is invisible unless you already knew the answer. - **The wrong row.** Tables nest: line items indent under subtotals, merged headers span columns. Flattened to text, "Total current assets" and the line above it become adjacent strings with no hierarchy between them. - **The units.** A statement headed "(in thousands)" prints `4,737,251` and means $4.74 billion. A model that drops the header reports $4.7 million: a **1000x error, stated with total confidence.** - **The subtotal folded into the total.** Ask for a sum of segments and the model adds the subtotal row it already contains. - **GAAP mixed with non-GAAP.** "Adjusted EBITDA" and "net income" sit paragraphs apart. Only one is audited. A model asked for "earnings" silently chooses. - **Restated versus as-originally-reported.** The same fiscal year appears in three consecutive 10-Ks, and it is not always the same number. ### Why this happens **A financial table's meaning lives in two dimensions.** A cell means "FY2025 revenue" only because of the column above it and the row beside it. The units apply because of a header three rows up. The indentation says this line rolls into that one. **The text that reaches the model is one-dimensional.** PDF extraction flattens the grid into a sequence of tokens: column boundaries become whitespace, merged headers collapse, and indentation — which *was* the hierarchy — becomes nothing at all. This is a hard, decades-old problem in [data processing](https://howaiworks.ai/glossary/data-processing), not a defect of any particular model. **The model then reasons flawlessly over a mangled representation,** returning a confident assertion built on a corrupted premise. It is [hallucination](https://howaiworks.ai/glossary/hallucinations) in its most expensive costume, because the number is usually *almost* right — and it is why "use a better model" does not fix it. ### What the research says **FinanceBench** (Islam et al., 2023) tested 16 model configurations — GPT-4-Turbo, Llama 2, Claude 2, with vector stores and long-context prompts — on a 150-case sample, manually reviewing 2,400 answers. **GPT-4-Turbo with a retrieval system incorrectly answered or refused to answer 81% of questions.** All models examined, the paper concluded, "exhibit weaknesses, such as hallucinations, that limit their suitability for use by enterprises." **DocFinQA** made the point about length: it put 7,437 FinQA questions back into their full source filings, pushing average context from **under 700 words to 123,453 words**. The paper's own summary of what that did to a frontier model is blunt — **GPT-4's error rate roughly doubled**. (For a model the paper reports end to end: GPT-3.5 scores about 67% when handed the gold context and **36–43%** when made to find it inside the full document.) Retrieval did not rescue it either: a fine-tuned ColBERT retriever hit the right chunk first only **35%** of the time. Finding the right page is most of the job; see [information retrieval](https://howaiworks.ai/glossary/information-retrieval). **FinQA** established that this is expert work, not a reading test: financial experts scored **91.16%**, non-expert crowd workers **50.68%**. These are 2023–2024 evaluations and models have improved since. What has not changed is the **shape** of the failure: silent, plausible, numeric. ## How AI Solves It The most useful distinction on this page is between two operations that look similar and carry opposite risk. > **Finding and summarizing is strong. Computing and asserting is dangerous.** **Finding** is recall-shaped. "Where is the lease-commitment schedule?" The model says page 87; you open page 87. The answer **verifies itself in three seconds**, and being wrong costs a page turn. **Asserting** is generative. "What was total revenue?" The model says $4.1 billion. You cannot check that without going back to the document — which is what you were trying to avoid. Being wrong costs you a number in a model, a memo, or a client deck, and it propagates silently. Everything below follows: use the model to **locate**, not to compute; force **provenance** on every figure; verify with **arithmetic, not with the model**. And before any of it — check whether you need to parse the document at all. ### Do not parse the PDF if a structured version exists Most articles on this topic never mention this, and it is the first thing a competent data engineer would say. **SEC filings are already machine-readable.** Since the Inline XBRL mandate completed its phase-in — fiscal periods ending on or after **June 15, 2021** for all remaining operating-company filers — the financial statements in every 10-K and 10-Q are tagged fact by fact. The SEC publishes those facts free at `data.sec.gov`: **no API key**, just a declared `User-Agent` header and a 10-request-per-second limit. An actual response fragment from the `companyconcept` endpoint for Roku's revenue, fetched July 2026: ```json { "start": "2025-01-01", "end": "2025-12-31", "val": 4737251000, "fy": 2025, "fp": "FY", "form": "10-K", "accn": "0001628280-26-008114", "filed": "2026-02-13" } ``` The **value in dollars, unscaled** — no "(in thousands)" header to miss. The exact period. The form. The accession number, a permanent pointer back to the filing. No column to misread, no unit to misinterpret, no subtotal to confuse with a total. **That is not an extraction. That is the number.** No model touched it, so no model can have been wrong about it. The rule: **if the figure exists as an XBRL fact, take the XBRL fact.** Use AI for what is *not* tagged — narrative, footnotes, risk factors, segment commentary — which is where AI is good anyway. ## Recommended Tools & Models **Start with the data source, not the model.** In descending order of trustworthiness: | Source | What it gives you | Cost | |---|---|---| | **XBRL company facts API** | Every tagged fact for a filer: value, unit, period, form, accession | Free, no key | | **XBRL company concept API** | One concept across all periods: time series, YoY, restatement checks | Free, no key | | **XBRL frames API** | One concept across all filers, one period: cross-company comparison | Free, no key | | **EDGAR full-text search** | Keyword search across filings since 2001 — finding *which* filing says something | Free | | **The filing, read by an LLM** | Narrative, footnotes, risk factors, MD&A — anything untagged | Token cost | | **A PDF, parsed and handed to an LLM** | Everything, unreliably: private companies, non-US filers, old documents | Tokens + verification time | The bottom row is where every generic guide starts. It should be your **last** resort. Any current frontier model reads a filing competently; the prompt matters far more than the choice. | Model | Why it fits | Price (per MTok, September 2026) | |---|---|---| | **[Claude Sonnet 5](https://howaiworks.ai/models/claude-sonnet)** | Literal about instructions — including "refuse to output a number you cannot cite" | $2 in / $10 out | | **[Claude Opus 5](https://howaiworks.ai/models/claude-opus)** | Hardest multi-document reasoning; reserve it | $5 in / $25 out | | **[Claude Haiku 4.5](https://howaiworks.ai/models/claude-haiku)** | Cheap enough to triage a whole filing set | $1 in / $5 out | | **[GPT-6 Astra](https://howaiworks.ai/models/gpt)** | OpenAI's flagship since September 2026, replacing GPT-5.6 Sol; strong generalist, large context | $10 in / $50 out | | **[GPT-5.6 Luna](https://howaiworks.ai/models/gpt)** | Previous generation, still callable, still a million-token window — the cheap OpenAI slot for bulk extraction | $0.20 in / $1.20 out | | **[Gemini 3.8 Flash](https://howaiworks.ai/models/gemini)** | Large context, low price; whole-filing passes | $0.75 in / $3.75 out | | **[DeepSeek](https://howaiworks.ai/models/deepseek)** V4-Pro | Capable, but no longer the cheap outlier: peak/off-peak billing since August 2026 | $1.32 in / $3.96 out at peak, half off-peak | **Two of those prices are temporary, and both expire inside a cost model's useful life.** Gemini's Flash rate is introductory through 31 December 2026 and **doubles to $1.50 / $7.50 on 1 January 2027**. GPT-5.6's reduced rates are promotional, and OpenAI commits to them only through **21 November 2026**. [Claude](https://howaiworks.ai/ai-tools/claude), [ChatGPT](https://howaiworks.ai/ai-tools/chatgpt) and [Google Gemini](https://howaiworks.ai/ai-tools/google-gemini) all read uploaded filings. [NotebookLM](https://howaiworks.ai/ai-tools/notebooklm) earns a mention for one reason: it **cites the passage each claim came from** — the exact behaviour this workflow exists to enforce. [Perplexity AI](https://howaiworks.ai/ai-tools/perplexity-ai) helps find *which* filing to read; [n8n](https://howaiworks.ai/ai-tools/n8n) wires the pipeline together. For documents that cannot leave your machine, [Ollama](https://howaiworks.ai/ai-tools/ollama) and [LM Studio](https://howaiworks.ai/ai-tools/lm-studio) run open models locally — at a real accuracy cost, on the task where accuracy matters most. ## Step-by-Step Implementation ### 1. Ask whether this number needs a model at all - **In the audited statements of a US-listed company, period ending mid-2021 or later?** It is an XBRL fact. Fetch it. Parse nothing. - **In the narrative — MD&A, risk factors, footnotes?** A job for a model, with citations. - **A non-GAAP measure, a private company, an old filing?** You are parsing a document, and the rest of this guide applies in full. Fetching a fact is one HTTP request with a `User-Agent` header identifying you: ``` https://data.sec.gov/api/xbrl/companyconcept/CIK0001428439/us-gaap/RevenueFromContractWithCustomerExcludingAssessedTax.json ``` Two caveats. **Companies use extension tags** — custom concepts outside the standard `us-gaap` taxonomy — so a missing concept does not mean a missing disclosure. And **facts are versioned by filing**: the same fiscal year appears in several 10-Ks, and differing values mean a restatement, which is information rather than an error. ### 2. Extract with forced provenance The prompt has one job: make it impossible for the model to hand you a number without also handing you the means to check it. **Prompt template — Citation-Forced Extraction From a Filing** ````text You are extracting figures from a financial filing. You are NOT interpreting them and you are NOT offering any view on the company. **The absolute rule.** Every number you output must be COPIED from the document and must carry the evidence that lets me verify it without re-reading the filing. If you cannot supply every evidence field for a figure, you MUST return `"value": null` and `"status": "not_found"`. **A number you cannot cite does not exist.** An honest "not_found" is a correct answer. An uncited number is a failure even when it happens to be right. **Before reading any figure, find and state the table's units header** — "(in thousands)", "(in millions)". If a table has no units header, say so rather than assuming dollars. **For each requested item, return exactly this structure:** ```json { "item": "Total net revenue", "value": 4737251, "units_as_printed": "thousands", "value_in_dollars": 4737251000, "fiscal_period": "FY2025 (ended 2025-12-31)", "column_header_verbatim": "2025", "row_label_verbatim": "Total net revenue", "statement": "Consolidated Statements of Operations", "page": 54, "quoted_line": "Total net revenue $ 4,737,251 $ 4,112,898", "basis": "GAAP", "status": "found" } ``` **Field rules, all mandatory:** - `column_header_verbatim` — the literal text atop the column you took the number from. When two or three fiscal years print side by side, this field proves you took the right one. - `row_label_verbatim` — the literal row label; note the indentation if it sits under a subtotal. - `quoted_line` — the raw line as printed, INCLUDING the adjacent columns. Do not tidy it. - `basis` — "GAAP" or "non-GAAP". If the label says "adjusted", "pro forma", "underlying" or "core", it is non-GAAP. Say so. Never silently substitute one for the other. - `value_in_dollars` — the figure with the units header applied. **Items to extract:** [LIST THEM] **DOCUMENT:** [PASTE THE FILING TEXT, OR ATTACH THE FILE] ```` Every figure must carry a pointer back to where it came from. The refusal clause is the mechanism, not decoration: a model permitted to say 'not found' will say it; a model that must produce a number will produce one. Three things there do the work. **The refusal clause**: models produce numbers because they are asked for numbers — give the model an honourable exit and it takes it. **`column_header_verbatim`**: to write down which column it read, the model has to look at the column. **`quoted_line` with its neighbours**: verification becomes "does this string appear in the source" rather than "re-read the filing." ### 3. Run the deterministic checks **This step involves no AI at all, and it is what makes the workflow defensible.** **Cross-foot.** Do the components add to the stated total? From Roku's FY2025 10-K: Platform $4,144,886 plus Devices $592,365 equals **$4,737,251** — exactly the printed total. If your extracted segments do not sum to your extracted total, one of the three came from the wrong row, column or year. You do not yet need to know which; you need to know you cannot use them. **Tie out to the primary statement.** A figure quoted in MD&A must equal the same figure in the audited statement. If they disagree, one of them is non-GAAP. **Units sanity check.** A mid-cap streaming company has neither $4.7 million nor $4.7 trillion in annual revenue. One order-of-magnitude assertion against the prior year catches the 1000x error every time. **Prior-period delta.** Pull last year from XBRL (free) and compute the change. A 0.0% change usually means the model read the same column twice; a 900% change usually means it read a different line item. Run these as code, not as a prompt. **A model asked to audit its own arithmetic tends to agree with itself** — you have then spent tokens buying a second opinion from the same witness. ### 4. Run an adversarial verification pass A second model pass is genuinely useful, but only framed as an audit against the source, and only in a *fresh context*. **Prompt template — Adversarial Extraction Audit** ```text You are auditing figures another system extracted from the attached filing. Your job is to FALSIFY each one. Assume each is wrong until the document proves otherwise. You are not evaluating the company and not commenting on its performance. You are checking whether these extractions are faithful to the document. **For each claimed figure, check all six and report each independently:** 1. **Does the quoted line actually appear in the document?** Search for it. If the string is not present verbatim, the extraction is unverifiable — say so and stop there for that item. 2. **Units.** Find the table's units header yourself. Does the stated `value_in_dollars` equal the printed figure with that header applied? A "(in thousands)" table whose figure was not multiplied by 1,000 is a 1000x error. 3. **Column.** Print the verbatim column headers left to right. Is the figure under the column for the claimed fiscal period, or an adjacent one? Prior-year comparatives sit directly beside the current year and are the most common wrong answer. 4. **Row.** Is the claimed row a line item or a subtotal? If several rows were summed, confirm none of them is itself a subtotal of the others. 5. **Basis.** GAAP or non-GAAP? Check the row label and surrounding text for "adjusted", "pro forma", "underlying", "core", "constant currency", "excluding". 6. **Vintage.** Does the document describe this figure as restated, recast or reclassified? **Output one row per figure: PASS, FAIL or UNVERIFIABLE, with the specific evidence.** "It looks plausible" is not a PASS. If you cannot locate the figure in the document, the answer is UNVERIFIABLE — not PASS. **CLAIMED FIGURES:** [PASTE THE JSON FROM STEP 2] **DOCUMENT:** [RE-ATTACH THE FILING] ``` Run this in a NEW conversation with the source document re-attached. A model shown its own prior output in the same thread will defend it; a model shown an anonymous claim will test it. **Expect false alarms, and budget for them.** FinVerBench, a 2026 preprint, tested 14 model runs on detecting injected errors in statements built from 10-K XBRL data and found that *calibration*, not detection, is the binding problem: **nine of the fourteen runs flagged 95–100% of the clean, error-free statements as erroneous.** GPT-4.1 posted a **95.3%** false-positive rate — 41 of 43 clean statements misclassified. The cause was not bad arithmetic. The paper is precise about it: 95.1% of those false positives *"flag real but benign numerical discrepancies — the model's arithmetic is correct, but it misinterprets missing line items as errors."* It **mistook a simplified presentation for an inconsistent one.** That does not make the audit pass worthless. It means **the audit pass produces work, not verdicts.** ### 5. Use AI where it is genuinely strong **Locating a disclosure in a 300-page filing.** "Which note covers customer concentration?" This is [retrieval-augmented generation](https://howaiworks.ai/glossary/retrieval-augmented-generation) doing what it is good at, and the answer verifies itself the moment you open the page. **Summarizing MD&A.** Narrative, no arithmetic. The model summarizes management's stated explanation — a claim *about what the document says*, which is checkable. **Comparing language across years.** A real analyst technique, and AI is unusually good at it. Feed it this year's risk factors and last year's and ask what changed: added, deleted, quietly reworded. Academic finance has documented that these changes carry information — Cohen, Malloy and Nguyen's "Lazy Prices" (*Journal of Finance*, 2020) found that year-over-year changes in 10-K language predict future earnings, profitability and even firm bankruptcies. **Analysts have always redlined filings. The model does it in a minute instead of an afternoon, and it does not get bored on page 190.** (Finding the change is document work. What it means for a security is not, and is outside this guide.) **Structured extraction from narrative,** with [function calling](https://howaiworks.ai/glossary/function-calling) and a schema — subsidiaries from Exhibit 21, lease commitments from the footnotes. The citation rule still applies. And **triage**: "which of these forty filings mentions a going-concern qualification?" is a magnificent use of a cheap model, because a false positive costs you one document opened. ### 6. Automate last With the prompts stable and the checks passing, [n8n](https://howaiworks.ai/ai-tools/n8n) can watch for new filings, pull XBRL facts, run extraction and arithmetic, and route only failures to a human. **Do not automate before the manual version works** — you will produce wrong numbers faster, and at scale nobody reads them before they reach a spreadsheet. Automate the checks *first* and the extraction second. ## Real-World Examples ### Example 1: The 1000x error, with real numbers **The document.** Roku's 10-K for fiscal 2025, filed 13 February 2026. The revenue table in MD&A is headed **"(in thousands, except percentages)"** and prints `Total net revenue $ 4,737,251 $ 4,112,898`. **Before.** A model reads the flattened text. The units header, three rows up and in smaller type, did not survive the flattening: > "Roku's total net revenue for fiscal 2025 was **$4,737,251**." That is not a hallucinated number. It is the *right digits*, faithfully copied, and it is wrong by a factor of a thousand. Roku's FY2025 revenue was **$4.74 billion**, not $4.7 million. Put that in a spreadsheet and you have a company with the revenue of a mid-sized restaurant group. **After.** The prompt requires `units_as_printed` and `value_in_dollars` as separate mandatory fields, so the model must find the header before it can fill the form: `"value": 4737251, "units_as_printed": "thousands", "value_in_dollars": 4737251000`. The order-of-magnitude check would have caught it regardless — no model involved. **Verify independently.** The XBRL fact for the same concept reads `"val": 4737251000`, unit `USD`. **The structured record has no units problem, because the value is the value.** ### Example 2: The column that was one over The same table. Two fiscal years, adjacent columns: | | 2025 | 2024 | |---|---|---| | Platform | $4,144,886 | $3,522,776 | | Devices | 592,365 | 590,122 | | **Total net revenue** | **$4,737,251** | **$4,112,898** | **Before.** Asked for FY2025 total net revenue, the model returns **$4,112,898 thousand — $4.11 billion.** It is off by $624 million. Notice what makes this worse than an invented number. **$4.11 billion is a real Roku revenue figure.** It is in the document, correctly transcribed, and it would survive a spot-check by anyone who half-remembers that Roku does about four billion in revenue. It is simply the **wrong year** — and nothing in the output says which column was read, so there is nothing to catch. **After.** `column_header_verbatim` is required. The model must write `"2025"` or `"2024"`, and to write it, it must look. If it writes `"2024"` while claiming FY2025, the mismatch is mechanical and a script catches it without a human. The same trap sits in Apple's income statement, which prints three years at once: total net sales of **391,035 / 383,285 / 394,328** (in millions) for FY2024, FY2023 and FY2022. FY2022 is *larger* than FY2023 — so a model grabbing the wrong column does not even produce a plausible growth story. It produces a plausible-looking number that quietly inverts the trend. ### Example 3: Cross-footing catches what nobody was looking for **Before.** An extraction pulls three figures from the Roku table: Platform $4,144,886, Devices $592,365, and — because the model drifted one row while reading a nested table — a total of **$4,112,898**. Nothing about that looks wrong. Three real numbers, correctly transcribed, from one document. **After.** The cross-foot runs as three lines of code, not as a prompt: ``` 4,144,886 + 592,365 = 4,737,251 4,737,251 != 4,112,898 -> FAIL ``` **The check does not need to know which number is wrong.** It only needs to know the three cannot all be right, and that is enough to stop the figure reaching a spreadsheet. This is why the arithmetic lives in code: cheap, deterministic, never tired, no opinions. ### Example 4: Where AI actually wins — the risk-factor redline **Before.** An analyst covering forty companies wants to know which changed their risk-factor language this year. Item 1A runs 15 to 40 pages per filing. By hand this is a two-week project that nobody has ever finished. **After.** Both years' Item 1A sections go into a long-[context window](https://howaiworks.ai/glossary/context-window) model with a diffing prompt. Output, in about a minute per company: > **Added:** a new risk factor on supply concentration in a single manufacturing region. > **Deleted:** last year's risk factor on a now-settled legal matter. > **Reworded:** "we may experience" became "we have experienced" in the paragraph on component shortages — hypothetical to actual. > **Unchanged:** the remaining 22 risk factors, verbatim. That tense change is exactly what four hours of human concentration tends to miss. **What changed:** nothing about the model. The *task* changed — from "assert a number" to "find a difference between two texts I can then read myself." The output is a **pointer**, and every pointer is checkable in seconds. No arithmetic is involved at all. ## Industry-Specific Applications **Equity research.** XBRL for the model inputs, AI for the narrative. Reading forty MD&A sections is where the model saves you days. **Credit and lending.** Covenant compliance is both the killer application and the sharpest edge: a covenant test is arithmetic on figures pulled from a statement, and a units error propagates straight into a breach determination that is wrong in either direction. Tie every input to a primary statement or an XBRL fact — never to a model's summary of one. **Audit and accounting.** The FinVerBench result lands here: models flagging clean statements as erroneous generate review work, not assurance. Use AI to *triage*, and keep the conclusion with the human who signs. **Private markets and non-US filers.** The hardest case: no XBRL, often no HTML, sometimes a scan. Every warning here applies at full force and none of the escape hatches exist. ## Best Practices - **If a structured version exists, use it.** An XBRL fact is not an extraction with a confidence score; it is the number. - **If the model cannot cite the page, the number does not exist.** Make `not_found` an honourable answer. - **Require the column header verbatim** — the cheapest defence against the most common error. - **Require the units header before any figure**, and the dollar value with the units applied. - **Cross-foot in code, not in the model.** Deterministic checks catch non-deterministic errors. The reverse does not hold. - **Tie every quoted figure back to the primary statement.** MD&A is commentary; the statement is the record. - **Audit in a fresh context.** A model shown its own answer defends it. - **Label GAAP versus non-GAAP explicitly, every time.** Only one of them is audited. - **Keep the boundary.** Summarize what the document says. Do not ask a model whether the numbers are good news. - **Log every figure with its provenance.** In six months, "where did this come from" must have an answer that is not "the AI said so." ## Common Pitfalls **The number that is wrong in a way that looks right.** The defining failure, and the reason every other rule exists. A prior-year figure, a subtotal, a non-GAAP measure — all real, all in the document, all wrong for the question asked. **The units error.** A 1000x mistake delivered with total confidence, caused by a header the flattener dropped. The most catastrophic error here, and the most preventable. **Trusting the model's own verification.** Ask a model to check its arithmetic and it will frequently confirm it, at length and persuasively. Self-consistency is not correctness. **The false-positive tax.** FinVerBench found nine of fourteen model runs flagging **95–100% of clean statements as erroneous** — GPT-4.1 at 95.3%. An audit pass that cries wolf on everything gets switched off within a week, and then you have no audit pass at all. Treat its output as a queue, not a verdict. **Long-context decay.** DocFinQA is the honest picture of pasting in a whole filing and asking a precise question: GPT-4's error rate roughly doubled once the answer had to be *found* rather than handed over. A large [context window](https://howaiworks.ai/glossary/context-window) means the model *can* see the document. It does not mean it will find the right row. **Restatements and fiscal calendars.** The same year filed twice with different numbers is a restatement, not a bug. And Apple's "FY2024" ended 28 September 2024, not 31 December — a model comparing "2024" across companies with different year-ends is comparing different things. **Extension tags and scanned pages.** A concept missing from the `us-gaap` taxonomy may just be a custom tag, so a missing XBRL fact is not a missing disclosure. And if a document is a scan, OCR runs first: a misread digit in a scanned table is invisible, silent, and upstream of every check on this page. **Scope creep into advice.** The most seductive pitfall. The model will happily tell you whether the results look good, and it will sound authoritative. Extraction and summarization are document tasks with checkable answers. "Is this a good investment" is not, this workflow does not produce it, and nothing here should be read as though it did. ## Measuring Success - **Citation rate.** What fraction of extracted figures carry complete, verifiable provenance? Target 100%. Below that is a figure you cannot defend. - **Cross-foot pass rate.** Your primary quality signal, and it costs nothing to compute. - **Tie-out failures caught, per filing.** Should be non-zero. If your checks *never* fire, they are not running — nobody's extraction is perfect. - **Units errors reaching a human.** Should be zero: the order-of-magnitude assertion catches them first. - **Audit-pass precision.** Of the figures the adversarial pass flagged, how many were genuinely wrong? If that ratio is bad, the pass is generating busywork and you will stop running it. - **Zero uncited numbers published.** Pass/fail, not a percentage. The one that matters. ## Cost Analysis **The tokens are cheap. Measure them anyway, because everyone overestimates them.** Two real filings, measured July 2026 by stripping the markup and counting: | Filing | Words | Approx. tokens | Gemini 3.8 Flash ($0.75) | Haiku 4.5 ($1) | Sonnet 5 ($2) | GPT-6 Astra ($10) | |---|---|---|---|---|---|---| | Apple 10-K, FY2025 | 32,607 | ~55,000 | $0.04 | $0.06 | $0.11 | $0.55 | | Roku 10-K, FY2025 | 67,579 | ~114,000 | $0.09 | $0.11 | $0.23 | $1.14 | (Tokens estimated as characters divided by four, the standard rough heuristic; dense numeric tables tokenize worse than prose, so treat these as a floor. Prices are input, per million tokens. The Gemini column doubles on 1 January 2027, to $0.08 and $0.17.) **Eleven cents to read an entire annual report on the model you would actually triage with — just over a dollar even on the most expensive frontier model.** Extraction plus the adversarial audit means reading it at least twice, so double it. Across a 40-company universe, four filings a year each, on [Claude Sonnet 5](https://howaiworks.ai/models/claude-sonnet), two passes: roughly **$100 a year in tokens** — a rounding error, not a budget line. **XBRL costs nothing at all.** **One cliff to stay clear of.** If you batch several filings into one [GPT-6 Astra](https://howaiworks.ai/models/gpt) request, watch the input size: above **272,000 tokens** the entire request reprices at 2x input and 1.5x output, not just the tokens over the line. Two Roku-sized 10-Ks fit; three do not. Send filings one at a time and the question never arises. **The real cost is human.** One to two days to build the extraction schema and the deterministic checks, then 20–40 minutes per filing working the exception queue. **That number does not go to zero, and any vendor telling you it does is selling you the 81% failure rate at a discount.** **What you are buying** is not "AI reads the filing for you." It is: retrieval that took an afternoon now takes a minute; narrative comparison that was never done at all now gets done; and the arithmetic you always should have been checking now gets checked automatically. **What you are not buying is the right to publish a number nobody looked at.** ## Related Use Cases - **[AI for Survey Analysis](https://howaiworks.ai/use-cases/ai-for-survey-analysis)** — the same discipline on qualitative data: fixed schema, classification pass, agreement check - **[AI for Literature Review](https://howaiworks.ai/use-cases/ai-for-literature-review)** — citation verification as a first-class workflow step See also our write-up of [Mafin 2.5 and reasoning-based RAG on financial documents](https://howaiworks.ai/blog/mafin-2-5-reasoning-rag-finance-breakthrough). ## Related Tools - **[Claude](https://howaiworks.ai/ai-tools/claude)** — long filings; follows the "refuse if you cannot cite it" rule - **[ChatGPT](https://howaiworks.ai/ai-tools/chatgpt)** — extraction, and code for the deterministic checks - **[Google Gemini](https://howaiworks.ai/ai-tools/google-gemini)** — large context for whole-filing and multi-year passes - **[NotebookLM](https://howaiworks.ai/ai-tools/notebooklm)** — cites the passage each answer came from - **[Perplexity AI](https://howaiworks.ai/ai-tools/perplexity-ai)** — finding which filing to read in the first place - **[Microsoft Copilot](https://howaiworks.ai/ai-tools/microsoft-copilot)** — Excel and Word, where the outputs land - **[n8n](https://howaiworks.ai/ai-tools/n8n)** — wiring extraction, checks and the exception queue together - **[Ollama](https://howaiworks.ai/ai-tools/ollama)** and **[LM Studio](https://howaiworks.ai/ai-tools/lm-studio)** — local models for confidential documents ## Related Models - **[Claude Sonnet 5](https://howaiworks.ai/models/claude-sonnet)** — the default; literal about instructions, which is what a citation rule needs - **[Claude Opus 5](https://howaiworks.ai/models/claude-opus)** — multi-document reasoning across filings and years - **[Claude Haiku 4.5](https://howaiworks.ai/models/claude-haiku)** — cheap enough to triage a whole coverage universe - **[GPT-6 Astra](https://howaiworks.ai/models/gpt)** — strong generalist, large context; reprices above 272K input tokens - **[Gemini 3.8 Flash](https://howaiworks.ai/models/gemini)** — large context at low cost, until the introductory rate ends on 1 January 2027 - **[DeepSeek](https://howaiworks.ai/models/deepseek)** — cheapest capable option for bulk narrative work New to this? Start with [prompt engineering](https://howaiworks.ai/glossary/prompt-engineering) — the difference between a prompt that asks for a number and one that asks for a *citable* number is the difference between this workflow and the 81% failure rate. Then read [hallucinations](https://howaiworks.ai/glossary/hallucinations), because in financial documents they do not look like nonsense. They look like $4,737,251. For the wider picture, see [AI in finance](https://howaiworks.ai/glossary/ai-finance) and [data analysis](https://howaiworks.ai/glossary/data-analysis). ## Frequently Asked Questions ### Can I just upload a 10-K PDF to ChatGPT and ask for the revenue? You can, and you will get a number. The problem is that you cannot tell a right one from a wrong one by looking at it. A model reading a flattened financial table can grab the prior-year column, miss the '(in thousands)' header, or fold a subtotal into a total — and it states all of those with the same confidence as a correct answer. Use AI to find the number, then verify it against the source before you use it. ### Why do LLMs get numbers wrong in financial tables specifically? A financial table's meaning lives in its two-dimensional layout: which column is which fiscal year, which row is indented under which subtotal, and what the units header says. PDF text extraction flattens that layout into a one-dimensional string. The model then reasons perfectly over a mangled representation and produces a confident, wrong number. ### Is there a machine-readable version of SEC filings I should use instead of the PDF? Yes. Every operating-company filer has been required to tag financial statements in Inline XBRL for fiscal periods ending on or after June 15, 2021. The SEC exposes the tagged facts free at data.sec.gov through the companyfacts, companyconcept and frames APIs — no API key, just a User-Agent header. An XBRL fact carries its own value, unit, period and source accession number, so units and column ambiguity simply do not arise. ### How accurate are LLMs on financial document question answering? Worse than the marketing suggests. The FinanceBench paper (2023) found GPT-4-Turbo with a retrieval system incorrectly answered or refused 81% of questions in its 150-case sample. DocFinQA found that moving the same questions from short excerpts into full-length filings roughly doubled GPT-4's error rate. Frontier models have improved since, but the failure mode has not changed shape. ### Is this page giving investment advice? No. This is a document-processing guide. Everything here is about extracting and summarizing what a filing says, with provenance, and verifying that the extraction is faithful. Nothing here is a recommendation to buy, sell or hold any security, and no workflow on this page produces a valuation opinion. ### What is cross-footing and why does it catch AI errors? Cross-footing is checking that the components of a statement actually add up to their stated totals — segment revenues to total revenue, current plus non-current assets to total assets. It is deterministic arithmetic you run in code, not in the model, and it catches exactly the errors LLMs make: a subtotal read as a line item, a row picked from the wrong level of a nested table, or a figure pulled from a different period than its neighbours. --- Source: https://howaiworks.ai/use-cases/ai-for-financial-report-analysis — HowAIWorks.ai