How often is AI right about insurance facts?
Brainpan checks specific, verifiable claims AI engines make about named insurers (ratings, rankings, market share and where a company sells) against authoritative sources. This page sets out what the Q3 2026 check found and exactly how each claim is scored.
Fewer than six in ten checkable claims were fully right
| Verdict | Claims | Share of 1,065 |
|---|---|---|
| Supported | 626 | 58.8% |
| Partially supported | 185 | 17.4% |
| Outdated | 164 | 15.4% |
| Contradicted | 84 | 7.9% |
| Unsupported or unverifiable | 4 | 0.4% |
| Insufficiently specific | 2 | 0.2% |
The Verified Accuracy Rate leaves out the six claims that could not be scored, so 626 supported claims are 59.1% of the 1,059 scoreable ones (and 58.8% of all 1,065). The remaining 40.9% of scoreable claims were partially supported, outdated or contradicted. It should not be reported as a 40.9% false-claim rate: only 7.9% were contradicted outright.
Ratings and rankings hold up best; market share worst
| Claim type | Claims | Verified Accuracy Rate |
|---|---|---|
| Rating-agency score | 357 | 65.3% |
| J.D. Power ranking | 185 | 62.5% |
| Binary state availability | 284 | 58.9% |
| Market share | 239 | 47.7% |
Only four claim types are checked, because each has an authoritative answer: a rating-agency grade, a published J.D. Power ranking, a filed market-share figure, or whether a carrier sells in a given state. General praise (“great customer service”) is scored under Sentiment, not Accuracy.
No engine is reliable enough to take at its word
| AI surface | Claims | Verified Accuracy Rate |
|---|---|---|
| Perplexity | 44 | 70.5% |
| Meta AI (supplementary) | 369 | 68.6% |
| Gemini† | 268 | 54.5% |
| ChatGPT (OpenAI) | 68 | 52.9% |
| Copilot | 217 | 52.3% |
| Claude | 99 | 50.5% |
Perplexity (44 claims) and ChatGPT (68) made few checkable claims, so their rates are less stable than the others. † Q3 Gemini results come from a governed replacement configuration; see Methodology v1.1.
Two errors that repeat across engines
Invented ratings
One insurer has 8 of 9 contradicted claims that invent the same kind of financial-strength letter grade. Three independent AI engines make the error. The company’s own public filings say it does not hold that rating. Its Verified Accuracy Rate is 10.5% and its False-Claim Rate 47.4%, the highest of any scored entity.
Stale facts
One widely discussed, high-volume insurer scores a 25.9% Verified Accuracy Rate. The claims were once true: AI answers still repeat a rating before a downgrade and a market position that has since shifted. Both changes are confirmed by several independent trade-press sources.
An error repeated by several independent engines points to a pattern in the sources AI systems learn from, not a one-off mistake. Brainpan does not name brands in public Accuracy findings; the named entity and evidence trail are shared only with that brand through a private diagnostic.
How each claim is checked
- 1. Extraction
- A structured pass reads every one of the 1,800 responses and pulls out claims in the four types. Q3: 1,082 claims, no failed responses.
- 2. Verification
- Each claim gets two separate calls: a grounded research call that gathers evidence, then a structured call that records the verdict. Government filings and rating-agency releases rank above other sources, and an AI-generated or unsourced statement never counts as evidence on its own.
- 3. Resolution
- Claims are matched to the study’s canonical list of 226 brands. Q3: 1,065 claims resolved, a 98.4% match rate.
- Verdicts
- Supported, partially supported, outdated, contradicted, unsupported or unverifiable, or insufficiently specific.
- Three rates
- Verified Accuracy Rate (supported ÷ scoreable), False-Claim Rate (contradicted ÷ all eligible) and Evidence Coverage Rate (scoreable ÷ all eligible). Each answers a different question, so they are never blended into one number.
- Quality checks
- A 20-claim spot-check confirmed all 20 verdicts, plus one independent web fact-check.
- Full rules
- AIVI™ Methodology v1.1, section 1a.
What these numbers do and don’t say
- They describe checkable claims in AI answers to insurance-shopping prompts, not every sentence AI writes about insurers.
- 59.1% is the share of scoreable claims that were fully supported. It is not the share of AI answers that are accurate.
- They are not a measure of any insurer’s product quality, claims handling or financial strength.
- This is one quarter’s sample: 300 prompts and 1,800 responses. Accuracy will be tracked edition over edition.
How to cite this research
Brainpan.AI. (2026, November 3). AI accuracy in insurance answers: Insurance AIVI™ Benchmark, Q3 2026 (Methodology v1.1). https://www.brainpan.ai/research/ai-accuracy/
Figures on this page are licensed CC BY 4.0 with attribution to Brainpan.AI. Press contact: press and media kit.
What does AI get wrong about your brand?
The public findings are aggregate. A Multi-Engine Diagnostic names the specific claims AI engines make about your company, which are wrong or out of date, and which sources they come from.
Kevin Walsh
Kevin Walsh is the founder and lead engineer of Brainpan.AI, which builds the Visibility Engine, the measurement platform behind the AIVI™ benchmarks. Brainpan.AI independently measures how brands are mentioned, recommended, cited and described across AI answer engines, and has no client relationship with any brand named in this report.