AI Insurance System Case Study

The client is a mid-sized independent property and casualty administrator and agency group in the southeastern United States. About 85 people handle personal auto and homeowners business across North Carolina, South Carolina and Georgia, with roughly 2,400 inbound service contacts and 750 first notices of loss every month.

I'm Yuri Musienko, co-founder of Merehead. I expected the language model to be our main risk on this project. It wasn't. The documents were.

The Starting Point: Good People, Fragmented Procedures

The company had spent years refining how it takes a claim, answers a billing question and escalates a complaint. That expertise sat in PDFs, email threads, shared drives and the heads of long-serving specialists.

Where service answers came from

An experienced specialist answered a water-damage question in seconds. A new representative searched shared folders for the same guidance. Some teams worked from current checklists, others kept local copies. When a procedure changed, managers had no way to check who had read the update or whether the old file still surfaced in search.

The claims intake bottleneck

New claims arrived by email, portal upload and phone notes, usually with photos, contractor estimates, police reports or repair invoices attached. Handlers copied the key details into the claims platform by hand, checked what was missing and wrote follow-up messages one by one. The work was repetitive, and a premature assumption at intake could damage the customer's experience for the rest of the claim.

What leadership ruled out

Management set two conditions in the first meeting. First, the AI would not decide whether a claim gets approved. Coverage determinations, payment amounts, denials and fraud allegations stay with people in the existing workflow. Second, department heads had to own and update their own content without filing a ticket with developers every time a process changed.

The first condition also lines up with where U.S. regulators are heading. The NAIC adopted its Model Bulletin on the Use of Artificial Intelligence Systems by Insurers on December 4, 2023. It expects a written AI program, testing for errors and documentation regulators can request during an examination. A system that keeps humans on consequential decisions and logs every retrieval is far easier to defend in that conversation.

Decisions That Shaped the Architecture

We spent the first two weeks on discovery and process mapping with the claims and service teams. Three decisions came out of that phase, and each one cost us something.

Department workspaces instead of one shared folder

The fast option was to dump every document into one folder and index it. We rejected it. Each department got a managed workspace with its own library, editors, approvers and access rules:

• Claims Operations owns loss-intake scripts, evidence checklists and escalation protocols.
• Customer Service owns approved response templates plus billing and renewal guidance.
• Compliance owns restricted procedures and final policy wording.
• Training owns onboarding playbooks and internal FAQs.

Staff sign in with company accounts through single sign-on. A contributor drafts or edits only inside the workspaces assigned to their role, and a department approver reviews every change before publication. Each published item carries an owner, a revision date, applicability tags, an effective date and a review schedule. The price was a slower start, since every department had to agree on who owns what before we indexed a single file.

Permissions enforced at retrieval, not after generation

Our ingestion service extracts text from PDFs and office files, keeps the source link, splits content into retrievable passages and indexes each passage with six metadata fields: department, state, insurance line, user permission, effective date and document status.

We check permissions before any passage reaches the language model. Filtering the answer after generation looks cheaper, but by then the model has already read the confidential text and can paraphrase it. With retrieval-side filtering, a service agent never receives a claims investigation note just because its wording matches the question. We chose retrieval over agent tool calls for this layer because procedures change slowly and every answer needs a citation; the RAG vs MCP trade-off looks different when the data is live.

Separating knowledge from live state

The knowledge base never answers questions about live claim status or personal policy details. Authenticated, account-specific requests go through controlled integrations with the operational systems. This rule kept stale or wrong customer data out of the retrieval index entirely.

Two Assistants on One Governed Source

With the knowledge layer in place, we built two assistants that read from it under different rules.

AI Claims Processing Assistant

At first notice of loss, the assistant pulls together the intake form, call summary, photos and supporting paperwork. It extracts the core fields (incident date, location, policy reference, parties involved, document types) and checks which ones are missing. Using the company's own claim-intake protocols, it then produces a draft case summary, a missing-information list and a suggested next action for the handler.

In the water-damage example, the assistant marks the missing incident date and repair estimate and drafts the request using the approved customer-facing template. Where the right procedure depends on the carrier, the state or the policy type, it shows the source and flags its uncertainty. We configured it to flag rather than guess.

AI Customer Service Agent

The second assistant works in two places: a web chat for customers and an internal console for agents. It is a customer-facing AI chatbot with one hard rule: it answers only from approved content. Typical questions include how to report a claim, where to upload documents, how to get proof of insurance and what happens after submission.

When a question falls outside the approved sources, the agent says it cannot confirm the answer and hands the conversation to an employee with the transcript and retrieved references attached. Coverage disputes, complaints, urgent safety issues and sensitive cases follow stricter review and escalation rules.

What Broke During Testing

We underestimated the source material. Our plan assumed the documents were roughly correct and only needed indexing.

The source material was the real problem

A content review turned up duplicate checklists, procedures nobody owned and state-specific instructions buried inside generic documents. Indexing that material as-is would have given the assistants several conflicting answers to the same question, each with a valid-looking citation.

Folder permissions failed on mixed documents

We started with folder-level access rules. They broke on documents that mixed internal notes with customer-safe content, because a single file could not be both visible and hidden to the customer channel.

Correct answer, wrong state

During early testing, both assistants sometimes returned a procedure that was accurate but did not apply: guidance written for another state or another insurance line. For an insurance operation spread across three states, that is a compliance problem, and a handler who catches it twice stops trusting the tool.

How We Fixed It

Each failure got its own fix, and none of them involved changing the model.

Content inventory before ingestion

We built a content inventory, sat down with department owners to resolve conflicting guidance and retired outdated files before any of them went into AI search. This step took longer than the assistant build itself.

Document-level access and an audit trail

We separated internal-only material from approved response content, moved access control to the document level and added an audit trail covering publication, retrieval and human overrides.

Tighter filters, visible citations, harder test scenarios

We tightened the state and policy filters in retrieval and made source citations visible in the staff console, so a handler sees which document an answer came from before acting on it. We also expanded the evaluation set to four scenario types: conflicting documents, outdated revisions, missing data and intentionally ambiguous customer requests. This matches what we see across most AI integrations into existing apps: the model call is the easy part, and data quality and access rules decide whether staff trust the output.

Rollout sequence

After discovery and knowledge cleanup, we launched the first controlled ingestion pipeline. We then built the claims assistant, connected it to a non-production claims dataset and tested it with a small group of handlers. Customer service went live only after the approved answer library and escalation rules were ready. The final phase covered quality assurance, permission testing, staff training and operational dashboards.

Benchmarks and How We Measure Them

We scoped the 90-day pilot to 12 claims handlers and 18 service representatives, with staff reviewing every AI-produced output. We captured a baseline before rollout and compare it with the same workflows during the operating period. The figures below are the targets we set for that pilot, not guaranteed savings.

Real results depend on the ticket mix, source quality, integration depth, how many customers adopt the chat channel and what share of conversations qualifies for automation. The benefit I'd bet on long term is consistency: an updated procedure reaches every authorized employee through the same publishing workflow on the day it is approved.

What the Stack Looks Like

Why the Knowledge Layer Is the Product

This client did not need another chatbot trained on generic insurance material. It needed a system that knows which internal instruction applies, who may see it and how a change reaches the front line. Claims staff kept responsibility for every decision. Service teams got faster access to approved answers. Department managers now maintain their own procedures without an IT queue.

For insurance agencies, third-party administrators and insurtech platforms, the two assistants are the visible part. The governed knowledge layer underneath is what makes them reusable for a new state, a new product line or a new team. At Merehead we apply the same knowledge-first structure in enterprise AI development projects where access rights and audit trails carry as much weight as answer quality.

FAQ