From two days to under ten minutes for case evaluations

The problem

Operations at an AI underwriting company depended on engineering to run case evaluations. The capability existed as a script with no user interface, and running it required one engineer's knowledge. Requests took two days to turn around.

Operations translated insurance policies and regulatory guidelines into 30–40 prompts per policy, but had no workspace for managing those prompts through their lifecycle.

Deliver the immediate need

Production Slack app

• Case evaluation in <10 minutes
• Human final approval

→

Explore what comes next

Broader workflow prototype

• Policy and prompt lifecycle
• Reporting and exploration

User feedback shaped the broader prototype.

What I did

Deliver the immediate workflow in Slack

The CTO wanted a full operations console. Its supporting capabilities were not yet in place, so I pushed for a Slack app first. Operations already worked there; making case evaluation accessible did not need to wait for the larger system.

I built a prototype and working proof of concept around the existing evaluation capability. The app brought together policy logic, retrieved documentation, and model predictions to highlight risk factors and recommend determinations. A person retained final approval.

My proof-of-concept front end became the front end of the production Slack app during the engagement.

Use the prototype to discover the broader workflow

I also prototyped prompt evaluation for user feedback. Working through that experience raised questions about policy setup, evaluation, and connections to external systems. The prototype expanded to let users explore that lifecycle, with simulated experiment results.

Reporting feedback changed the design. Operations and sales were pulling from different sources, so a source of truth was plainly needed, and I drew on existing artifacts and prior reporting experience to propose standard reports. Operations pushed back on two grounds. Their reporting needs were still too new, and the team needed room to investigate questions clients might ask. Separately, nobody wanted to own a report. A report needs a maintainer in a way a dashboard does not.

Both objections were right, and together they set the sequence. The dashboard became the report: we shaped that as the MVP, carrying the questions that were already common in a surface nobody had to maintain as a document. Data exploration followed as phase two, once there was enough evidence of what people actually needed to explore. The prototype let users work through what to standardize and where to preserve flexibility.

The outcome

The production Slack app reduced case-evaluation turnaround from two days to under ten minutes and freed more than 20 engineering hours per week previously spent on support tickets. Final approval remained with a person.

Separately, an intern reused much of my front-end work in the broader framework's proof of concept. That framework reached POC during my engagement and was productized afterward.