The problem

Renewal emails are generated from prompts, and each brokerage wants its own voice. Editing a raw prompt is intimidating for a non-engineer, so the idea was simple: let admins talk to an agent that writes and refines the prompt for them, then preview the emails it produces before anything goes live.

The hard part wasn't the chat. It was everything around it: three services involved, more than one person able to edit at once, and the fact that a bad prompt sends real emails to real clients.

The decisions

I wrote the technical design first and locked a handful of decisions before any code:

The frontend never talks to the agent directly. The backend sits in the middle, owns the session and proxies the chat stream. One place to authenticate, log and rate-limit.
The backend is the system of record. Sessions, drafts and versions live there, not in the agent and not in the browser.
One editor at a time. An edit lock per prompt, so two admins can't overwrite each other.
Saving means publishing a version. Every save is a new version you can see and restore, so a bad change is one click to undo.
The prompt text is the structure. No hidden form fields that drift from what the model actually reads.

What I built

The backend: authoring sessions with edit locks, a streaming (SSE) chat proxy to the agent, publish, version history and restore, draft previews and test sends.
The frontend wiring, and integrating a teammate's stacked UI work into one coherent flow.
A push for a validation and rollout milestone. The original plan was all build and no proving, and I argued that something that sends emails needs a deliberate rollout.

After launch

Shipping was half the work. I led the hardening pass:

gating the feature properly;
making test sends safe, so they only ever go to the person testing;
fixing signature and preview correctness;
caching previews, so opening the page no longer fired an LLM call every time.

What I took away

With LLM features, the model is rarely the risky part. The risky parts are state, concurrency and blast radius. Deciding where state lives, and making every change versioned and reversible, is what made this safe to hand to non-engineers.