The nine prompts behind HurricaneWise

What's around the model

All nine are versioned files, and CI validates them.

The nine jobs

I split the work up mostly so I could test each prompt by itself. If one of them gets worse I can swap it out and the other eight stay as they are.

Structured output

Five of the nine return structured JSON, and it's validated against a schema before anything downstream uses it. The postmortem writer returns prose and the resolver prompt drives a coding session. For the storm read the schema wants a one-sentence summary, a confidence level (high, medium or low) and a synthesis. The text fields have length limits and extra fields aren't allowed.

If an answer is missing a field or has a bad value, it gets rejected and nothing is published for that storm. Text that runs too long is handled differently. Since 31 August 2026 it just gets cut to length, because one long field doesn't make the rest of the record wrong.

The schema can only check shape, though. For content there are grounding rules (next section), and the temperature is set low, 0.1 to 0.3 where the model takes one, so the same input gives basically the same read. Each prompt also says which model tier it runs on. Heavy reasoning goes to a capable model and the short proximity sentence goes to a small fast one.

Rules in the prompts

The prompts spell the rules out. No speculating past the data. Region tags have to come from the allowed list. The storm gets called by its real name and never by its tracking code, and the timing stays exactly as the source gave it. And if one of the two source documents is missing, the model summarizes what it has and leaves the gap alone.

Pretty much every one of those went in after I caught the output wrong. The tracking-code rule is from a stretch when the site showed Tropical Storm Cristina as EP032026. That was actually a database bug and I fixed it, but I left the instruction in the prompt anyway, and there's a test that pins it.

The model also has to say what it doesn't know. Every storm read has a confidence level and a short list of the actual uncertainties, and it compares the official forecast with the spread of the other models, including where they disagree. Eleven models go in, the official forecast and two consensus aids among them, and HurricaneWise runs no weather model of its own.

Reading the discussions

For the basin summary I don't want the official discussion reworded sentence by sentence, and the prompt says so.

It has to work out why a weather feature matters, so it can tell a routine trade wind from something that's actually developing. The all-basin prompt reads both documents, orders what it finds by what matters to a person instead of by basin, and drops the noise. When a basin is quiet its section is just short.

Plain English for homeowners

NHC writes its discussions for meteorologists. So the last step in every prompt is translating for a homeowner: plain language, any jargon that can't be avoided explained in the same sentence, and coordinates like 12N28W turned into places people actually know.

The tone is pinned too. The prompt asks for reassuring but honest, and its "never catastrophize" line sits next to a rule against downplaying a storm.

Incidents use three prompts

The first time I handled an outage I used one prompt, and the status page it wrote wasn't something a customer could use.

So now an incident starts two prompts at once, and a third one runs when it resolves.

The user notice is what the public sees. It says what a person can and can't do right now and what still works, and it leaves out service names, error codes and anything about the infrastructure. It's aimed at somebody in the storm's path who wants to know if the forecast on the page is still good.

The engineering report is for an engineer. That prompt works out the most likely root cause from the failure pattern and gives it a severity from a written scale. It also gathers the evidence and puts direct links into the right log view in the report, so whoever picks it up can start there.

The postmortem gets written last, after the service recovers. It's blameless and covers what broke and why, plus the detection gap and what would prevent a repeat. Every resolved incident gets one because the project's constitution requires it.

The five failures in The phantom storm are quoted from those postmortems.

Prompts as versioned files

Each prompt is a file, and its header declares the model tier, temperature, token budget, version number and last review date, plus the schema its output has to match if it has one. If I change a prompt I bump its version. If I change a schema I update the validator in code too. CI fails the build when a prompt goes missing or stops resolving.

That's in the constitution as well (prompts live in one place, versioned and validated), and the coding agents that work on the system have to follow it too.

No visitor input reaches any of the nine prompts. The weather prompts read official government feeds and at most a computed value, like your distance from a storm. The incident, cost and repair jobs only read the system's own evidence.

Upgrading the model doesn't change the prompt files, so the "never catastrophize" line is still there afterwards at the same version number, until I edit it myself.