Nine specialists, one contract each

The model is the commodity

People assume the interesting part of an AI product is the model. For something like this the model is the commodity. What makes the output trustworthy, useful and cheap is everything wrapped around it: how the work is divided, what the model is permitted to say, and what happens to an answer before a reader sees it.

More of this product's behaviour lives in the prompts folder than in the application code around it.

The nine jobs

Splitting the work this way is the same discipline as small functions. Each prompt can be tested in isolation and replaced without touching the others. A vague mega-prompt that does everything is the prompt equivalent of a thousand-line function. It demos beautifully and disintegrates in production.

Make it fill in a form

Every one of those prompts returns structured JSON, never prose, and that JSON is validated against a schema before anything downstream is allowed to trust it. The storm read has to carry a one-sentence summary, a confidence level restricted to high, medium or low, and a synthesis, with hard length caps on the text fields and no additional fields permitted.

If the answer comes back off-contract, the code rejects it and publishes nothing for that storm. It fails closed. During a hurricane, a blank space is a better outcome than a confident wrong sentence, and that trade is not close.

This is the single most useful technique I know for making a language model dependable. Stop asking for open-ended text. Require a fixed shape with named fields, allowed values, and nothing extra.

A schema only guarantees the shape though, never the truth. A confidently wrong answer can be perfectly well-formed. Keeping the content honest is a separate job, done by grounding rules and by a temperature held between 0.2 and 0.3 so the same input produces a steady read rather than a creative one. Each prompt also names the model tier it needs, so the heavy reasoning runs on a capable model and a cheap proximity sentence runs on a fast small one. Both of those are choices made per job, not defaults inherited from an SDK example.

The honesty rules, and where they came from

The prompts say, in plain terms: do not speculate beyond the data. Use only region tags from the allowed list, never invent one. Refer to the storm by its real name, never its internal tracking code. Preserve the timing the source gives, exactly. If one of the two source documents is missing, summarize only what you were given and do not fill the gap.

Not one of those was designed in advance. Each marks a place where the output was quietly worse than it looked. The rule about tracking codes exists because for a stretch the site showed Tropical Storm Cristina as EP032026. That one turned out to be a database bug rather than a model failure, but the instruction stayed anyway, and a test now pins it. Cheap insurance against a class of small dishonesty that erodes trust faster than an outage does.

The model is also required to admit what it does not know. Every storm read carries an explicit confidence level and a short list of the actual uncertainties, and it compares the official forecast against the model spread rather than papering over the places they disagree. The posture is simple and it is stated on the site: interpret the professionals, never fake a certainty nobody has.

Reason, do not extract

A weaker version of this product would summarize by extraction. Lift a few sentences from the official discussion, reword them, publish. These prompts are told to do something harder.

The basin briefing is instructed to understand the significance of a weather feature rather than copy the text, and to tell a routine trade wind apart from genuine development. The all-basin briefing is more pointed still: synthesize rather than concatenate, reason across both documents, decide what deserves attention, order what you surface by what matters to a person rather than by which basin it sits in, drop the noise, and do not pad to look balanced.

That is judgment, requested inside tight factual rails. It is the difference between a summary you can act on and a wall of text you have to read twice.

Then translate it

The source material is written by meteorologists for meteorologists. The audience is somebody deciding whether to move the patio furniture. So the last job in every prompt is translation: plain language, any unavoidable jargon explained in the same sentence, coordinates turned into recognizable places rather than 12N28W, distances made relatable.

Tone is pinned down too. Reassuring but honest, never catastrophizing, never minimizing. That is a safety rule more than a style one. During a storm the wrong emotional register is its own kind of misinformation, and it is far too important to leave to whatever the model feels like that day.

An incident is at least three jobs

Here is the part that surprised me. I assumed handling an outage was one task. It is at least three, and the moment I tried to do it with a single voice the result was bad in a very recognizable way, like a brilliant engineer who has never spoken to a customer writing the status page.

So an incident now triggers two specialists at the same instant, with deliberately opposed briefs.

The communication specialist writes what the public sees. Its whole brief is restraint: name the specific thing a person can and cannot do right now, say what still works, and never mention a service name, an error code, or anything about the infrastructure. The standard it is held to is the homeowner in the storm's path who needs to know whether the official forecast on the page can still be trusted.

The technical commander writes what only an engineer will read. Its brief is the opposite: be deep and specific, reason from the failure pattern to the single most likely root cause, assign a severity from a written scale so that "bad" is a defined word here, and gather the evidence. It drops deterministic deep links straight into the right log view, so whoever picks it up lands on the evidence instead of hunting for it.

The postmortem writer arrives last, when the service recovers, and produces the blameless write-up. What broke, why, the detection gap, the prevention. Every resolved incident gets one. That is a law in the project's constitution, not a habit I am proud of keeping.

That third one is why this project has an institutional memory at all. The five failures in The phantom storm are all quotes from it.

Prompts are governed data

Each prompt is a file with a header declaring its model tier, its temperature, its token budget, a version number, the date it was last reviewed, and the schema its output must match. Changing a prompt means bumping its version. Changing a schema means updating the validator in code. The build fails if a prompt goes missing or stops resolving.

That started as my discipline and is now the project's law: prompts live in one governed place, versioned and validated, which binds the agents working on this system exactly as it binds me. None of it would survive a year of changes if these lived as strings scattered through the codebase, so they do not.

One rule sits above all nine. No free text a visitor types ever reaches any of them. Their inputs are official government feeds plus, at most, a computed value like your distance from a storm, so nothing a stranger typed gets a vote in what the prompts do.

The models will keep getting better on their own. The contract, the grounding rules, and the judgment about how to say a frightening thing to a frightened person do not arrive for free with the next release. Someone has to decide that "never catastrophize" is a rule and write it down. On this project, that decision and every rule like it is mine.