Separate Assistance from Automation
Classify each candidate workflow by the cost of a wrong output and the effort to reverse it. Drafting a reply to a support ticket is assistance: a human reads and sends, so an error is cheap. Approving a refund or posting a journal entry is automation: an error moves money. Start with assistance, measure the acceptance rate, and only then promote a workflow to automation with a written rollback procedure and audit logging.
This ordering is not caution for its own sake. It produces the labelled data that later automation needs. Every time an operator edits a generated draft, the diff records what good looks like. After a few thousand reviewed examples, that corpus supports fine-tuning or at least a reliable evaluation set, which is far more valuable than a clever prompt written on the first day.
- Assistance: generate, rank, summarise, or classify for a human decision.
- Automation: execute a write with a defined authority limit and audit trail.
- Keep a per-workflow switch so automation can fall back to assistance instantly.
- Log model version, prompt version, and retrieved context for every run.
Ground Answers with Retrieval, Then Evaluate
Retrieval augmented generation works when the retrieval step is measured separately from the generation step. Build a labelled set of questions with the documents that should support each answer, then track retrieval recall at k before touching the prompt. If the correct passage is not in the top five results, no amount of prompt engineering will recover the answer. Chunk on semantic boundaries and store source metadata so results can be filtered by tenant and permission.
For generation, evaluate faithfulness, answer relevance, and citation accuracy on a held-out set of at least two hundred items. Use a rubric and score with both a model judge and periodic human review, because model judges drift with prompt changes. Treat every prompt edit as a code change: version it, run the suite, and gate deployment on scores that do not regress beyond a defined tolerance.
// Retrieval evaluation: recall@k against labelled passages.
const hits = cases.filter((c) =>
retrieved[c.id].slice(0, 5).some((docId) => c.goldDocIds.includes(docId)),
);
const recallAt5 = hits.length / cases.length;
if (recallAt5 < 0.85) throw new Error(`Recall regressed: ${recallAt5}`);Design for Human Review and Escalation
An automation without a review path will eventually produce a confident wrong answer at scale. Route low-confidence outputs, policy-sensitive intents, and any request touching money or personal data to a human queue. Show the operator the retrieved sources, the model reasoning summary, and a one-click correction. Measure the edit rate per intent; a rising edit rate is an early signal that the underlying data or prompt has drifted.
Set authority limits explicitly: a maximum refund value, a maximum discount percentage, a list of blocked intents, and a daily action ceiling per operator. Encode these as policy checks outside the model so a prompt injection cannot bypass them. Every automated write should carry an idempotency key and a link to the run that produced it, so a reviewer can reconstruct exactly what happened.
Control Cost and Latency
Two numbers decide whether an AI feature survives contact with finance: cost per resolved task and p95 latency. A support assistant that costs forty cents per ticket and answers in nine seconds competes with a human at ten dollars and four minutes, but a summariser that costs two dollars per document rarely does. Cache retrieval results, route easy requests to a small model, and reserve the largest model for the hardest fraction.
Latency budgets are architectural. Streaming improves perceived speed without reducing total time, so use it for reading tasks and avoid it for workflows that must return a validated object. Precompute embeddings offline, batch generation requests where the interface allows it, and set a hard timeout with a deterministic fallback. Instrument token usage per feature so cost regressions show up in a dashboard rather than an invoice.
Run Automation as a Product, Not a Pilot
Pilots succeed on novelty and quietly stall when the champion changes role. Treat each automation as a product with an owner, a backlog, a service-level objective, and a deprecation plan. Track deflection or completion rate, human edit rate, latency, cost, and escalation volume weekly. Retire workflows whose usage falls below a threshold for two consecutive quarters rather than carrying them indefinitely.
Governance should be lightweight but real. Maintain a registry of models, prompts, data sources, and evaluation results; review it monthly with security and legal. Document which data leaves the perimeter and under what agreement. The organisations that sustain value from AI automation are rarely the ones with the most ambitious demos; they are the ones that can explain, measure, and switch off what they deploy.