Define a baseline, measure the complete task and agree a decision before a pilot begins.
Write the decision first
“Explore AI” cannot be evaluated meaningfully. “Can assisted tagging reduce review effort without missing urgent requests?” identifies a task and a risk. Write the decision the pilot must support before selecting a model. Otherwise a convincing demonstration can quietly replace evidence as the definition of success.
Observe the existing process
Measure a representative sample of current work, including correction, waiting and escalation. Use the same completion definition for the pilot. Measuring only the step AI accelerates can overstate the benefit if another team must spend longer checking the result. Preserve enough anonymised examples to repeat the comparison after changes.
Separate important errors
Missing an urgent request has different consequences from applying an optional tag incorrectly. Report those errors separately. Track review effort, escalation and rejected suggestions alongside accuracy. A fast draft may be valuable, but the complete reviewed result still needs to meet the standard the business relies on.
Agree when to stop
Set a review point, budget boundary and decision owner. Continue when evidence supports a useful improvement; revise when failures are understood and fixable; stop when the task does not justify the cost. Keep the evaluation set and failure examples so future changes can be assessed against the same baseline.
Your next working session.
- Define the investment decision
- Measure end-to-end effort
- Include a stop criterion
Bring one real, anonymised example to the discussion. Work through the choices above with the person responsible for the outcome, record the unresolved questions and agree who will investigate them.
Elite Apps editorial · Practical planning guidance. Examples and diagrams are illustrative; implementation depends on your requirements. Technical and supplier guidance can change.
