How to evaluate an AI opportunity

How to decide whether a process needs AI

Choose the least complex intervention that can meet the outcome and control requirements. These categories can be combined: an AI step may sit inside a deterministic workflow. Combining them does not remove the need to evaluate each boundary and failure path.

• AI: interpretation, prediction, or generation

  Consider AI when inputs vary and the task cannot be expressed adequately as stable rules. Require representative test cases, explicit quality criteria, monitoring, and appropriate review. A fluent answer is not evidence of correctness.

• RPA: stable, repetitive system interaction

  Consider robotic process automation when a task follows known rules through existing interfaces and a supported integration is unavailable or unsuitable. Check interface stability, credentials, exception handling, and maintenance burden.

• IDP: extracting information from documents

  Consider intelligent document processing when documents must become structured fields. Test layout variation, field accuracy, missing values, and review paths. IDP may use AI internally; its output still needs business validation.

• Workflow automation: coordination and routing

  Use workflow automation for predictable approvals, notifications, assignments, and handoffs. If structured inputs and rules already determine the action, an AI decision may add uncertainty without benefit.

• Process redesign: remove the cause

  Simplify steps, clarify ownership, change a policy, or fix upstream data when those cause the problem. Reassess technology after redesign rather than automating waste.

• No technology change: retain or observe

  Retain the current process when the problem is not material, the process is about to change, evidence is too weak, or intervention cost and risk outweigh plausible improvement. Record why and what would trigger reconsideration.

Assess value without turning assumptions into ROI

Establish the current baseline and the change being proposed. For effort-related value, a useful starting relationship is eligible annual volume × avoidable minutes per case ÷ 60. That estimates potential hours before adoption, exceptions, review, and support; it is not a cash-savings forecast.

Convert time to money only with agreed cost assumptions and a credible plan for using the released capacity. Distinguish cost avoidance, capacity redeployment, reduced risk, service improvement, and actual spend reduction. Do not count the same benefit in multiple categories or across overlapping opportunities.

Estimate total cost across discovery, implementation, integration, data preparation, model or software use, human review, change management, monitoring, maintenance, and eventual replacement. Use conservative, base, and favorable scenarios and show which assumptions change the conclusion.

Assess production feasibility—not just demo feasibility

• Data and access

  Can authorized teams obtain representative inputs and validate expected outputs? Check permissions, data quality, retention, freshness, and whether the proposed use is allowed. Accessible sample data does not prove production access.

• Process and integration

  Are rules, boundaries, interfaces, and exception paths understood? Check whether upstream and downstream systems can support the change, and identify prerequisite work.

• Operational readiness

  Who reviews low-confidence or incorrect outputs, handles outages, and maintains the solution? Confirm support ownership, required skills, training, and a fallback process before rollout.

• Evaluation design

  Define representative normal, edge, and failure cases with expected outcomes. Agree on acceptance criteria with the process owner. For AI, assess task quality, not just model benchmark performance.

Assess risk by consequence and control

Risk assessment asks what can go wrong, who is affected, how failure is detected, and whether recovery is possible. High potential value does not cancel a legal, safety, security, or policy constraint. Bring relevant control owners into the assessment before a pilot uses sensitive data or takes consequential actions.

• Data and security exposure

  Identify sensitive data, access boundaries, retention requirements, supplier exposure, and the possibility of untrusted inputs influencing outputs. Decide what information may enter the solution and who may access its results.

• Incorrect or unfair outcomes

  Assess who could be harmed by a wrong answer, biased result, or missed exception. Define human authorization points and whether reviewers can realistically detect errors rather than merely approve them.

• Operational failure

  Plan for unavailable services, drift, changed documents, and integration failure. Name a fallback, escalation owner, rollback condition, and monitoring signal. If the consequences cannot be controlled, redesign or stop.

Use an experiment to resolve a specific uncertainty

A useful experiment has a hypothesis, representative inputs, acceptance and stop criteria, a time or spend boundary, and an owner. Test the most consequential uncertainty: extraction accuracy, review burden, data access, integration reliability, or user adoption. Do not run a broad pilot merely because a demonstration looked promising.

Illustrative example: before assisting document review, compare extracted fields with independently checked expected values on authorized samples, including unusual layouts and missing information. Measure correction effort as well as accuracy. This is an assessment design, not a claim about achieved results.

Document the result even if it invalidates the proposal. A failed fit test can redirect the opportunity to workflow automation or process redesign. A successful test supports the next decision; it does not remove production controls or guarantee financial returns.

End with a recommendation and its limits

• Proceed or investigate

  Proceed when evidence supports the next bounded commitment. Investigate when a specific, resolvable uncertainty changes the decision. State the evidence required, owner, and next review.

• Redesign, hold, or stop

  Redesign when the underlying process is unsuitable. Hold for named dependencies or capacity constraints. Stop when the intervention is unjustified or unacceptable, retaining the rationale to prevent repeated rediscovery.

• Do not confuse assessment with ranking

  Assessment establishes whether a candidate is credible and under what conditions. Prioritization compares credible candidates within strategy and capacity. A good use case may still not be the next investment.