What to Measure in an Automation Pipeline: A Practical Guide to Analytics and KPIs

Useful automation pipeline analytics answer a small number of specific questions: how much is flowing through the pipeline, how much of it is real automation work versus disguised system fixes, how much value the qualified backlog represents, and where the pipeline is concentrated or thin. A dashboard that shows dozens of numbers but can't answer those four questions in under a minute isn't doing its job, regardless of how sophisticated it looks.

Intake-Level vs. Portfolio-Level Metrics

It helps to separate two very different kinds of analytics. Intake-level metrics describe a single opportunity: its score, its ROI estimate, its complexity tier. Portfolio-level metrics describe the pipeline as a whole: total intakes, completion rate, average automation potential across the backlog, and estimated annual ROI summed across every qualified report. Automation teams need both, but they answer different questions for different audiences - intake-level detail for the team doing the work, portfolio-level rollups for the sponsors funding it.

Core Metrics Worth Tracking

• Total intakes and completion rate - a simple volume and conversion health check on the top of the funnel.
• Average automation potential across analyzed intakes - a single number that tracks whether the pipeline's overall quality is rising or falling over time.
• Estimated annual ROI, aggregated across all qualified reports - the portfolio's total addressable value, useful for budget conversations even before any of it is built.
• Root-cause mix - the split between genuine automation candidates, disguised system fixes, and unclear cases - which tells a CoE how much of its apparent pipeline is actually buildable.
• Pattern distribution - how candidates split across the seven automation patterns, revealing whether a program is over-indexed on one pattern (often RPA) relative to what the data actually supports.
• Department and business-unit roll-ups - where pipeline value concentrates and where coverage gaps exist.

Metrics to Be Suspicious Of

1. 1Raw intake count with no completion or quality filter - it rewards volume of submissions, not quality of the pipeline.
2. 2Cumulative hours saved with no baseline - see Why 'Hours Saved' Misleads Executives for why this specific metric collapses under scrutiny.
3. 3Bot count as a standalone KPI - it measures activity, not value; the fuller argument is in Measure Outcomes, Not Bots.
4. 4Average score across the backlog with no distribution - a healthy-looking average can hide a bimodal pipeline of a few excellent candidates and a long tail of weak ones.

Making Analytics Refresh Automatically

Analytics are only trustworthy if they reflect the current state of the pipeline, not a snapshot from whenever someone last pulled a report. A live dashboard that recalculates as new intakes complete - rather than a periodically exported spreadsheet - keeps the numbers a CoE reports to leadership synchronized with what's actually in the backlog at the moment someone asks.

Frequently Asked Questions