Five Essentials of Effective Metrics

I ran across this post I wrote on another blog where I used to write about project management. The original post is from February 2007, two months before I first downloaded Tableau.

Reading it again, the message about effective metrics still holds true today.  I've edited the post a bit to reflect data analysis projects rather than project management. At the time, I ran across a white paper that summarized the things we need to keep in mind for a metrics program. The paper didn't specify the metrics to collect, just the properties they should all have.

From my perspective, for any metric to be useful, it needs to help the stakeholder make decisions. All metrics should be actionable. If it's not actionable, then it's not useful.

Introduction

Information should be made available to all stakeholders throughout the lifecycle of a product. To be effective, metrics must be properly planned, managed and acted upon. What is measured, how it’s collected and how it’s interpreted are the difference between brilliant insights and blind alleys on the path to metrics-driven decision making.

The key is to ensure metrics are meaningful, up-to-date, unobtrusive, empirical and actionable.

#1 - MEANINGFUL

Metrics should focus on simple and fundamental units of measure for the given project. Understanding the key metrics across a portfolio of products can provide an important level of insight that enables organizations to understand opportunities and risks. It also provides a uniform basis for comparison across products, time, etc. Select metrics that will enable you to steer your company in a meaningful way.

#2 - UP-TO-DATE

It is important to look for metrics that can be captured automatically. Ensure that the metric is consistently based on up-to-date data.

#3 - UNOBTRUSIVE

The process of collecting data for your metrics program should be seamless and unobtrusive, not imposing new processes or asking stakeholders to spend time collecting or reporting on other data to get the answers to their questions.

#4 - EMPIRICAL

Metrics solutions should capture updated data as soon as reasonably possible, eliminating all of the issues that compromise the integrity and accuracy of data. Additionally, the use metrics that ensures data consistency; e.g., an working hour should be normalized to be the same in Boston, Bangalore, Mumbai and Beijing.

#5 - ACTIONABLE

It is critical that the metrics you gather inform specific decisions. Avoid information that is nice to know, but doesn’t help you make decisions or solve problems.

The litmus test for any metric is asking the question, “What decision or decisions does this metric inform?” Be sure you select your metrics based on a clear understanding of how actionable they are and be sure they are tied to a question you feel strongly you need to answer to effect the outcome.

It is also critically important to ensure that you are able to act on and react to metrics in a clear and meaningful way.

Finally, be sure that metrics are inclusive and that data is available to all stakeholders. Data that is widely available is empowering.