How to Avoid Vanity Metrics: Getting Under the Hood of Business


Photo by juanillooo

Mental Models Used: ,

Most organizations have analysts reviewing financial and operational information on a regular basis – the objective being to gain some kind of meaning from information, and to capture that meaning with a metric or metrics.  Analysts are generally providing descriptive information (telling us how we’ve done) or predictive information (telling us how we suspect we will do).

But many commonly used metrics don’t provide any actionable insight.  In other words, they’re just for show.  These are called vanity metrics.  Other times metrics don’t properly measure the underlying data, potentially resulting in what only appears to be a valid metric on the surface.  This is called an Isomorphism.

A metric is only as valuable as its ability to decipher underlying data.  When metrics are properly developed and implemented, they become meaningful because they capture the drivers that lead to the behaviors and decisions desired.

A great resource for understanding metrics is the book Lean Analytics.  Although geared to start-ups, the logic used is widely applicable to organizations large and small.  You will find much of this logic in the following paragraphs. 

In an effort to limit confusion and concentrate focus, our search for meaningful metrics should be aimed towards finding the one metric that matters the most.  If we Optimize the organization to Maximize this one metric, it will reveal the next place for us to focus our efforts.  And we continue this process over and over again, improving this one metric (through experimentation) until it is good enough for us to move on the “new” most important metric.

A good metric is usually comparative, understandable, and takes the form of a ratio or rate.  It can be either a lagging (descriptive) or leading (predictive) indicator.  At the outset, we’re stuck with lagging indicators since we don’t have much data to work with.  However, once we do have enough data, we want a mix of both leading and lagging indicators.  And if possible (once we have the data), we want our metrics to be more leading indicator-focused than lagging indicator-focused.  Becoming too past-focused (through lagging indicators) can cause an organization to stagnate.  That hasn’t stopped the typical organizational scorecard from including 80-90% past-focused metrics, though.

Ideally, a scorecard or collection of performance metrics for an organization should consist of about 75% leading and 25% lagging metrics.  The best leading indicators commence at the beginning of the customer lifecycle.  From the very first “touch point” (in marketing-speak), we should be collecting data.  This data will ultimately help us craft the proper leading indicators.

Collecting data doesn’t just lead to better metrics (and hence, better decisions).  It also improves organizational efficiency.  Good metrics can create a flatter, more autonomous organization once everyone buys in to a data-informed approach.  There is no longer the need to propagate decisions across an organization.  We can empower employees to make more decisions themselves, provided they have the data in place to support them.  We can create a culture of responsibility.