Less is more: Metric directionality

Tue Feb 14 2023

Matt Garnes

Software Engineer, Statsig

For most measurements we make in product development, we want the value to go “up and to the right.”

Daily active users, revenue, clicks on “add to shopping cart,” etc. Within Statsig, we celebrate these increases when they are statistically significant with a deep, satisfying, green color:

statsig green

When up isn’t good

But what about when this isn’t the case?

You might want to measure things that are not always better when they go up, like

  • the count of crashes in your app

  • removals of items from a shopping cart

  • page loading time

For all of these, if we see significant increases, we want everyone using the platform to recognize that this is not desired.

In Statsig, we make this easy now from the detail page for any metric. Just click into the overflow menu and look for “Set Metric Directionality.”

error count

From there, simply choose whether an increase or a decrease is desired for this metric, and confirm!

setting metric desirability statsig

Real-world example: Performance improvement

We ran into a case like this at Statsig when we ran an experiment to load content from the next page when your cursor hovers on the link, since you might visit there imminently. After collecting data for several weeks, we saw that we significantly reduced the time it takes to load a page (avg_time_to_first_paint)!

Yet, this was displayed in a harsh red color, giving the impression that this effect isn’t what we want. At a glance, people might overlook the meaning of the metric here and draw the wrong conclusion.

statsig red metrics

After setting the directionality of avg_time_to_first_paint to “Decrease,” that effect is now illustrated in the satisfying green color it deserves!

statsig metrics green

You can use Statsig to measure all kinds of things you care about, through all of the ups and downs of product development!

What will your team measure?

Get a free account

Get a free Statsig account today, and ping us if you have questions. No credit card required, of course.
an enter key that says "free account"

Try Statsig Today

Get started for free. Add your whole team!
We use cookies to ensure you get the best experience on our website.
Privacy Policy