Product Updates

Inline Comments for Dynamic Configs

You asked and we answered! One top feature request has been the ability to add inline comments to dynamic configs to help provide clarity and visibility on field values and definitions.

With commenting, you can add comments to any new or existing dynamic config by escaping with "//". Comments will render in the dynamic config editor, but will not be sent down to your SDK.

Inline comments on a dynamic config

🧮 More Metric Flexibility on WHN

Ratio Metrics: In addition to creating ratio metrics from Metric Sources, you can now create ratio metrics over existing metrics from your metric catalog.

Multi-source metrics: You can now create metrics that span multiple Metric Sources. e.g. If you have setup Online and Offline Purchases as separate metric sources, you can create a metric that sums up purchases from both metric sources.

🌸 Dashboards Spring Cleaning

We've recently made some quality of life improvements to help you find dashboards more easily and quickly derive meaningful insights.

Dashboard Summaries and Settings

When viewing dashboards, we've made it simpler to grasp the most important insights at a glance. Charts on dashboards now, by default, show the latest metric value and the percent change over time for the metric being plotted. This makes understanding current product health more straightforward than ever. You can also edit the summary value being display as the latest value, average value, or cumulative value of the metric over the time range.

Favorite Dashboards

You can now mark dashboards as favorites if they are particularly important to you. Your favorite dashboards will always appear at the top of your dashboard list, making them easier to access and ensuring you quickly get to the data that matters most.

Popular Dashboards

We've also added a new "Popular Dashboards" section to the dashboard list view. This feature makes it easy to discover dashboards that are popular within your project, helping your team share insights and context around the dashboards and product data that are proving most insightful.

We've enhanced our dashboards with new features to improve user experience. Now, charts display the latest metric value and percent change over time, making it easier to understand product health at a glance. Users can also favorite dashboards for quick access and explore a new "Popular Dashboards" section to discover trending insights within their project.

📍 Local Metrics

Local Metrics are metrics that are scoped to an individual experiment or gate rollout. Local Metrics enable you to create the exact specific custom metric you want to measure in the context of your experiment or gate, without having to clutter up your Metrics Catalog on an ongoing basis.

Local Metrics can be created from the Setup tab or the Scorecard (or Monitoring Metrics) sections of your entity, will be calculated for the duration of your experiment or rollout, and then will cease to exist when you make a decision on your experiment or launch your gate.

Rolling out now on Statsig Cloud; on Statsig Warehouse Native in mid-April.

See docs

Local Metrics creation wizard in Experiment Setup tab

📤 Templates

Have a standard rollout you want to leverage across the team? Want to standardize best practices for experiment design across the company? Templates enable you to codify a blueprint for config creation that fellow team members can use to bootstrap their own feature gates and experiments.

Key features of Templates

• Create a new template from scratch from within Project Settings or easily convert an existing experiment or gate into a template from the config itself

• Manage your templates all in one place within Project Settings, restricting which roles on your team have the ability to create and modify templates via Statsig's role-based access controls

• Restrict which templates a given team can select from via "Allowed Templates" settings within team settings

Read more about Templates via our documentation here.

Templates home within Project Settings

🍯Persistent Assignment on Experiments

This is a new SDK feature that makes user bucketing decisions on experiments sticky even in situations they wouldn't have been previously. Users exposed to an experiment are bucketed into Control or Test deterministically (see how); however allocation or targeting changes can cause a user to to be excluded from an experiment after they were exposed to it. Persistent Assignment ensures that users stay bucketed in the experiment even when allocation or targeting changes.

Some scenarios this unlocks

1. You can roll out an experiment to 100% of users for a week, and then drop allocation to 0%. Users exposed to the experiment in that first week will continue to experience the experiment; other users will not.

2. When you target an experiment at set of users (e.g. low engaged users), if the user state changes they usually fall out of the experiment. With Persistent Assignment they will continue to see the experiment (e.g. even if they move from low engaged -> high engaged).

Learn more about Persistent Assignment on Client SDKs, Server SDKs

🚨Alerts++

We’re excited to launch a big update to our rollout alerts product today. Here’s a quick overview of what’s changing:

• Alerts will now have any experiment-level configured stats methodologies applied- If you’ve enabled CUPED or Sequential Testing for your experiment, this will now be incorporated into your alert firing logic post-the first 24 hours of a new gate/ experiment going live (i.e. as soon as daily Pulse results are available).

• Alerts now have confidence intervals attached- Instead of just surfacing metric value relative to a threshold, we will surface metric value with confidence intervals attached, to provide you more context on how seriously you should be concerned about an alert firing.

• Alerts only fire if they are statistically significant- This should drastically reduce the noisiness of alerts on Statsig and ensure you’re only getting pinged with high-signal regressions.

As a reminder, alerts can be set up and configured via the Metrics tab, at the per-metric level. Hop on in, set up alerts for your key regression metrics, and let us know if you have any feedback!

Alert configuration UX

Support for Percentile Metrics [Beta]

We're excited to announce a beta of Percentiles on Statsig Warehouse Native! Percentiles are often used to optimize app performance, understand feature adoption or even manage resource utilization when experimenting on backend infra and AI models.

Percentiles are particularly useful when applied to metrics that exhibit large variances. They also help understand the distribution of a metric, and can be critical to understand outliers or unusual metric behaviors. Customers can now visualize understand impact (or even alert on) p90, p95, p99, p99.9 or any other percentile.

Reach out in Slack if you want to opt into this! If you're interested in the underlying math, we'll be writing about it but it's loosely patterned on the thinking here - Applying the Delta Method in Metric Analytics: A Practical Guide with Novel Ideas.

🔍 Diagnostics 2.0

Today, we’re thrilled to introduce two upgrades to our Diagnostics tab, which enable easier debugging of gates & experiments-

Upgraded Logstream- We’ve added the ability to access longer-term log history, as well as filter by things like rule, reason, experiment group, user properties, and other metadata to enable easily pinpointing the most important logs for your debugging.

New Statsig logstream, with filtering capabilities

SRM Debug Helper

Imbalanced exposures are the last thing you want to see when launching a new experiment, and often seeing this failing health check kicks off a deep-dive into isolating where the imbalance is coming from. We’ve now exposed more detail into the SRM we’re observing, including how the p-value is trending over time, as well as some auto-generated cuts of p-value (e.g. by browser_version, os, region, etc.) to help you isolate where the imbalance may be disproportionately coming from.

P-value broken down by group

🥽 Statsig now supports visionOS

The Statsig iOS SDK just added support for visionOS. You can now use Statsig in your apps for the Apple Vision Pro (iOS SDK versions v1.39.1 or higher).