Product observability is an emerging term that speaks to the desires of PMs, engineers, and data scientists alike.
In the age of constant experimentation, feature gates, partial rollouts, etc., products now have more moving parts than ever. At any given time, users of big apps are delivered an ever-changing configuration of features and experiences.
For those who are unfamiliar with this concept, think back to when some Facebook users had the ‘thankful’ react, and some did not. Before the feature was rolled out to everyone, it was tested in a specific segment, and Facebook made decisions about it based on user experience.
These experiences, and the way users interact with them, are thoroughly measured and scrutinized in order to determine which configuration of features and UI are the best for a business.
This process goes on and on: In the name of relentless experimentation, every metric lift becomes the new standard, and teams reshape their hypotheses, their goals, and try again and again.
This is the epitome of product observability: Imagine a command booth with large windows overseeing a factory assembly line. The command booth has line-of-sight into every piece of machinery, and can measure their effectiveness, make on-the-spot decisions with the pull of a lever, and so on.
What product observability offers, in essence, is visibility and control over a team’s dynamic products and applications.
Product observability is the condition of having insight into all of the features of an application, platform, or software. Technically, product observability is made possible by having a fully-deployed experimentation platform or otherwise leveraging services to monitor, control, and gain insight into features.
Product observability is colloquially used to describe the state of companies that not only have literal visibility into all of their features, but also actively leverage experimentation results to make decisions about the product.
By combining experimentation, product observability, and real-time analytics, companies achieve a 360° view of their product, meaning being able to see (and manage) which features are driving engagement and growth, and which ones are underperforming.
Tempo leverages experimentation behind all of its feature launches to deliver the best home fitness experience possible.
One of the reasons product observability is becoming commonplace at large software companies is because it allows teams to experiment with new features and configurations without risking the stability of their products. Teams can roll out features to a segment of their users, monitor their performance, and then make decisions about whether or not to roll them out to a wider audience.
This ultimately leads to an inherent experimentation culture which affords many benefits including the following:
When experimentation is implicit in a company’s culture, every feature is shipped behind a feature gate. This lets teams build faster and learn faster, as each feature rolled out will produce insights that can be used for future development.
This is a better operating model than even the most well-thought-out product roadmap: As things change (eg market trends, customer behavior, competition, etc.) the roadmap will need to be modified. Built-in experimentation facilitates built-in iteration, which is the only way to assure you’re building product features that stick.
Furthermore, truly customer-obsessed teams learn and adapt based on customer behavior. They build to delight, rather than throw darts in the dark.
Having an observable product allows teams to quickly get answers about the features they release. Especially in larger organizations, where different teams have different objectives and assumptions, having a solid experimentation culture gives all stakeholders something to agree on: “Is this feature’s metric lift statistically significant?”
Another benefit is that teams can navigate through uncertain territory more reliably with data-driven insights. Whether trailblazing through new competitive space or simply responding to macroeconomic factors, receiving fast and definitive answers about every unknown helps teams to stay on course.
Let’s face it, it’s impossible to invent without experimenting. Experiments allow us to achieve a deeper understanding of all types of issues including problems, user behavior, configurations, and more.
Observability lets teams truly be relentless about growth, as individual biases and hunches aren’t included in proper experimentation. By cutting out the nonsense, evaluations are made solely based on results and data, instead of other leading precedents like opinions and guesses.
Take our “sandbox” project for a spin! In this project, we have generated real-time sample data for you to explore and experiment with.
AI technology has been here for years, but the new wave of AI products and features is game-changing. We covered this, and other topics, at the Seattle AI Meetup.
Building a culture of experimentation benefits greatly from things like reviewing experiments regularly and discussing the results.
Thanks to our support team, our customers can feel like Statsig is a part of their org and not just a software vendor. We want our customers to know that we're here for them.
Migrating experimentation platforms is a chance to cleanse tech debt, streamline workflows, define ownership, promote democratization of testing, educate teams, and more.
Calculating the right sample size means balancing the level of precision desired, the anticipated effect size, the statistical power of the experiment, and more.
The term 'recency bias' has been all over the statistics and data analysis world, stealthily skewing our interpretation of patterns and trends.