Metrics 101 for Product Managers - A complete guide with examples!

Introduction

Product managers are responsible for the success of a product, from development to launch and beyond. One of the key tools they use to achieve this is metrics, which provide valuable insights into how a product is performing. Metrics allow product managers to make data-driven decisions, identify areas for improvement, and measure progress towards goals. In this article, we will explore the importance of metrics for product managers and discuss the different types of metrics they can use to optimize their product strategy.

Problems faced by Product Managers in defining metrics

• Unable to get  products prioritized because the company’s priority is not the metric they are moving to.
• Chosing the wrong metric for their product and figuring things out later after launching the product.
• Unable to conclusively achieve the success of the product that they have launched.
• Being dependent on other teams to achieve their success metric and have been unable to do so.
• After successfully achieving a metric for the product, they later discover that in the process, they negatively impacted another metric for a different team.

The success of a Product depends on choosing the right metric

The difference between a successful product and an unsuccessful product is the selection of the right metric to measure the success of the product. Without a measurable success metric your product is directionless and hence there is no way to figure out when to take what actions. It’s like a race where one doesn’t know where the finish line is.

What is the product metric?

These are well defined quantifiable and directly measurable entities that determine the progress made by a product and provide input to further improve the product to achieve better results.

Why are product metrics important?

• Product decisions are shaped by metrics
• Metrics facilitate communication and collaboration with stakeholders to prioritize objectives
• Metrics enable problem identification and corrective action
• Feedback loops are created through metrics for launched products
• Product success is acknowledged and celebrated across the organization with metrics

What are some product decisions dependent on metrics?

• Estimation of the potential impact of the product
• Prioritizing amongst conflicting initiatives by normalizing products to a common metric
• Scoping user stories within an initiative based on the highest impact first
• Defining the features of the product based on the metrics to move

What are the different types of products?

There are two different types of products on which metrics can be defined and they are as follows :

• New Product Inception Metrics
• Optimizations/Additions on top of existing product metrics

Piece of Advice: If you don’t have the right measurable metric, you should not build the product, and conversely if you chose to build a product ensure you have the metrics in place to measure success.

Ask the right questions to get your metric?

Primary Output Metric

Question — When will you call your product a success or a failure?

Why is this metric important?

This is the most important measurable outcome of your product which establishes the true impact of your product. The reason why this is important :

• This is the most important key result for the product and significantly helps to align stakeholders to start product development.
• Over a period of time product managers should be able to judge the success and failure of the product based on this metric and also figure out if additional product investments are required
• Before building the product PM’s should estimate the change in the success metric so that they can set their targets accordingly
• This metric should be defined in such a way that it is measurable and can be tracked on a regular basis. (if possible should be traceable without an experiment)

Eg: Monthly ad revenue (See Case Study Below for Details)

Input metrics

Question — What will be the important milestones your product will reach before it becomes an overall success?

Why is this metric important?

These metrics help to segregate the monolithic success metric into various independent secondary success metrics which are functions of different components of the product. The reason for the importance of this metric is as follows :

• Assign the success metric for each of the stakeholders to ensure proper accountability
• Use these metrics as milestones to ensure that the primary output metric is right on track.
• Break the monolithic output metric equation as a function of dependent input variables to achieve the output metric

Eg: Click Through Rate * Cost Per Click, Fill Rate of Ads (See Case Study Below for Details)

Check metric

Question — Are you screwing up some other metric of the company while building your product?

Why is this metric important?

These metrics prevent the moving round and round in a circle paradox where the net value the product delivers to the company is zero. The reason why this is important :

• It helps product managers to think more holistically about the general good of the company rather than only their own stream.
• Also since most metrics are interconnected and improvement in one metric may lead to the degradation of the other and hence this fact should be known while making product decisions.

Eg: Orders/discovery session on Swiggy (See Case Study Below for Details)

External Dependent Metric

Question — What is the most important external metric which the product does not control and which is required for your product to attain the desired primary output metric?

Why is this metric important?

The success of the product is also dependent on the right inputs and if these right inputs are not achieved the desired outcomes are not reached. This metric can be of two different types :

• Type 1: The input metric is not dependent on your product and is a function of the market forces which is beyond your control.
• Type 2: The input metric is a dependent metric and is either the outcome metric of another product.

Eg: Total monthly impressions on ads (See Case Study Below for Details)

Best Practices of defining metrics

Few Practices to define metrics :

• Make your primary output much independent from other teams as much as possible. (For Eg: at Zomato if a product manager is responsible for bringing discounts from merchants than his primary output metric should be y% of GMV funded as discounts by merchants and not x million $ of discounts)
• Always define the increment in the metric that your product will bring. (For example: Increase metric y from x1 to x2 and not Increase metric y to x2)
• Always define a time period as to when that metric will be achieved. (For example: Increase metric y from z1 to z2 within x months)
• The input metrics can also be attributed to different teams/stakeholders so that there is clear accountability on the product.

Iterative Product Development

This is a simple approach to product development based on metrics. The key points here are as follows :

• Check for external dependent metrics and ensure that they are right or normalize input/output metrics so that the output metrics become independent.
• Iterate the product to improve the input metrics.
• Once input metrics are right after a lag period check the output metrics and they should 99% be right. (If it does not happen you have defined the wrong input metrics)
• Once primary metrics are achieved, then see the check metric to confirm that your product is not screwing up the other metrics.
• If you achieve your output metric within the time period then your product is a success.

Case Study: Let’s apply the framework in three different problem statements

Case 1: Amazon Prime Subscription

Case 2: Netflix Recommendations

Case 3: Uber User-Based Pricing

Change in metrics defines the roadmap

The way product features are added to a product is changed by the metrics in which that particular product is chasing and these metrics for a product change with time leading to different features on the product roadmap to optimize for the newer metrics.

This article is written by Asad Abrar and was orginally published in his medium blog. If you liked reading this and wish to learn from seasoned Product Managers like Asad, then do check out our 10-week Product Managament Launchpad Program.