I recently attended a 2-day product. The very simple truth I took away from the course is that tracking metrics alone is insufficient. Your team must create a culture of working with the metrics. I learned that, ultimately, metrics will only tell you the next decision to make. But let's break this down a little further.
What value are we trying to create?
Tracking metrics is a great practice, but it won't lead you anywhere. Before you start to look at any data or figure out what you should measure, you must do one thing: Set a clear goal. What is the problem that we're trying to solve? What is the value that we're going to create? Do this upfront. Metrics without a clear goal are useless. So, as with anything, you've got to start with why.
Feature factories are an excellent counter-example. In a feature factory, stakeholders (or basically anyone) can request something. Backlogs are filled with endless inquiries, and these are rarely what anyone needs. That is because goals are commonly focused on time and on a budget. There is little reason to go back and evaluate past projects to see how they performed. Instead of answering the question about the value we will create, feature factories ask themselves: "Did we ship the feature?" That's their number one metric. To be honest, I don't know any business where this has an impact on the bottom line.
Don't be a feature factory; start with a crystal-clear goal. If you tell yourself that "my product is hard to measure," ask yourself why my company funds my team? This will help you find the value you should be looking to drive.
Are we there yet?
The simplest definition of metrics is that they are an agreed-upon way to measure something that matters to us. So, after you define a clear goal, ask yourself: "How can I measure this?" As a PM, your role in metrics is to:
- Understand the data that drives your product.
- Know where to find it.
- Understand how different metrics affect the outcomes you want.
- Structure the team's outcomes in a way that is measurable.
I struggled with this in my first year as a PM. We knew where we wanted to go and what customer problem to solve. However, we didn't have data to answer the simple question: Are we there yet? We didn't build the product in a way that could help measure its own success, we didn't know where to find the data, and I didn't understand how different metrics affected the outcomes we desired.
Developing stuff with no idea of whether we were getting closer to our goal meant we were going in circles. There was no end in sight, which brings me to the last point.
Act (or move on)
When you define a clear goal and understand how metrics will help you track it, there is just one thing left. That is, deciding on what to do next. The ultimate goal is to make better decisions. Metrics only tell you the next decision to make.
Sometimes, this also means to stop. Often, we spend time on stuff that doesn't add value. Once, after successfully deploying an MVP and raising our success metrics to ~95%, I didn't consider moving on. I relentlessly tried to achieve the remaining 5%. I didn't think of the opportunity costs. These last 5% would have made up a giant investment but didn't really improve the overall situation. Sometimes, we spend time on stuff that doesn't add value. This can be fateful! Make sure to question your goal from time to time. Consider opportunity costs and move on.