Lean Analytics Cycle: an important tool for Startup’s decision making
A Lean Startup is an enterprise that must constantly face uncertainty while building a product to change the world, but following an only gut feeling most of the time will bring a company to disaster and failure. One aspect of Lean Startups that are trying to lower the probability of failure is the use of strategic metrics. In Figure 1 we can see that “Measure” is part of the constant process of a Lean Startup in trying to find the right product to sell.
Strategical choices like if keep persevering or changing is based on specific data that we are measuring. What to measure and what not will decide if we are spending energies in the right direction. Said so, picking the right metrics is crucial. Before listing the good metrics, we need to specify which are the characteristics of a good metric:
- A good metric is comparative: comparing recent data with old data is more meaningful than” conversion of 2%”.
- A good metric is understandable: something understood is easy to remember and discuss.
- A good metric is a ratio or rate: ratios are preferred because they are easy to understand and easy to use as a comparative element.
- A good metric is reactive: it will change depending on how you behave.
Once we know which are good metrics, we can pick our metrics by keeping in mind five differences:
- Qualitative vs quantitative Qualitative are hard to understand but give more insight, quantitative, instead, involves numbers and statistics but provide fewer insights. Startups should discover aspects of their business qualitatively and prove them quantitatively. An example is when Airbnb sent out photographers to test if having better photos will improve their business, the result was that the control group received more bookings and they made the decision to add photography as a new service for all hosts.
- Vanity versus actionable Vanity metrics will make you feel good but don’t change how you act, instead, actionable metrics will help you to act. A Startup should not focus on vanity metrics like number of users, number of logins, or downloads since those will not tell anything about the important aspect of the information, like what the users did after login, or if after downloading the app they have used it.
- Exploratory versus reporting Exploratory metrics will let discover important unknown insight and lately can bring you to strategical knowledge. Reporting metrics will guide you through day-to-day operations.
- Leading versus lagging Leading metrics will give you a glimpse on the future, and lagging metrics are telling you something about the past. Focusing on Leading metrics will let a start-up to have time to act.
- Correlated versus casual Metrics that change together are correlated, but metrics that change only because of other metrics are called casual.
The Lean Analytics Cycle (as shown in Figure 2) is about finding the important metric with those characteristics in mind. Picking a metric, it’s like drawing a line in the sand, we experiment with changes to see if the results will cross that line. At the end of each experiment, we will know if we have crossed that line, or we must draw a new line, try again, pivot, or stop.
Video:
