How to work with product churn
There are three ways to grow product: bring more customers, increase the value per customer, or cut churn. It’s simple to measure first two areas; count how much customers joined in the last month, how much customers were lost at every step of the conversion funnel, how many sessions or transactions active customers did on average and what was the average bill. Measuring churn is harder, as in most cases, it's not clear if a particular customer has churned or just dormant. And when we can't measure it, its hard to prioritize the work on it.
In this article I’ll share a simple method that helps to answer the following questions:
- How many customers stop using the product per month?
- What are the main drivers for churn?
- How to estimate retention improvement opportunities?
Challenges with churn measurement
With most products, customers interact irregularly with random intervals between sessions or transactions. Moreover, the product audience usually is a blend of customers with different patterns and frequency of use. For example, imagine someone who has not used Uber for one month, is because he switched to another taxi app or just did not need to use a taxi during that time?
The absence of clarity if customer churned or not makes measurement and work with churn difficult. It’s hard to say how much customers are churning per month, understand the reasons for churn and estimate the size of improvement opportunities. If you can’t estimate the impact of particular retention problem you can’t prioritize work on a solution and put it in the backlog. It’s much simpler to justify another project that will increase conversion by 2% or cut the costs by 5%, but it does not mean that it is the best thing to do for a product team.
Mainstream approaches to churn measurement
There are two widely used metrics to measure product retention: churn rate and cohort-based metrics like Live Time Value (LTV). There is a great book from Amptitude on product retention that describes these metrics in a well-structured way, which I highly recommend.
LTV it a crucial tool for a marketing specialist to evaluate the quality of different acquisition channels, it is also great as a general temperature check. However, LTV hardly can be used as a prioritization tool for retention improvements. It takes a lot of time to measure since we need to gather historical data for cohorts. With the use of historical data and machine learning technics it's possible to predict LTV at an early stage of a cohort or even on per customer basis, but at the current stage, it’s quite complicated and costly.
Churn rate works well for subscription-based products or ones that have regular nature of interactions like, for example, utility payments. It’s clear that customer has churned at the moment when he missed the next planned interaction. When we know which customers have churned we can ask them for reasons and prioritize fixes for them. But for not regular, transactional products churn rate is hard to measure since we don’t know which customers churned and which are dormant.
Usually, it gets addressed by defining some specific timeframe and all customers that have not interacted with the product for longer than this time considered to be churned. For example, we can say that user stops playing the game if he has not opened the game app for a week. The problem with this approach is that it has false positives, in our game example customers who will open and play the game after the 1-week interval will be marked as churned but in reality just had a pause in usage.
There is another challenge with this, defining the time after which we consider the customer as churned. If we select too long time interval we will wait too long before naming customer as churned and miss the opportunity to react. If too we select a shorter time we will have a high level of customers defined as churned false positively. How to define inactivity time after which mark customer as churned? A good time interval for a game or social network might be too short for money transfer service or a shoe store. To define this time, it is helpful to think about retention in terms of conversion to the next transaction or session.
Conversion to the next transaction
Conversion funnels used widely in customer acquisition analysis. We look at all the steps on the customer way to the first transaction, for example, customer acquisition funnel can look something like: learned about the product > visited landing page > registered in the service -> selected the product > made a transaction. Conversion to the second transaction is just the next step of the same funnel.
After customer made her first transfer if she has the repetitive scenario and she enjoyed the service her life in the service continues. She converts to second payment, then converts from second to a third, etc. Looking at retention at this angle allows to use all conversion funnel technics we know understand how much of customers we are losing at each step, and by talking with customers who have not converted, understand why they churned.
Let's build a chart that shows conversion to next transaction over time. I will use transactions below, but it can be any meaningful customer action like a visit, booking, session, etc. To build the chart we should take some significantly large number of transactions, 1000 or more will be good enough. Let’s put days passed since the transaction on X-axis and on Y-axis show % of transactions that had a subsequent transaction after X days. The resulting curve will look something like this.
The point on this chart can be read in the following way: for selected transactions, 60% of them have at least one following successful transaction from the same customer 1 week after the transaction creation date.
Some transaction has consequent ones the same day. Most of them, however, converting in the first few days/weeks. These transactions represent customers with high-frequency use-cases, not much time passing between payments for them. Then the chart growth slows down. Those are the transactions of customers with non-frequent use cases.
Eventually, the chart tends to horizontal asymptote. Transfers above this asymptote will never convert to the next one. They would be the last transfers for the customers who made them. Customers churn after making them. And this gives first practical insight. On our example chart asymptote lies on 90% level, it means that 10% of transactions never convert to the next. All customer that made those payments churned. So you can measure the true churn rate using this chart. Let’s say in our example we have 1000 transactions per month, then 1000*10% = 100 customers churn per month.
This can be used for prioritization if, for example, we will figure out the way to rise this asymptote by 3% to 93% level we will save 1000 * 3% = 30 customers per month by this. This can be compared with the number of new customers brought by improvement in the acquisition funnel.
This chart also helps to understand the quality of the simplest churn prediction model. For example, on our chart, if you wait for 2 and a half weeks, approximately 80% of transactions would have consequent ones. We can mark all customers who had not done the transaction for 2 and a half week as churned. Since asymptote lies at 90% and only 10%, would truly churn. half of the customers we marked as churned would “resurrect” at some point later. So for the product of this type, it will take us 2 and a half weeks to predict churn with 50% false positive rate. Having this chart you can select inactivity time after which you mark a customer as churn, making a tradeoff between false positive rate and delay after churn event.
Retention improvement opportunities
We can get quantitative insights into areas of retention improvement opportunities. Let's say we define customers who have not had a transaction for 2 and a half week. We can survey all customers marked as churned. As we saw earlier 50% of this customers will be “false positive” churned and in reality will still do transaction later. The question we should ask is “I’ve noticed that you have not used product recently, can you tell why?” and provide with the following answer options:
- “I use another product instead.” This will represent actionable retention opportunities. If the customer selected this option you can ask following open-ended question “Why?”. This will be a good way to collect qualitative feedback. Alternatively, you can provide the customer with the product pillars, for example, “It's cheaper”, “it's faster”, “it's more convenient”, “It has a feature that I need”, etc. which will give a good overview of what pillars are most important and require more focus. You can also ask what product they start using instead, this will give a quantitive picture of competition.
- “I don’t have a need to use your product anymore”. This is natural churn, representing customers who lost use-case, for example, for food delivery application the customer could move to another city where the application is not working yet. This usually hardly addressable churn and requires launches in new markets or launching new products.
- “I will use the product soon.” This will be picked by customers who were false positively identified as churned. Note, it's important to add “soon” as people are optimistic and if you do not emphasize that you are interested in the meaningfully short-term interval customers will pick this option even if they don’t have use cases anymore, thinking “I might have this need in the future and then I’ll use this product again”, which will spoil your results.
Note, you need to have a large set of surveyed customers to get statistically significant results.
When you get survey results you can estimate total retention opportunity you have. Let's say in our example we will have 50% of respondents answered that they will use product soon, 25% said that they use another product and 25% more claimed that they don’t have use-case anymore. It means 25% of customers that was marked as “churned” after 2.5 weeks of inactivity actually switched to another product and potentially something we can do to retain them. As in our example, we have 1000 transaction per month it means 1000*5% = 50 customers move to competitors every month. We can call this metric Monthly Churned Users and use it in KPI map and prioritize projects using it. We might also have good insights among answers to the question of why they switched.
I plan to share how this approach can be used to measure the impact of particular events on customer retention later, for example, calls to the customer support, failed transaction attempts, some bad experience, bonus etc. Meanwhile will be happy to answer your questions and learn how the conversion to the next transaction/session chart looks for your type of the product, and if this technique was useful for you.
