Executive Summary
Product leads in B2B software companies often have plenty of event data from tools like Mixpanel or Amplitude, but find it hard to say which specific features actually keep users around for the long term.
Engineering and marketing effort is spent on features that don't really affect whether customers stay or go. Without knowing what drives retention, every roadmap decision is a bit of a shot in the dark.
The way forward isn't to track more things or buy a new dashboard. It's about taking a more structured approach: first, defining what retention means for you, then carefully managing the metrics that matter, and getting rid of the rest.
Why tracking every click doesn't always lead to better insight
It's a common belief that if we just track every user event, every click and hover, a perfect product roadmap will somehow emerge from the data. It’s an appealing thought, but in my experience, it doesn't quite work like that.
Most software companies I work with aren't short on data. They're short on insight, often because they have too much of it. You've probably moved to the cloud and hired good engineers, but if the underlying process is messy, all you've done is create that mess more quickly.
What tends to happen is that teams get a bit stuck. They're surrounded by dashboards full of metrics, but the fundamental questions about the business are still difficult to answer.
The common challenges of having too much unstructured data
This is a situation I see quite often, especially with companies that have found product-market fit and are starting to scale quickly. In the rush to grow, it's natural to start tracking everything you can. The side effect is that this can create a fair bit of data debt down the line.
Here’s what this often looks like:
The issue usually isn't the dashboarding tool itself. It's that the business logic is spread out across lots of separate SQL scripts and reports. And it's often not about the speed of your data warehouse, but about the fact that no one has agreed on a single definition for something as basic as 'a retained user'.
How to move from lots of metrics to a clear model
To get out of this situation, the key is to subtract, not add. The solution isn't really a technical one, it's more about structure and strategy. It means shifting from a reactive, bottom-up way of working to a more deliberate, top-down one. This is how we put a better system in place.
Step 1: Define and build a core model
The first step isn't to build more dashboards. It's to ask the questions that are currently hard to answer, like: 'What set of actions in a user's first week makes them most likely to stick around for a year?'. We'd typically look at all the existing metrics and dashboards with the aim of clearing out most of them, perhaps as many as 80%.
The goal is to create a single, agreed-upon set of dashboards that reflects a clear model for retention that everyone in the business understands. This becomes the one reliable place to go for a view of what's driving value.
Step 2: Centralise and document the business logic
Once the model is clear, we put the logic for it in one central place. This is sometimes called a 'semantic layer'. It just means your official definitions for things like 'Churn' or 'Feature Adoption Rate' are written down in shared code (using a tool like dbt or LookML), instead of living in a spreadsheet on someone's laptop or just in an analyst's head.
This way, when someone asks 'What's our retention rate?', the answer is the same no matter who asks or which department they're in. Trust in the data grows because the logic is open for anyone to see and is used by everyone.
Step 3: Help the team use the new system
With a simpler, trusted foundation in place, the teams can do more for themselves. We can set up a self-serve area with reliable, curated data. I often find it helpful to train a few people in the product team to be the local expert who can answer their colleagues' follow-up questions without needing to ask the data team every time.
This changes the data team's job. Instead of spending all their time running basic reports, they can focus on more complex problems, like building predictive models for churn, because the day-to-day reporting is now automated and reliable.
The result: Clearer, more confident decisions
Picture your next board meeting. Instead of spending time debating whose numbers are right, you can show a single chart and say, 'It looks like users who adopt Feature X in their first fortnight are 45% less likely to churn in their first year. Based on that, we're planning to use the next engineering sprint to improve its onboarding.'
This is the difference between being data-rich and being insight-driven. It isn't about getting more data. It's about having the right data, managed properly, and focused on what really matters: keeping your customers.