Riddle Me This, Riddle Me That: Are You Using the Right Metrics or Just Chasing Stats?
In 1995’s Batman Forever, Jim Carrey's Riddler taunts Bruce Wayne (Val Kimer RIP) with the iconic line:
"Riddle me this, riddle me that…"
It's a fitting metaphor for product management because let's be honest, figuring out which metrics matter can feel like solving a riddle. There's no shortage of data to pull, dashboards to build, or charts to share. But knowing when to use a metric and when to walk away from it takes more than analytics black belt. It takes judgment.
Think of it like Kenny Rogers' "The Gambler":
You got to know when to hold 'em,
Know when to fold 'em,
Know when to walk away,
And know when to run…
Good product leaders treat metrics like poker hands. Some are strong bets, customer retention, lifetime value, engagement scores. Others are bluff-worthy, vanity metrics dressed up to impress stakeholders but offering little insight. And then there are the ones that used to be useful but no longer reflect the story your product is telling.
Used wisely, the right metrics give you clarity, alignment, and traction. Used poorly, they'll lead you in circles while making you think you're making progress. The difference? Knowing the game, you're playing and what counts as a winning hand.
The Winning Hand: Playing Metrics Like a Pro
In poker, not all hands are created equally. A royal flush is rare—and when you have it, you go all in. The same goes for product metrics. Some are powerful, tied directly to business outcomes. Others might look good but won't help you win. The trick is knowing which metrics are your aces and which are just noise.
Royal Flush – North Star Metrics
These are your game-changers. Metrics like Customer Lifetime Value (CLTV), Net Revenue Retention (NRR), or Activation Rate reflect true product health and long-term growth. When you're confident in these, you double down. They align the team, shape roadmap priorities, and give executives something they can bank on.
Play these when: You need to focus the org on a single, unifying outcome that drives compounding value over time.
Illustrative example:
Imagine a streaming service focusing on "Time Spent Listening" as their north star metric. This approach aligns product, marketing, and content teams around a single goal that directly correlates with subscription retention and revenue growth. When they see this metric climbing steadily quarter over quarter, they know their personalization algorithms are working.
Full House – Engagement + Retention Combo
Separately, engagement and retention are solid hands. But together, they show if users love your product and keep coming back. Think of daily active users (DAU) paired with feature adoption or cohort retention curves.
Play this hand when: You're evaluating product-market fit, planning growth investments, or testing the stickiness of new features.
Illustrative example:
Consider a B2B SaaS platform that notices high initial engagement with their new reporting feature but poor 30-day retention. This combination reveals that while users are curious about the feature, it isn't delivering lasting value. After interviewing customers, they revamp the reports to include actionable insights, not just data. Six months later, both engagement AND retention improved by 40%.
Two Pair – CAC + LTV
Customer Acquisition Cost (CAC) on its own is good. So is Lifetime Value (LTV). Together, they help determine whether your business model is sustainable. If LTV outweighs CAC, you're in good shape. If not, you're bleeding chips and don’t even know it.
Play this hand when: You're assessing marketing efficiency, pricing models, or investor-readiness.
Illustrative example:
Picture a DTC brand spending heavily on influencer marketing, celebrating their rapid customer growth. When they finally calculate their CAC:LTV ratio, they discover they're spending $50 to acquire customers who only generate $45 in lifetime profit. They quickly shift the budget to retention campaigns for existing customers, and within two quarters achieve a healthier 1:3 ratio.
The Bluff – Vanity Metrics
Pageviews. Social likes. Downloads without context. These can be dangerous if they're all you're watching. Like a bluff, they might fool others (or even yourself) into thinking you're winning. But when it's time to show cards, they fall flat.
Fold these when: You're trying to make strategic decisions. They're fine for awareness campaigns but don’t bet the roadmap on them.
Illustrative example:
Think of an app startup boasting about crossing 1 million downloads in their investor updates. Meanwhile, only 3% of those users are active after day 7, and revenue is stagnant. The vanity metric masks the real problem: the core product isn't solving a meaningful problem for users.
The Fold – Stale or Misleading KPIs
Some metrics used to matter. Maybe they were useful in early growth stages or during a launch sprint. But if they no longer reflect what success looks like, it's time to walk away.
Fold these when: The product has evolved, the customer base has matured, or you're solving a new problem altogether.
Illustrative example:
Imagine a fintech app initially tracking new account sign-ups as their primary KPI. As they mature, they realize that accounts without deposits are worthless. They fold the sign-up metric and replace it with "accounts with first deposit completed"; immediately gaining clarity on their true conversion funnel.
Reading the Table: Setting a Metrics Cadence That Pays Off
In poker, seasoned players don't just watch their own hands, they read the table. They pay attention to what's happening around them, anticipate moves, and make decisions based on the flow of the game. Product managers need that same situational awareness.
Create a Cadence That Matches the Product Life Cycle
Metrics aren't meant to overwhelm; they're meant to inform. Instead of treating all metrics equally at all times, align them with the rhythm of your product's stage; whether it's a software platform, consumer good, service offering, or something in between:
Early Stage / MVP (Validation Mode)
Weekly – Operational metrics and early signal indicators
- Software: Uptime, error rates, bug volume, user onboarding friction
- Consumer Products: Manufacturing defects, fulfillment speed, early reviews
- Services: Client onboarding time, service delivery quality, NPS from pilot clients
Goal: Validate stability, usability, and initial customer reaction.
Growth Stage (Experimentation Mode)
Biweekly – Experiment metrics and feature/offer performance
- Software: A/B test results, feature adoption, funnel progression
- Consumer Products: SKU-level conversion rates, pricing sensitivity, DTC campaign ROI
- Services: Package performance, client conversion from trials, referral behavior
Goal: Test what resonates. Scale what works.
Product-Market Fit / Maturity (Optimization Mode)
Monthly – Retention trends and behavior analytics
- Software: Retention cohorts, usage frequency, upsell opportunity
- Consumer Products: Repeat purchase rate, subscription renewal, shelf velocity
- Services: Contract renewal, cross-sell acceptance, CSAT/NPS tracking over time
Goal: Understand behavior, identify churn points, and deepen loyalty.
Scale / Optimization Stage (Strategic Mode)
Quarterly – Business health and profitability metrics
- Software: Net Revenue Retention (NRR), LTV/CAC ratio, churn
- Consumer Products: Gross margin, customer lifetime value, return rate
- Services: Revenue per customer, delivery cost per engagement, customer expansion rate
Goal: Ensure long-term sustainability and efficient growth.
Tailor the Table for Your Audience
What your engineers care about isn't what the CFO wants to see. Customize your reporting and storytelling to match the stakeholder. That's how you win trust and get buy-in.
Example of tailoring metrics for different stakeholders:
- For engineers: Show feature usage depth and technical performance metrics like load time and error rates.
- For marketing: Focus on conversion rates, acquisition channels, and campaign ROI.
- For executives: Highlight business outcomes like revenue impact, market expansion, and competitive positioning.
- For the board: Emphasize strategic KPIs that show long-term value creation and market opportunity.
Tell the Story Behind the Numbers
The best metric reviews feel like stories with stakes, not just charts with labels. What's trending? Why? What should we do next? Numbers without context won't drive decisions. Narratives will.
Know When to Walk Away
Don't let outdated KPIs or misaligned metrics steer the ship. If it no longer serves your strategy, let it go. Clear it off the dashboard. Make room for what matters now.
The Metric Evaluation Framework
Before adding a new metric to your dashboard or making a big decision based on an existing one, run it through this quick four-question test:
- Connection: Does this metric directly connect to customer value or business outcomes?
- Actionability: If this metric changes, do we know what actions to take?
- Reliability: Is the data consistently available and trustworthy?
- Simplicity: Can anyone on the team understand what this measures and why it matters?
If you can answer "yes" to all four, you've likely got a metric worth keeping in your hand. If not, it might be time to fold.
The River
Metrics aren't magic. They don't solve problems on their own, and they're not always straightforward. Sometimes they ask questions. Sometimes they mislead. And sometimes when used at just the right moment, they unlock the next move like a well-played hand.
As a product leader, your job isn't to chase every data point. It's to decode the signals, filter the noise, and apply the right measure at the right time. That's the real riddle. And like Kenny Rogers sang:
There'll be time enough for countin'
When the dealin's done.
So the next time someone asks, "What's our metric for success?" you can smile like the Riddler and say, riddle me this, riddle me that, who's afraid of a metric like that?
Playing Another Hand: Read On
If the idea of choosing the right metric at the right time resonates with you, here are two books worth bookmarking:
- Lean Analytics by Alistair Croll & Benjamin Yoskovitz
A practical guide to finding the one metric that matters based on your product’s stage. Whether you're validating product-market fit, optimizing retention, or scaling growth, this book will help you separate signal from noise. - Product Operations by Melissa Perri
An essential read for product leaders building structure around metrics, decision-making, and cross-functional alignment. If you're scaling teams or need to turn data into rituals, this is your playbook.
This post contains affiliate links. If you purchase through these links, I may earn a small commission at no extra cost to you.
