Each week we break down a real experiment from the dating apps you use — and run the same playbook for clients who'd rather ship winners than guesses.
We didn't invent this discipline. We're just translating the best of it for the dating-product world.
Our success is a function of how many experiments we do per year.
More experiments, more money. If you are slow, you get eaten.
Within hours the new headline triggered the "too good to be true" alert. The change had increased revenue by 12% — over $100M a year. It was the best revenue idea in Bing's history, and almost no one believed in it before the test.
The $100M headline that almost died in the backlog.
In 2012 a Microsoft engineer on the Bing ads team proposed moving a few words of ad text from the secondary line into the headline. The idea sat in the backlog for months. Nobody believed in it.
When a developer finally implemented it on a slow afternoon, revenue spiked so abnormally fast it tripped the "too good to be true" alarm — the team assumed it was a tracking bug. It wasn't.
The change lifted ad revenue by 12%. In US dollars: more than $100 million a year, in the United States alone. It became the single highest-ROI feature in Bing's history.
The lesson isn't about ad copy. It's that the team's best estimate of which ideas were worth shipping was off by orders of magnitude. Ship the test. Your intuition is not as good as you think it is.
Source: Ronny Kohavi, formerly GM, Analysis & Experimentation, Microsoft BingWe run the same playbook for dating, social discovery, and community apps — fractional design + experimentation leadership.
We picked three this week. All three changed something small and watched something big move. Read each setup, guess the winner, then see how badly your gut lied to you.
Free and paid users both see the Boost button. The question was whether to sell it on time pressure or on crowd behavior.
"Boost starts in 12 min — peak hours nearby." Clock ticking down, static after that.
"38 people used Boost near you in the last hour." Number updates live, no clock.
A countdown to peak hours is a promise about the future — it asks the user to come back later, which is exactly the moment you lose them to a different app. The social-proof counter is a claim about right now: other people, near you, already paying for an edge you don't have yet. That's FOMO with a body count attached, not a scheduling problem.
The countdown also implicitly told users Boost was worthless outside peak hours, which killed off-peak purchases entirely — the social-proof version had no such ceiling.
Bumble's photo verification badge boosts trust and reply rates. Nobody disputes that. The fight was over when to ask.
Verification lives in Settings → Profile, surfaced once via a dismissible banner three days after signup.
"Add a quick selfie to get verified" prompt fires the second the primary photo upload completes, before onboarding continues.
Verification asked later competes with a dozen other settings nobody visits. Verification asked right after the first photo rides the same motivational wave that got the user to upload a photo in the first place — they're already thinking about how they look to strangers.
The completion dip is real, but it's the users who were never going to verify anyway; the ones who stayed were higher-intent from minute one, and that showed up two weeks later in retention, not on day one.
Standouts (Hinge's daily curated picks) is partly free, partly paywalled. The team argued about whether to show users what they're missing.
"Unlock unlimited Standouts" card with feature bullets, no preview of who's behind it.
Three blurred profile cards, each tagged "picked you" plus a running count, with "Unlock to see who" beneath.
This is the same open-loop mechanic that wins on every "who likes you" paywall — showing the shape of what's missing (a blurred face, a specific number) creates a debt the brain wants to settle, where a feature-bullet list is just information you can decide to skip.
The retention lift among people who didn't convert is the more interesting number: even a rejected paywall can do work if it leaves an open question behind instead of a closed pitch.
Adapted from Ton Wesseling's customer-behaviour-study framework. We don't run highlights from press releases — every test passes all five.
GA4 funnels, heat maps, scroll maps, session recordings. Where users actually drop, click, and bail.
Support tickets, in-app surveys, feedback widgets, moderated interviews. The story behind the numbers.
The team's internal knowledge base of wins, losses, and inconclusive results. Don't re-run yesterday's experiment.
Peer-reviewed behavioural science, competitor monitoring (Visualping, the optimizer plugin), industry benchmarks.
Mission, vision, KPIs. A test that wins on a metric the business doesn't care about is a test that loses.
Each one is a real test pattern run by Hinge, Bumble, Tinder, or one of the smaller niche apps. Click to vote.
Bumble gives women 24 hours to message first. The question was whether to put a ticking clock on it.
A loud countdown sits on top of every new match.
Same 24-hour rule, but the clock stays out of sight.
A free user hits the daily like limit. What you put on that screen decides whether they pay, leave, or come back tomorrow.
Show when free likes refill, with a quiet "or get Gold" link below.
Straight to the Gold offer, no countdown in sight.
Hinge has voice prompts. Almost nobody records one. So they tried pushing it during setup.
Setup encourages a quick voice recording before finishing.
Standard setup. Voice prompts stay buried in settings.
Same screen, same price, same CTA color. Only the headline changed.
Headline: "See who already likes you"
Headline: "Get unlimited likes & 5 Super Likes a day"
The "See who likes you" headline activates an unresolved curiosity loop — there are people who already like you, and you can know who. The value-stack version is rational and complete, which actually closes the loop before payment.
This is the same psychology behind unread notification badges and Hinge's "Who you've liked" tab. People will pay to resolve unfinished business about themselves.
More content = better profiles. But does it cost completion?
Users must answer 4 prompts before they can swipe.
Users must answer 3 prompts; rest are optional later.
Every additional required field in onboarding is a tax on commitment. The match rate didn't drop, which means the marginal 3 prompts weren't carrying real signal — they were carrying friction.
The bigger surprise was D7 retention. More users got to first-swipe faster, which is the only state where the product actually has a chance to hook them.
Helping users write first messages should boost reply rate. Right?
Composer pre-filled with an AI-suggested opener about the match's profile.
Blank composer with a soft placeholder. User writes their own first message.
The pre-fill increased the volume of sent messages but tanked the reply rate. Recipients could pattern-match an AI-written opener within two lines — the formulaic "I saw you love X, what's the best Y" structure is now widely recognized as a bot tell.
Authenticity is the actual scarce good in dating product UX. Anything that makes one user's effort feel cheap reduces the other user's reason to respond.
Modal interruption is bad UX. Except, sometimes, when it isn't.
Match triggers a full-screen takeover with both faces and a "Send Message" CTA.
Subtle toast at the top of the swipe deck. User keeps swiping uninterrupted.
The fewer-swipes cost was real — the modal interrupts the swipe flow. But the message-send lift more than compensated. Emotional moments (a match) need a visual peak; without it, the match becomes just another notification in a stack of 40.
This is the same principle behind Duolingo's streak fireworks. Don't optimize away the celebration — it's the part of the loop people return for.
Free users see one Super Like per day. What's the best way to label that button?
Label: "Send Super Like (1 free today)"
Label: "Stand out — try Super Like"
The "1 free today" framing does three things at once: it removes the "is this going to cost me?" uncertainty, it triggers loss aversion (use it or lose it), and it educates the user that the resource is replenishable. The benefit framing did none of these and required a second cognitive step to evaluate.
Once users tried it once, packs became plausibly worth buying. The CTA was a wedge for the entire Super Like monetization line.
Trust signals should lift conversion. So why did mentioning "safe" do the opposite?
Listings include direct phrases like "Really safe hostel", "Safe neighbourhood".
Same hostels, but safety is implied via cleanliness, 24/7 staff, neighbourhood — never named.
Behavioural research established years earlier that safety is the most important attribute for hostel bookings — guests sleep in 8-to-16-bed dorms with strangers. So the team did the rational thing: surface safety. And conversions dropped.
The word "safe" is a stop word. Naming it explicitly activates the question it's meant to answer — is this place unsafe? Implicit signals (excellent cleanliness, 24/7 staff, wonderful location) carry the same trust content without lighting up the worry. People infer safety; they don't want to be reassured of it.
Story originally documented by Ton Wesseling, Online Dialogue.
Two CTAs. Same intent. One pair is mismatched in verb family — the other reads as one system. Does that 1mm of polish move the needle?
Primary CTA: "Register". Secondary: "Log In". Two verb families.
Primary CTA: "Sign Up". Secondary: "Sign In". One verb family, mirrored weight.
"Register" and "Log In" come from two different mental dictionaries — one is administrative ("register a vehicle"), the other is conversational. Pairing them forces the user to do a half-second translation: is "Log In" the partner of "Register", or did I miss the right button? That hesitation costs you returning users who tap the wrong CTA and bounce back to the home screen.
"Sign Up" and "Sign In" share a verb stem and a rhythm. The eye reads them as a matched pair before the brain finishes parsing. The lift isn't coming from new users converting better — it's coming from returning users no longer mis-tapping. The ~15% drop in wrong-button taps is the real story behind the +7.2%.
This is what we call the golden detail. It's not a redesign. It's two words. But on an auth screen — the highest-traffic screen in the entire product — every micro-friction compounds across millions of sessions. Terminological consistency isn't polish; it's a load-bearing piece of the funnel.
We synthesised thousands of support tickets, product reviews, and operator interviews into the five biggest retention questions — and what the data actually says.
The #1 driver of cancellation is finding a relationship — about 30% of churn. Users who match and enter a relationship naturally graduate from the product. The second biggest driver is match quality or volume (27%): users who don't see enough relevant profiles, or whose matches don't lead to conversations, leave within days. Poor onboarding, notification fatigue, and paywall shock round out the top reasons.
The key insight: not all churn is bad. "Success churn" — users who leave because the app worked — is your best marketing engine. The dangerous churn is "failure churn" — users who leave because the app didn't work. Your retention strategy should separate the two and fight failure churn aggressively.
As of 2026, the US dating-app benchmark is roughly 12.4% monthly churn1, which translates to about 79.2% annual retention. Top-quartile apps (Hinge, Bumble in strong cohorts) push monthly churn below 10%. Apps in the bottom quartile see 18–22% monthly churn.
But the real number to watch is Day 7 retention2. If you retain 35%+ of new users to Day 7, you have product-market fit in the dating category. If you're below 25%, your onboarding or matching algorithm is likely broken — fix that before optimising anything downstream.
Most drop-off happens in the first 48 hours. The highest-ROI interventions are:
The principle: the first week is not about feature depth. It's about proving value in the shortest possible time. One match, one conversation, one dopamine hit — that's the retention hook.
Yes — it's the second-biggest churn driver after "found a relationship." Users who rate their matches 4+ stars on a 5-star scale have 2.3x higher 30-day retention3 than users who rate them 2 or below. The problem is that "quality" is subjective: what one user considers a great match, another considers irrelevant.
The most effective approach is perceived relevance, not objective compatibility. Apps that explain why a match was suggested ("You both love hiking and indie music") see 15–20% higher conversation rates — even when the underlying algorithm hasn't changed. Transparency builds trust, and trust keeps users swiping.
Win-back campaigns have a narrow window: users are most receptive in the first 30 days after churn. Beyond 90 days, open rates drop below 2% and you're better off spending that budget on acquisition.
The most effective re-engagement tactics, ranked by win-back rate:
The golden rule: never say "we miss you." Lead with new value, not nostalgia. Users didn't leave because they forgot about you — they left because the product didn't deliver. Show them something changed.
Two directors who scaled CRO across 19 dating domains and 120+ web properties. Now available to your team.
14+ years architecting design and CRO programs across 19 dating domains, validated at near-100% statistical confidence. Led product design including a 10-step paywall-optimized onboarding flow for a production dating product. Specializes in design systems, multivariate testing, and AI-integrated design workflows.
Spent 13+ years scaling a portfolio of 120+ dating web properties and 5 mobile apps as Director at a major dating-vertical company. Led 40+ person cross-functional teams across engineering, design, performance marketing, and monetization. Managed $300k+ monthly operational budgets across concurrent product initiatives.
We run the same playbook for dating, social discovery, and community apps — fractional design + experimentation leadership. Tell us what you're working on and we'll reply within 48 hours.