In-App Surveys Got 189 Responses. My Cancel Emails Got Zero.

Quick build-in-public update on my iOS app. Over the last few months I've tried four different ways to get GainFrame users to tell me what's broken and what to build. In the last six weeks, in-app surveys collected 189 responses. In roughly the same window, 116 automated "why did you cancel?" emails collected zero replies. Not a low number. Zero.

I think talking directly to users is the single most useful thing a new app can do. Analytics shows you what people did; it never tells you why, and it definitely never tells you what you're blind to. So I kept adding channels until something worked. Here's the full scoreboard, including the embarrassing rows.

In-app chat: good, but not the firehose I hoped for

The "Ask Mike" profile. Real human, real reply.

Back in April I added a Crisp chat behind an "Ask Mike" button and wrote about why. Ten weeks later: 74 people have opened the chat screen and 30 started an actual conversation.

The conversations I do get are great — bug reports with screenshots, feature asks, a couple of people just saying the app helped them. But it didn't move the needle as much as I hoped, and it produced one problem I did not see coming: GainFrame also has an AI coach you can chat with, and some users assumed "Ask Mike" was the same thing. They'd send the founder chat a question about their training and wait for the AI to answer. A human answering a day later, when they expected an instant bot, is a weird experience for everyone involved.

Keeping it anyway. Thirty real conversations is thirty more than I'd have otherwise, and the support tickets alone justify it.

Cancel emails: 116 sent, zero replies

This one I had real hopes for. When someone cancels a trial or a paid subscription, RevenueCat fires a webhook, the event lands in a database table, and a cron job emails them within about 15 minutes asking one question: what could I change to keep you around?

The pipeline works flawlessly. 74 trial cancellations emailed, 42 paid cancellations emailed. Replies: zero.

In hindsight I get it. Someone who just cancelled has already spent their last unit of attention on you, and answering "what could I change" is homework for a product they've decided to stop using. The people most able to tell you why they're leaving are exactly the people with the least reason to bother. I'm leaving the automation on because it costs nothing, but I've stopped checking that inbox with any optimism.

The feature board nobody found

In June I set up a Featurebase feature-request board and linked it from Settings. Four weeks later it has zero posts.

I don't think the idea is dead — I think I buried it. A voting board behind Settings → Feature Requests is invisible to anyone who isn't already digging around, and people who dig around in Settings are not usually in a "let me write a product suggestion" mood. It needs an entry point at the moment someone hits a wall, not a parking spot three taps deep. That's on me, not the users.

What worked: surveys that show up inside the app

The channel that finally produced volume was the least personal one. In late May I wired up PostHog's in-app surveys, restyled them to match the app instead of using the default popover, and targeted each one at a specific moment:

• Dismissed the paywall in the last week → "What's holding you back from trying Pro?"
• 10+ scored photos → "Does your physique score feel accurate?"
• Came back after 7+ days away → "What brought you back?"
• Just finished onboarding → "How easy was it to get started?"

Six weeks, 189 responses. For comparison, that's more feedback than every other channel combined, times three.

The trick that makes it work for me day to day: every submitted survey also posts to a Slack channel with the full context attached — device, iOS version, app version, free or Pro, photo count. My phone buzzes, I read it in five seconds, and when an answer needs a follow-up I can look the account up and reply personally. Feedback arrives where I already live instead of in a dashboard I have to remember to visit.

And the answers changed what I'm working on. The paywall survey has a 38% response rate — 61 answers from 160 people shown — because it fires five seconds after someone dismisses the paywall and takes one tap to answer:

The score-calibration survey hurt more. Of the users with 10+ scored photos who answered, 58% picked "it's all over the place" to describe their AI physique score. Two people out of twelve said it matches how they look. That's the kind of thing no analytics funnel will ever surface — the feature works, people use it, and a chunk of them quietly don't trust the number. Score consistency moved way up my list because of one survey question.

The dashboard was undercounting my responses

One honest caveat about the winning channel, because it nearly misled me.

While writing this post I noticed the PostHog surveys dashboard showed numbers that felt low — one survey showed 4 responses when I could see fresh ones in Slack from the day before, and another showed a flat 0 against 50 real submissions.

Because every response also writes to Slack and to my own database at the moment the user taps Send, I could line the two up. The analytics pipeline captured 88 responses. The direct pipes captured 189.

The why is mundane mobile-SDK stuff. The Slack and database writes are plain HTTP requests the app fires directly at submit time. The analytics event goes through an SDK callback chain into a batched queue that flushes later — and someone who just answered a survey popup very often closes the app in the next few seconds. Three of my surveys are also rendered natively and were never wired to PostHog's survey events at all. None of this is really PostHog's fault, but the lesson stands either way: I only knew the dashboard was undercounting because the same data flowed through a second, dumber pipe. If a number matters, send it down two paths and let them argue.

What I'd copy if I were starting again

Two of my four channels flopped and one undercounted itself by half. Still the best time I've spent on the app this quarter. The roadmap I had in May and the roadmap I have now are different documents, and users did that. Slower than guessing, but I'm no longer building for an imaginary person.