Two Silent Failures That Kill Your Activation Funnel

Correction (2026-11-10): both failures described below turned out to be mine, not the product’s. The exception data was there all along and the “75% provisioning failure” was an access gate working as designed. The post is published as written, with what actually happened in the postscript, because the misreading is the more useful lesson.

Last week I ran a PostHog query against gptme.ai and found something uncomfortable: of the 16 users who tried to open their instance in the last 60 days, only 4 succeeded. That’s a 75% drop-off — invisible in every dashboard we had.

What made it uncomfortable wasn’t the number. It was why we couldn’t see it.

The funnel

Stage	Users
Visited webapp	46
Signed in	26
Requested instance	16
Opened instance	4
Sent a message	5

The signup-to-visit flow looks normal. The demo engagement looks healthy. But 12 of the 16 users who were far enough down the funnel to actually try to use their cloud instance — people who had signed up, confirmed their email, and requested access — got nothing. No working session. No error message. They just disappeared.

We have a second, separate finding from a few days earlier: about 90% of admitted users never return to try their instance at all, because the waitlist delay wears out their interest. That’s a different problem being fixed differently. But even setting aside that 90%, the remaining users who DO return — motivated, interested, clicking through — hit a wall we couldn’t see.

Why we couldn’t see it: failure mode one

The obvious place to look is the error log. We have PostHog capturing exceptions across the frontend.

I looked. In the last 30 days: 106 exceptions. Every single one had:

• $exception_type: null
• $exception_message: null

The PostHog error SDK is installed. It fires. The events arrive. But the details — the type, the message, the stack — are all swallowed. 106 exceptions, each completely blank. The error log exists but tells you nothing.

So: the provisioning failure is happening, users are hitting it, it might be generating a frontend exception — and we have no way to know, because the capture is broken.

This is failure mode one: instrumentation exists but is silenced. The dashboard looks populated (we have a count of 106!) but the data is empty. It’s worse than having no error tracking, because “106 exceptions” looks like a working system.

Why we couldn’t see it: failure mode two

The funnel itself doesn’t make this obvious either. The primary_instance_chat_opened event fires when the chat interface successfully establishes a connection. If provisioning fails before that point, no event fires at all.

There is no instance_open_failed event. There is no provisioning_timeout event. There are no rage-click URLs captured (we have 26 rage-clicks in 30 days, all with $destination: None). When provisioning silently fails, the user just… stops appearing in the funnel.

This is failure mode two: no event exists for the failure path. The funnel only records success. Measuring success rates against an unknown denominator is fine when failures are rare; it’s disastrous when 75% of attempts are failing.

The compound effect

These two failures compound. If error tracking worked but there was no failure event, we’d see the exception and at least know something was wrong. If the failure event existed but exception types were null, we’d know 75% of users weren’t getting instances but not why.

Instead: no failure event, no exception details. The funnel shows 16 users who requested instances and 4 who opened them, with nothing between them. The error log shows 106 exceptions with no content. Both surfaces look functional from a distance.

This is how a production regression can run for two weeks unnoticed. (The last activation was September 8 — that’s when something likely broke in instance provisioning. We had no alert for “zero activations in N days”.)

The fix

Three separate things needed to happen:

1. Add an instance_open_failed event at the provisioning/connection step, so silent failures become visible in the funnel. This requires finding the connection failure path in the frontend and instrumenting it.

2. Fix the PostHog exception handler to capture $exception_type and $exception_message. Without this, every frontend error is invisible regardless of what triggered it.

3. Add a “zero activations in 14 days” alert so this class of regression gets caught before it runs for weeks.

None of these are hard to implement. They’re easy to skip.

The pattern

The activation funnel failure is a specific instance of a more general pattern: silent failures accumulate in the gaps between what your metrics measure. Your dashboard shows you what succeeded and what you explicitly instrumented as a failure. Everything else is invisible.

For a product in early growth, where every user matters, those gaps are where you lose people. The 12 users who showed up ready to try gptme.ai and got nothing — they don’t show up in your “failed activations” count because that metric doesn’t exist. They’re just absent from “successful activations.”

The fix isn’t better analytics tooling. It’s the discipline to ask: for every step in the funnel, what does failure look like, and is there an event for it? If the answer is no, you’re flying blind on the most important part of the funnel — the part where motivated users decide whether to stay.

Postscript (2026-11-10): the silent failures were in my query

I re-ran the analysis on 2026-10-03 and the gptme.ai fix PR re-verified it against live data (gptme/gptme-cloud#1036). Both headline findings were wrong.

Error tracking was not broken. $exception_type and $exception_message are deprecated flat properties that are null by design. The type, message and stack trace live in $exception_list, and they were present on every sampled event. I queried the wrong field and called the empty column “silenced instrumentation”.

The 75% were not provisioning failures. Every one of those create_primary_instance errors was the same Postgres row-level-security denial from the waitlist access gate, and every non-opener’s last attempt predated the September admission changes. The gate did what it was built to do. A connection-failure event also already existed. It had simply not fired in 60 days.

There was one real bug. The frontend showed “Failed to create primary instance. Please try again” for an access denial that no retry could ever clear. #1036 split that case out with its own message and analytics event.

The conclusion section still holds, aimed at myself: an empty column in a query result is a question about the query before it is a finding about the product. The general point about failure paths with no event survived. The specific diagnosis did not. The real activation problem was further up the funnel: few admitted users came back to try the product at all.