Diagnose a drop in NPS

Question

The activation rate dropped by 15% this week. Walk through your investigation process.

Codemia · Accepted Answer

To diagnose the drop in NPS and the associated 15% decline in activation rates, we need to define the problem scope clearly. This involves analyzing the user journey from onboarding to active usage, identifying at which stage users are dropping off. We should segment users into cohorts based on their behaviors, such as new users versus returning users, and analyze feedback specifically related to the onboarding experience, app performance, and product offerings. Understanding whether this drop is a widespread issue or localized to specific user segments is crucial.

The primary users for Instacart are busy professionals, families, and individuals who value convenience in grocery shopping. They are looking for a seamless, efficient experience, especially during onboarding. Key factors for these users include ease of app navigation, personalized product recommendations, and the reliability of delivery times. Given the competitive landscape, with alternatives like Amazon Fresh and local grocery delivery services, Instacart must ensure a superior user experience to retain customers, particularly during their initial interactions with the platform.

I recommend enhancing the onboarding process by introducing a guided tutorial that highlights key features of Instacart, such as shopping lists and personalized recommendations. Additionally, we should implement tooltips and in-app prompts that help users navigate the app during their first few sessions. This solution addresses potential confusion and helps users understand how to maximize their use of the platform, ultimately improving their initial experience and increasing activation rates.

To evaluate the impact of the proposed onboarding enhancements, we should track the following metrics: 1. **Activation Rate** - Measure the percentage of users who complete the onboarding process and make their first order within the first week. 2. **NPS Score** - Regularly survey users post-onboarding to gauge their satisfaction and likelihood to recommend Instacart. 3. **Time to First Order** - Calculate the average time taken for new users to place their first order after onboarding. 4. **Drop-off Rate** - Monitor the percentage of users who exit the app during the onboarding process. 5. **Feature Usage Rate** - Track how often users utilize the features introduced in the tutorial within their first few sessions.

Implementing a guided tutorial may require additional development resources and time, which could delay the rollout of other planned features. Additionally, there’s a risk that a more complex onboarding process could overwhelm some users if not executed well. Balancing the depth of the tutorial with simplicity is crucial to avoid adding friction. It's also important to consider that while enhancing onboarding may improve immediate NPS, we need to ensure that the overall user experience remains streamlined and that we don’t lose focus on other critical aspects of the product.

Diagnose a drop in NPS

Instacart

What the Interviewer Expects

Key Topics to Cover

How to Approach This

Possible Follow-up Questions

Sharpen Your Skills on Codemia

Sample Answer

Problem Framing

User & Market Context