Calculate probability for user retention

Last updated: November 5, 2025

Quick Overview

Given the following scenario about user conversion, calculate the the sample size needed.

Goldman Sachs

Analytics & Experimentation

Data Scientist

Goldman Sachs

November 5, 2025

Data Scientist

Technical Screen

Analytics & Experimentation

Medium

311 solved

Given the following scenario about user conversion, calculate the the sample size needed.

This analytics question from Goldman Sachs's Technical Screen tests your ability to think critically about data. The interviewer expects you to consider confounding variables, selection bias, and the difference between correlation and causation.

What the Interviewer Expects

Design a rigorous experiment with proper randomization and sample size calculation
Define primary and guardrail metrics with clear rationale
Address novelty effects, network effects, and interference
Segment results appropriately and identify heterogeneous treatment effects
Propose follow-up analyses when results are ambiguous

Key Topics to Cover

Novelty and primacy effects

Segmentation and heterogeneous effects

Metric definition and success criteria

Network effects and interference

Long-term vs short-term metrics

How to Approach This

Define success metrics carefully. A good metric is measurable, actionable, and aligned with business goals.
Run experiments long enough to account for novelty effects and weekly seasonality.
Use funnel analysis to identify where users drop off for maximum optimization impact.
Segment results by key dimensions (platform, country, user cohort) to catch hidden patterns.
Consider network effects and interference between treatment and control groups.

Possible Follow-up Questions

What if the experiment shows a positive short-term effect but you suspect a negative long-term impact?
How would you handle seasonality in your experiment?
How would you handle an experiment where the control and treatment groups are different sizes?
How would you handle interference between treatment and control?

Sharpen Your Skills on Codemia

Practice similar problems with our interactive workspace, get AI feedback, and track your progress.

Browse Analytics Questions

Sample Answer

Problem Setup

To assess user retention, we need to calculate the sample size necessary to detect a statistically significant difference in retention rates between users exposed to a new feature and those who are no...

Methodology

We'll use the formula for sample size calculation for comparing two proportions:

$n = \frac{(Z_{\alpha/2} + Z_{\beta})^2 \cdot (p_1(1-p_1) + p_2(1-p_2))}{(p_1 - p_2)^2}$

Where:

( Z_{\alpha...

Submit Your Answer

Markdown supported

Goldman Sachs Data Scientist Interview Guide

Interview process, tips, and preparation timeline