Calculate variance for click-through rates

Last updated: November 18, 2025

Quick Overview

Given the following scenario about user conversion, calculate the the sample size needed.

Anduril

Analytics & Experimentation

Data Scientist

Anduril

November 18, 2025

Data Scientist

Take-home Project

Analytics & Experimentation

Hard

771 solved

Given the following scenario about user conversion, calculate the the sample size needed.

This analytics question from Anduril's Take-home Project tests your ability to think critically about data. The interviewer expects you to consider confounding variables, selection bias, and the difference between correlation and causation.

What the Interviewer Expects

Design complex experimentation strategies for tricky scenarios
Handle multi-armed bandits, switchback experiments, and quasi-experiments
Address long-term effects vs short-term metrics
Propose causal inference methods when randomization is not possible
Build a measurement framework that connects metrics to business value
Discuss organizational experimentation culture and maturity

Key Topics to Cover

Guardrail metrics

Funnel analysis and cohort analysis

Sample size and power calculation

Segmentation and heterogeneous effects

How to Approach This

Define success metrics carefully. A good metric is measurable, actionable, and aligned with business goals.
Run experiments long enough to account for novelty effects and weekly seasonality.
Use funnel analysis to identify where users drop off for maximum optimization impact.
Segment results by key dimensions (platform, country, user cohort) to catch hidden patterns.
Consider network effects and interference between treatment and control groups.

Possible Follow-up Questions

How would you handle interference between treatment and control?
What if you discover a bug in the logging during the experiment?
How would you handle an experiment where the control and treatment groups are different sizes?
What would you do if a stakeholder wants to end the experiment early because initial results look good?

Sharpen Your Skills on Codemia

Practice similar problems with our interactive workspace, get AI feedback, and track your progress.

Browse Analytics Questions

Sample Answer

Problem Setup

The analytical question we need to address is: "What is the sample size required to accurately calculate the variance of click-through rates (CTR) for a new marketing campaign at Anduril?" To determin...

Methodology

For calculating the required sample size to estimate variance in CTR, we use the formula for sample size in the context of proportions:

$n = \frac{Z^2 \cdot p \cdot (1 - p)}{E^2}$

Where:

Submit Your Answer

Markdown supported

Anduril Data Scientist Interview Guide

Interview process, tips, and preparation timeline