Statistics & Experimentation Interview Questions

Vineet M. - "The distribution of daily minutes spent on Facebook per user is heavily right-skewed with a long tail. Most users spend a short amount of time while a smaller segment of heavy users push up the average with 2–3+ hours daily."See full answer

Mark S. - "Range captures the difference between the highest and lowest value in a data set, while standard deviation measures the variation of elements from the mean. Range is extremely sensitive to outliers, it tells us almost nothing about the distribution of the data, and does not extrapolate to new data (a new value outside the range would invalidate the calculation). Standard deviation, on the other hand, offers us an insight into how closely data is distributed towards the mean, and gives us some pr"See full answer

Sangeeta P. - "A/B testing is used when one wishes to only test minor front-end changes on the website. Consider a scenario where an organization wishes to make significant changes to its existing page, such as wants to create an entirely new version of an existing web page URL and wants to analyze which one performs better. Obviously, the organization will not be willing to touch the existing web page design for comparison purposes. In the above scenario, performing Split URL testing would be beneficial. T"See full answer

Saurabh K. - "P(A) = 0.6 P(B) = 0.4 P(D|A) = 0.05 P(D|B) = 0.03 Question asks to solve for P(A|D) P(A|D) = (P(D|A) x P(A))/P(D) = (0.05 x 0.6)/(P(D|A) x P(A) + P(D|B) x P(B)) = (0.05 x 0.6)/(0.05 x 0.6+0.03 x 0.4) = 30/42 = 5/7 = 0.714 Notice above that P(D) = P(D|A) x P(A) + P(D|B) x P (B)"See full answer

Sangeeta P. - "Marketing campaigns are run through different channels such as social media, emails, SEO, web advertising, events, etc. Let’s look at some of the overall success metrics at a broader level: Total views for your campaign Unique views for your campaign Returning visitors for your campaign Engagement for your campaign (If it’s a social media campaign, the marketer might be interested in knowing the number of users engaging with the campaign and the type of campaign positive/negative) 5"See full answer

Yenenash W. - "p- value --> Assuming the null hypothesis is true, probability of observing the data as extreme as the observed data. Example - You're running an experiment on a new checkout flow. Control converts at 10.0%, treatment converts at 10.8%. You run a two-sample z-test and get p = 0.03. What this means: If there were truly no difference between control and treatment (null hypothesis), there's only a 3% chance you'd see a difference of 0.8pp or larger just from random sampling noise."See full answer

Dessalew A. - "R² → goodness of fit (but can mislead) Adjusted R² → better for model comparison Multicollinearity → hurts interpretability, not always prediction T-test vs F-test → individual vs overall significance Weird case (F not significant, T significant) → likely multicollinearity or instability"See full answer

Lucas G. - "To speed up A/B tests results with limited sample sizes, we can apply advanced techniques like CUPED to reduce variance for faster statistical significance, interleaving to gather more comparative data per user (e.g., ranking), MAB to dynamically allocate traffic to winning variations for quicker optimization (e.g., campaigns), and Bayesian A/B testing which offers probabilistic conclusions that can be reached earlier. Each method, when appropriately applied, allows you to gain m"See full answer

Lucas G. - "Because testing many engagement metrics at once increases the risk of finding effects that aren't real (the 'multiple comparisons problem'), you must adjust your criteria for statistical significance. For social media data, the Benjamini-Hochberg procedure is often a practical choice as it controls the rate of false discoveries (FDR) while still allowing you to detect genuine changes; however, the ideal adjustment method will vary depending on your specific number of metrics (e.g., use Bonferron"See full answer

Lucas G. - "I would conduct a sample z-test because we have enough samples and the population variance is known. H1: average monthly spending per user is $50 H0: average monthly spending per user is greater $50 One-sample z-test x_bar = $85 mu = $50 s = $20 n = 100 x_bar - mu / (s / sqrt(n) = 17.5 17.5 is the z-score that we will need to associate with its corresponding p-value. However, the z-score is very high, so the p-value will be very close to zero, which is much less than the standa"See full answer

Connor W. - "This video is a duplicate of the other video in this lesson, "Design A/B test for New Campaign""See full answer

Saurabh K. - "Ask a follow up question : What is the primary goal of expanding into a new vertical ? Food vertical company may want to expand to a new vertical (say Grocery) for the following reasons : Attract new customers interested in grocery delivery instead of food delivery Increase usage/order frequency from existing customers Increase revenue and LTV of existing as well as potentially new customers Benefit from synergies between existing delivery engine by improving utilization of their network"See full answer

Yenenash W. - " A higher confidence level leads to a wider interval because we are more certain that the parameter of interest lies within that range. Conversely, a lower confidence level results in a narrower interval, but it also means we are less confident that the interval contains the parameter of interest.""See full answer

Katherine B. - "If we’re using an A/B test we have a few decision criteria that we can use to measure success. If our primary metric has been shown to be statistically significant (and our confidence interval does not cross 0), and the gaurdrail metrics that we created have not been negatively affected, we should consider shipping. If the our p-value is not significant we can still consider shipping beta if the guardrail metrics have not been negatively affected, and we weigh the opportunity cost of not shippin"See full answer