Statistics & Experimentation Interview Questions

Karthik T. - "It's mainly an experimentation technique for testing new features while the rest of the users are using the old product version of your product. In our case, we were using it for pre-release or announced release features for a specific group of users. We could at any point revert the experience or stop the feature and render the old product version of the product. Based on the success of the feature, we will then do a full rollout of the feature into production. How does it work ? Enable"See full answer

Sangeeta P. - "A/B testing is used when one wishes to only test minor front-end changes on the website. Consider a scenario where an organization wishes to make significant changes to its existing page, such as wants to create an entirely new version of an existing web page URL and wants to analyze which one performs better. Obviously, the organization will not be willing to touch the existing web page design for comparison purposes. In the above scenario, performing Split URL testing would be beneficial. T"See full answer

Farza S. - "It is the smallest level of significance at which the null hypothesis gets rejected"See full answer

Lucas G. - "To speed up A/B tests results with limited sample sizes, we can apply advanced techniques like CUPED to reduce variance for faster statistical significance, interleaving to gather more comparative data per user (e.g., ranking), MAB to dynamically allocate traffic to winning variations for quicker optimization (e.g., campaigns), and Bayesian A/B testing which offers probabilistic conclusions that can be reached earlier. Each method, when appropriately applied, allows you to gain m"See full answer

Lucas G. - "Because testing many engagement metrics at once increases the risk of finding effects that aren't real (the 'multiple comparisons problem'), you must adjust your criteria for statistical significance. For social media data, the Benjamini-Hochberg procedure is often a practical choice as it controls the rate of false discoveries (FDR) while still allowing you to detect genuine changes; however, the ideal adjustment method will vary depending on your specific number of metrics (e.g., use Bonferron"See full answer

Connor W. - "This video is a duplicate of the other video in this lesson, "Design A/B test for New Campaign""See full answer

Himani E. - "The cases where data is under heavy outlier influence. Since mean fluctuates due to the presence of an outlier, median might be a better measure"See full answer

Sangeeta P. - "Marketing campaigns are run through different channels such as social media, emails, SEO, web advertising, events, etc. Let’s look at some of the overall success metrics at a broader level: Total views for your campaign Unique views for your campaign Returning visitors for your campaign Engagement for your campaign (If it’s a social media campaign, the marketer might be interested in knowing the number of users engaging with the campaign and the type of campaign positive/negative) 5"See full answer