Skip to main content

Answering Classic Execution Questions

Premium

Don’t expect to escape traditional execution questions! You may get an execution followup to a more vague prompt, or you might get something more specific like these:

  • "You are about to launch a new app that is of strategic importance for the company. 1 month out from launch internal Dogfood suggests the app isn't ready (you are below target on several key metrics including CSAT). What do you do?"
  • "Facebook is experiencing a 6% decline in user base this month. What could be causing this decline? How would you investigate?"
  • "Walk through a launch day for a new Google search feature. What channels would you use to promote this feature on the day of the launch? Before? After?"
  • "How would you price a new competitor to the Amazon Alexa and Google Home? What factors would be important in your analysis?"

The idea is for interviewers to assess how you approach and solve realistic problems. It helps to have a firm grasp of common metrics, so we’ll review below.

The list of expectations for execution questions is long, but in general, hiring managers want to see you:

  • Isolate variables.
  • Use data effectively.
  • Propose hypotheses that make sense.
  • Think logically.
  • Show “problem intuition.”
  • Come up with a reasonable set of experiments.
  • Pick specific goals for initiatives
  • Pre-define and prioritize metrics that will track success.

There’s no one-size-fits-all approach to execution questions, but we have plenty of execution mock interviews and practice questions coming up.

For now, a quick review of metric-setting and the product lifecycle.

Setting Metrics

We find the GAME framework helpful for thinking through metrics in a systematic way. This was covered in the Analytical course, but it's worth repeating.

  • Goal: Define goals for the product. What is our product vision? Where do we see the product growing in the next five years? For example, if we’re working at Whatsapp, perhaps our overall goal is to see the platform grow in terms of user retention. We want to ensure that users who visit our platform discover useful features and stay on the platform.
  • Actions: What actions do we want our users to take that support these goals? For instance, which features or actions drive engagement? Maybe we want our users to send at least one message per day, since these users often stay on the platform for several months. Another reasonable action to consider here is payment — how many users upgrade their accounts?
  • Metrics: Now that we’ve clarified our actions of interest, let’s discuss which metrics capture these actions. For example, we can track the number of users that upgrade their account to premium, or the number of users who send one message per day for one week. Tracking these metrics helps indicate to us if our users are taking the Actions that align with our product’s Goals.
  • Evaluate: Of course, as with most PM questions, reflect after you deliver your answer. How might this metric be a false-positive or false-negative? What are some cases where your defined metrics may show positive, but cause deleterious effects for the platform? For instance, perhaps users are upgrading their accounts to premium, but then, after a few days, refunding their purchase because of dissatisfaction with the premium account. These false-positives or false-negatives don’t mean that your answer is wrong — it’s just always worth mentioning the tradeoffs and potential areas of concern with your selected metric. In fact, no single metric will ever be foolproof!

Here is a list of common metrics tracked at FAANG companies:

Acquisition

  • DAU
  • Downloads / installs,
  • Product page visits

Engagement

  • Registrations
  • Avg. key actions per session
  • Session length per user
  • Social engagements (likes, comments, shares, etc.)

Retention

  • Churn
  • N-day retention
  • MAU

Monetization

  • Revenue per customer / engagement
  • CPA, LTV

Once you've brainstormed a list, you'll want to choose 1-3 north star metrics to track. Here are a few rules of thumb for choosing the right north star:

  • Only track metrics that impact the business, the customer, or both. Avoid vanity metrics.
  • Know which way you want to go. “Total time spent in-app” might not be good for a dating app, for example.
  • Choose alert thresholds carefully. You don’t want to field calls in the middle of the night due to natural variations in usage! Consider trailing averages rather than raw data.
  • Identify leading and lagging indicators, and try to measure leading. Leading KPIs can help you make changes early on, when you have more time to course-correct.

Remember to include guardrail metrics to ensure you’re not optimizing one metric at the expense of something else important. For example, say you want to increase advertisement revenue on a social feed. You’d probably experiment with increasing # ads per page, but you’d want to track CSAT as a guardrail metric to ensure that the increased ads aren’t detracting from the user experience.

Next Lesson