Product execution questions are one of the most popular questions you’ll encounter during PM interviews, including interviews for Facebook. Even if you're interviewing for an APM position in your early product management career path, you'll likely still have to answer execution questions!
These questions aim to evaluate how you measure success, and distill user problems into a metric that a team can rally around. These questions appear easy at first, but in reality, are challenging because the question isn’t about naming metrics - it’s about first taking an educated guess at the north star goal for the team and quantifying it with a metric that expresses it succinctly.
There are three main execution question types:
These Success, Debug, and Tradeoff questions closely mirror the critical decisions PMs are accountable for on a regular basis. For this memo, we’ll focus on defining success metrics and cover the others in future memos.
It’s easy to fall into a trap and treat it like a math problem to be solved. A candidate might believe there is a correct metric the interviewer is looking for, and try listing every single aspect of the problem, leading to rabbit holes. Your metrics will differ, however, depending on the goals you define and the assumptions you make. The interviewer is looking for your ability to define a reasonable north star goal for a product, and suggest a metric that is aligned with the goal.
Example: Once, I was defining success metrics for a language learning app in an interview, and I suggested weekly active users (WAUs) as our goal metric. The interviewer pushed back on this choice.
My interviewer agreed with my justification. I could have, however, easily gone the other route, and responded with:
Actually, you’re right. Our goal should be to make language learning a daily habit, so I’m going to change the measurement cadence from weekly to daily and measure daily active users (DAUs) instead.
This is why it’s important to be really clear up front about what the product goals are, and the assumptions you are making. Here’s an example on how to do that:
When interviewees are asked to say, Define success for Bumble (an online matchmaking app), candidates will often list 10+ metrics, worried about covering all aspects of the problem. In this case, however, less is more. It’s more interesting to have a discussion on 5-6 metrics than 20-30 metrics. Then, it’s best to end with an opinion on what the one primary metric should be, and 2-3 secondary metrics that aid the primary metric in showing the full picture.
Here’s how to do that in action:
* 1: Use percentiles, which are not skewed like averages are. If some of your users have very fast response times and others have very slow response times, an average response time would not be representative of your users' experience from the data. Read more on our previous memo: The Allure and Trap of % Goals.
While an OK candidate might list general downsides to their metrics, and talk about counter metrics such as cannibalization of the company’s other products or abusive behavior, the best counter metrics are specific to the north star metric you’ve defined. It’s about asking: If we were to over-optimize for our north star, what are ways our decision-making would be flawed?
Metrics questions are hard because you can take them in so many directions, and interviewers are often intentionally vague. They are seeking to understand: does this candidate dive into coming up with metrics (OK interviewee) or do they take the time to define the landscape and the goals (great interviewee)?
It’s less important that you cover every single aspect, and more important to define all the pieces you need to solve the problem. As a product manager, most of the time you’ll have enough resources and people to help you, so it’s less about correctness, than about being able to frame what is success to your team in a way that enables them to frame decisions later on -- and this involves walking through how you think about success.
To read more on how to ace PM interviews and be at the top of your game as a PM, see more of Alexis and Adrienne’s memos here.
Exponent is the fastest-growing tech interview prep platform. Get free interview guides, insider tips, and courses.Create your free account