"Prompt:
Define success of reels
How do I investigate if reels engagement is up by x, posts are down by y
Clarification:
What is reels: video experience on explore page, play page. Can have comments, likes etc.
Why was it launched: to promote engagement on the app, 'addictive sessions back to back.
Product goal: Increase in app time, drive stickiness.
Big picture: Instagram launched reels post tiktok style success, way to saving market share
Define success"
Geet B. - "Prompt:
Define success of reels
How do I investigate if reels engagement is up by x, posts are down by y
Clarification:
What is reels: video experience on explore page, play page. Can have comments, likes etc.
Why was it launched: to promote engagement on the app, 'addictive sessions back to back.
Product goal: Increase in app time, drive stickiness.
Big picture: Instagram launched reels post tiktok style success, way to saving market share
Define success"See full answer
"I'd like to start by understanding the basic scope of the change we've observed.
Who is the user? Lyft is a two-sided marketplace so it could be the driver or rider. -Driver
When did the change happen? Was it slow or a sharp drop-off? - sharp, 2 weeks ago
Was there a specific geographic region? Urban v suburban? - no trends
Cancelling can happen at several points in ride acceptance, is there a particular point where we see most cancellations? - just after accepting a ride"
Anonymous Grasshopper - "I'd like to start by understanding the basic scope of the change we've observed.
Who is the user? Lyft is a two-sided marketplace so it could be the driver or rider. -Driver
When did the change happen? Was it slow or a sharp drop-off? - sharp, 2 weeks ago
Was there a specific geographic region? Urban v suburban? - no trends
Cancelling can happen at several points in ride acceptance, is there a particular point where we see most cancellations? - just after accepting a ride"See full answer
"The question is about the decision to show the right # of photos in Facebook.
Clarifying questions :
Is this specific to Consumers or Business users ?
Is there an drop or impact to the current metrics in terms of user engagement or the time spent on the post
Are there any pain points to solve in terms of technical details like page loads etc.?
Let's start with the users. There are 2 categories of users - Creators and Consumers. This can be further classified to say creators - Indiv"
Vasanth M. - "The question is about the decision to show the right # of photos in Facebook.
Clarifying questions :
Is this specific to Consumers or Business users ?
Is there an drop or impact to the current metrics in terms of user engagement or the time spent on the post
Are there any pain points to solve in terms of technical details like page loads etc.?
Let's start with the users. There are 2 categories of users - Creators and Consumers. This can be further classified to say creators - Indiv"See full answer
"The traffic congestion is said to be messedup by vehichles on road.While technically it is said to be a server issue or any other.By using some advanced tech I go to solve that."
Dheeraj sai P. - "The traffic congestion is said to be messedup by vehichles on road.While technically it is said to be a server issue or any other.By using some advanced tech I go to solve that."See full answer
"Problem Statement:
10% of Netflix users are inactive. How would you investigate?
Clarifying Question:
Define inactive users? Assuming these are the users who have not logged Netflix for last week/last month etc
Time span to measure inactive users? Assuming inactive users are inactive from last week.
Investigation Questions:
Has there been a change or error logged on the analytical tool we are measuring "Inactive users" metric?
Does the metric show Netflix inactive"
Satej M. - "Problem Statement:
10% of Netflix users are inactive. How would you investigate?
Clarifying Question:
Define inactive users? Assuming these are the users who have not logged Netflix for last week/last month etc
Time span to measure inactive users? Assuming inactive users are inactive from last week.
Investigation Questions:
Has there been a change or error logged on the analytical tool we are measuring "Inactive users" metric?
Does the metric show Netflix inactive"See full answer
"Clarifying Q
Is the search traffic meaning # visits clicked through the 'search' button from the homepage(search CTR)? or does this include # pages viewed after the search result pages (search depth)?
Hypothesis
I believe introducing Gen AI as a default feature on the Google's search box will increase #search traffic. I assume the biggest user pain point within current google homepage is that deciding on a keyword for search. Unlike other alternative platforms such as"
Cj K. - "Clarifying Q
Is the search traffic meaning # visits clicked through the 'search' button from the homepage(search CTR)? or does this include # pages viewed after the search result pages (search depth)?
Hypothesis
I believe introducing Gen AI as a default feature on the Google's search box will increase #search traffic. I assume the biggest user pain point within current google homepage is that deciding on a keyword for search. Unlike other alternative platforms such as"See full answer
"The North Star Metric for Google Photos should be: Monthly Active Users (MAUs) who successfully store, organize, or retrieve photos/videos.
Why?
This metric reflects Google Photos’ core value: helping users store, manage, and access their memories seamlessly. It ties user engagement with the platform's primary offerings: reliable storage, easy search, and organization.
Supporting Metrics:
User Engagement:
Monthly Active User (MAU) ratio: Growth and retention trends.
"
Brandy L. - "The North Star Metric for Google Photos should be: Monthly Active Users (MAUs) who successfully store, organize, or retrieve photos/videos.
Why?
This metric reflects Google Photos’ core value: helping users store, manage, and access their memories seamlessly. It ties user engagement with the platform's primary offerings: reliable storage, easy search, and organization.
Supporting Metrics:
User Engagement:
Monthly Active User (MAU) ratio: Growth and retention trends.
"See full answer
"Did a quick 30 min writeup on this and I'm submitting before I review/edit.
Success metrics for Spotify Podcasts
Clarify
Focus is on Spotify Podcasts, which are audio content posted by Spotify Content creators
Spotify Podcasts exists across all devices
Spotify as a company is at a mature state and is looking to enhance retention and revenue (monetization)
Spotify makes money through ads, Spotify Podcasts too - Assuming that Spotify Podcasts has some sort of ad product where adv"
Bryan K. - "Did a quick 30 min writeup on this and I'm submitting before I review/edit.
Success metrics for Spotify Podcasts
Clarify
Focus is on Spotify Podcasts, which are audio content posted by Spotify Content creators
Spotify Podcasts exists across all devices
Spotify as a company is at a mature state and is looking to enhance retention and revenue (monetization)
Spotify makes money through ads, Spotify Podcasts too - Assuming that Spotify Podcasts has some sort of ad product where adv"See full answer
"Super helpful video on metrics and experimentation design!
However, I did not see any discussion on the actual algorithm improvement ideas. So it left me with these questions:
1) How would you actually improve the algorithm? (e.g. sorting based on amenities preferences, miles from airport, etc.). Of course, totally makes sense that we would test how that improvement moved our chosen success metrics.
2) Should we be choosing one metric or one set of metrics we want to optimize for when we impro"
Anonymous Crab - "Super helpful video on metrics and experimentation design!
However, I did not see any discussion on the actual algorithm improvement ideas. So it left me with these questions:
1) How would you actually improve the algorithm? (e.g. sorting based on amenities preferences, miles from airport, etc.). Of course, totally makes sense that we would test how that improvement moved our chosen success metrics.
2) Should we be choosing one metric or one set of metrics we want to optimize for when we impro"See full answer
"Clarification Qs:
What does "split" mean here? - split the UI and if so how (through tabs?)
What counts as "media" - clarified already
What is prompting us to consider this change? - any negative feedback from the users/ decrease in certain Newsfeed metrics like engagement etc/ any particular business goal?
is this across or platforms eg desktop/mobile ?
Is this for all regions?
Mission of FB: Fb's mission is to bring people closer together by enabling them to build communiti"
A G. - "Clarification Qs:
What does "split" mean here? - split the UI and if so how (through tabs?)
What counts as "media" - clarified already
What is prompting us to consider this change? - any negative feedback from the users/ decrease in certain Newsfeed metrics like engagement etc/ any particular business goal?
is this across or platforms eg desktop/mobile ?
Is this for all regions?
Mission of FB: Fb's mission is to bring people closer together by enabling them to build communiti"See full answer
"Set Success metrics for Google Maps:
Assuming we are focusing on mobile experience…
Clarify the product:
Overall, it is a directions app:
Gets you from point a to point b
Allows for several methods of transportation - car, public transit, walking, biking, rideshare
After entering location, you click your transport method, and select get directions which leads you to steps to get to you location or if you are driving or walking it speaks to you giving youthe directions aloud based on sett"
Kiley S. - "Set Success metrics for Google Maps:
Assuming we are focusing on mobile experience…
Clarify the product:
Overall, it is a directions app:
Gets you from point a to point b
Allows for several methods of transportation - car, public transit, walking, biking, rideshare
After entering location, you click your transport method, and select get directions which leads you to steps to get to you location or if you are driving or walking it speaks to you giving youthe directions aloud based on sett"See full answer
"I would start with some clarifying questions:
What is the goal of the 'Report ad' feature? what do you think?
Are we looking at the blue app or Instagram? The ad placement can be adjusted to the both apps, so look at it has a whole
Are we looking at a specific location of the report ad? like stories or feed or reels? Nope
Are we looking at a specific OS? everywhere
Is there some timeline? or restrictions? no
Are we looking at a specific market? worldwide
Thank you, I"
R K. - "I would start with some clarifying questions:
What is the goal of the 'Report ad' feature? what do you think?
Are we looking at the blue app or Instagram? The ad placement can be adjusted to the both apps, so look at it has a whole
Are we looking at a specific location of the report ad? like stories or feed or reels? Nope
Are we looking at a specific OS? everywhere
Is there some timeline? or restrictions? no
Are we looking at a specific market? worldwide
Thank you, I"See full answer
"The key here is a) not to overreact to Sales bypassing me going straight to Engineering (while understanding every team acts according to their incentives), and b) communicate with all parties as much as possible to get everyone on the same page.
1) First, I'd speak to Sales and understand the feature being requested. I consider Sales to be one of the most important primary research funnels - they have direct, regular touchpoints with the client and are finely attuned to the lifeblood o"
Sachin P. - "The key here is a) not to overreact to Sales bypassing me going straight to Engineering (while understanding every team acts according to their incentives), and b) communicate with all parties as much as possible to get everyone on the same page.
1) First, I'd speak to Sales and understand the feature being requested. I consider Sales to be one of the most important primary research funnels - they have direct, regular touchpoints with the client and are finely attuned to the lifeblood o"See full answer
"My approach is try to narrow down by 4W1H.
When - When this happened? Still continuously happen the problem?
Where - Is this happen specific region or entire customers?
What - What kind of problem? One time big error or any data show the curve?
Who - Is it all platform or specific platform (Web, iOS, Android)
How - How serious problem? How many customer affected?
Next step is to narrow down the causes is from internal factors or external factors.
For internal factors:
Any recent up"
Takashi M. - "My approach is try to narrow down by 4W1H.
When - When this happened? Still continuously happen the problem?
Where - Is this happen specific region or entire customers?
What - What kind of problem? One time big error or any data show the curve?
Who - Is it all platform or specific platform (Web, iOS, Android)
How - How serious problem? How many customer affected?
Next step is to narrow down the causes is from internal factors or external factors.
For internal factors:
Any recent up"See full answer
"I broke down my answer into Understanding the goal, investigating the current state and its challenges, prioritizing opportunities, brainstorming solutions, and selecting solutions.
To understand the goal and current state, I asked questions about the onboarding funnel and the conversion percentages. They have clear metrics to share, with the specific conversion rates for each stage.
I then identified the subscription screen in the flow as one of the key opportunities, as most of the drop happ"
Jaison E. - "I broke down my answer into Understanding the goal, investigating the current state and its challenges, prioritizing opportunities, brainstorming solutions, and selecting solutions.
To understand the goal and current state, I asked questions about the onboarding funnel and the conversion percentages. They have clear metrics to share, with the specific conversion rates for each stage.
I then identified the subscription screen in the flow as one of the key opportunities, as most of the drop happ"See full answer
"First, let's align the Airbnb homepage with Airbnb's mission. Airbnb's mission is to "create a world where anyone can belong anywhere by providing healthy travel that is local, authentic, diverse, inclusive and sustainable." The homepage serves as the primary entry point for both potential guests and hosts, setting the stage for their journey and embodying this mission.
Now, for the goals of Airbnb homepage, we can consider different aspects like awareness, engagement, and ultimately, driving bo"
Rohit K. - "First, let's align the Airbnb homepage with Airbnb's mission. Airbnb's mission is to "create a world where anyone can belong anywhere by providing healthy travel that is local, authentic, diverse, inclusive and sustainable." The homepage serves as the primary entry point for both potential guests and hosts, setting the stage for their journey and embodying this mission.
Now, for the goals of Airbnb homepage, we can consider different aspects like awareness, engagement, and ultimately, driving bo"See full answer
"You're the PM for all of Instagram Stories. What is your North Star metric?
Meta's Mission: Give people the power to build community and bring the world
closer together.
IG's Mission: To capture and share the world's moments
User Types
Marketplace / 2-sided ecosystem
Creators and the Consumers
Metric Categories
Adoption
Engagement
Retention
[not pictured]: create a table with 2 columns: Creators and Consumers and 3 rows: Adoption, Engagement, Retention.
Talked through and priori"
Daniel L. - "You're the PM for all of Instagram Stories. What is your North Star metric?
Meta's Mission: Give people the power to build community and bring the world
closer together.
IG's Mission: To capture and share the world's moments
User Types
Marketplace / 2-sided ecosystem
Creators and the Consumers
Metric Categories
Adoption
Engagement
Retention
[not pictured]: create a table with 2 columns: Creators and Consumers and 3 rows: Adoption, Engagement, Retention.
Talked through and priori"See full answer
"Product Bugs can be categorised by severity associated with them.
Product Roadmap prioritization happens by MuSCoW technique, or WSJF for big ticket items.
Suggestions from Leadership normally fall in expedited bucket.
Overall, if there's a Scrum way of executing all priorities, then the PO and Team get together to decide the scope of the sprints for execution."
Anonymous Salmon - "Product Bugs can be categorised by severity associated with them.
Product Roadmap prioritization happens by MuSCoW technique, or WSJF for big ticket items.
Suggestions from Leadership normally fall in expedited bucket.
Overall, if there's a Scrum way of executing all priorities, then the PO and Team get together to decide the scope of the sprints for execution."See full answer