Product Manager Machine Learning Interview Questions

Ajay P. - "Designing an evaluation framework for ads ranking is crucial for optimizing the effectiveness and relevance of ads displayed to users. Here's a comprehensive framework that you can use: Define Objectives and Key Performance Indicators (KPIs):** \\Click-Through Rate (CTR):\\ The ratio of clicks to impressions, indicating the effectiveness of an ad in attracting user attention. \\Conversion Rate:\\ The ratio of conversions (e.g., sign-ups, purchases) to clicks, measuring how well"See full answer

Anonymous Partridge - "We will have to use a second more powerful LLM Model to validate the answers. LLM as a judge"See full answer

Alex N. - "The adjusting context window size in LLM change trade off between reasoning capability, accuracy, computation cost. It influence how attention mechanist allocate resources across the input. Longer context window let it you input greater number of words and have more context to generate proper next token. However llms have lost in the middle issue. They remember the beginning of text and end of text but lost information located in the middle of long input. Another problem is Attention Dilution."See full answer

Hardik saurabh G. - "Hallucinations are evaluated by measuring how often generated outputs contain information that is not supported by trusted sources. what hallucination means in context: Intrinsic hallucination: contradicts provided context Extrinsic hallucination: introduces unsupported facts Fabrication: confidently incorrect answers"See full answer

Chermaine Y. - "Zero in on the problem, the expectations of user are to find a restaurant but their feed is uninspired so they may bounce out of Yelp. Identify the impact size of user feeling like discovery is not personalised enough by seeing % of users that selected a restaurant from the homepage If large enough, I will look at who is likely the ones that want personalisation and why? Do they feel like they want to try new restaurants or are they finding it difficult to find restaurants they have been"See full answer