Product Manager Artificial Intelligence Interview Questions

Matthew W. - "I’d start by clarifying what the business outcome we are driving/what is the context of the AI platform I am owning. Assuming I am giving no further info, I would start with framing why safety is a must with GenAI, the market being saturated with new LLMs daily, so trust is the main differentiator for retention and defensibility. Safety failures can trigger regulatory action (FTC, EU, etc.), reputational damage, or user churn. Then I would identify some Key Buckets: • Kids/teen protections ("See full answer

Shalin G. - "Tell me about the most recent product you launched that you are very proud of?"See full answer

Debajyoti B. - "As per my understanding, Success of the AI product/feature has to be measured in 2 aspects: success of the problem it solves success of the model used Success of the problem- start with the overall business goal of the problem(acquisition, retention etc) & the high level metric(essentially north star) then comes the product/feature level goals & metrics(conversion, engagement, clicks etc) Success of the model used- general: precision, recall, latency ethical: bias, safety busine"See full answer

Kartikeya N. - "Clarifying questions What company are we?: Uber Why do we want to build the feature?: To increase transactions or revenue Do we have a particular problem in mind or am I free to select the same?: You are free to select the same Do we have any constraints or timelines in mind?: Assume working under realistic budget and a timeline of 6 months Can I assume we will have access to relevant data and AI models for the product?: Yes First let's discuss the Vision for Uber. It is to allow peop"See full answer

S R. - "For RAG systems, You need to evaluate the retrieval and generation. Typically, you have golden truth question and answers as evaluation data set. For retrieval, check if the retrieved contexts are relevant for the question For generation evaluation, You check the semantic similarity between the golden truth and RAG generated answer. Apart from this, you can evaluate the output using frameworks like RAGAS where the answers generated are evaluated based on completeness, faithfulness, to"See full answer

Alex N. - "Perplexity - the measure LLM is surprised when it predicts next word. For example: I love to eat --- if LLM selects as next word "fruits" it will be less surprising than if LLM selects as next word "metal". It is better to have lower perplexity score. Cross Entropy is the measure how well model match true labels. So if the next word is "cat" and LLM assigns 0.5 to it then cross entropy value is -log(0.5) = 0.69 and if it assigns 0.9 probability to word cat then cross entropy value is - log(0.9"See full answer

Tian H. - "to add! health assistant! remind me to do physical check, aggregate my health info including sleep, physic check results. ...."See full answer

Sumit P. - "CQs: Content moderation system finds Inappropriate contents - profanity, violence, privacy concerning. Misinformation - false info, false claim, fomenting wrong views PII Misinformation → Wrong info Twisted info Incomplete info Goal - reliability and trust on the platform Long term increased engagement on informational content RAG system - what and why? RAG system has 3 components Brain - reasoning models Tool"See full answer

Nathan B. - "There are many good answers to this that AI scientists around the world, I and my coworkers have tried over the years. For one, RAG is a great option to fact-check and enforce citation generation, update the data in the knowledge base of the generative AI, etc. "See full answer

Alex N. - "The adjusting context window size in LLM change trade off between reasoning capability, accuracy, computation cost. It influence how attention mechanist allocate resources across the input. Longer context window let it you input greater number of words and have more context to generate proper next token. However llms have lost in the middle issue. They remember the beginning of text and end of text but lost information located in the middle of long input. Another problem is Attention Dilution."See full answer

Hardik saurabh G. - "Hallucinations are evaluated by measuring how often generated outputs contain information that is not supported by trusted sources. what hallucination means in context: Intrinsic hallucination: contradicts provided context Extrinsic hallucination: introduces unsupported facts Fabrication: confidently incorrect answers"See full answer

Ross B. - "I most want to communicate a few principals of conflict resolution that I believe were integral in this situation, which are mutual respect, a results orientation, an unwavering focus on the user. To that end, here’s how I’d like to structure this answer: First, I’ll tell you about the project we were working on, to provide some background for you. Second, I’ll describe the disagreement. Third, I’ll describe how we arrived at a solution, and finally, I’ll discuss how those 3 conflict resolu"See full answer