"System Components
Data Collection Layer
Posts with hashtags are asynchronously sent to Kafka topics
Each message contains: hashtag, timestamp, userid, postid
Multiple Kafka partitions ensure scalability and fault tolerance
Processing Layer
Apache Flink processes streams in real-time
Implements sliding window aggregation (1hr, 24hr, 7d windows)
Calculates topic popularity using weighted metrics:
Post count
User engagement (likes, comments)
Unique user"
Usman B. - "System Components
Data Collection Layer
Posts with hashtags are asynchronously sent to Kafka topics
Each message contains: hashtag, timestamp, userid, postid
Multiple Kafka partitions ensure scalability and fault tolerance
Processing Layer
Apache Flink processes streams in real-time
Implements sliding window aggregation (1hr, 24hr, 7d windows)
Calculates topic popularity using weighted metrics:
Post count
User engagement (likes, comments)
Unique user"See full answer
"The question is bit vague (I guess deliberately) so I believe firstly we shall ask questions and resolve ambiguity. Some initial questions could be :
1) Is this one time activity or something that should be done on continuous basis. If continuous basis then at what frequency.
2) How much staleness is acceptable in SYSTEM Y data
3) Are there any limitation in SYSTEM Y and is it fair to assume that we would need some kind of transformation to bring data into SYSTEM Y schema.
4) What kind of vol"
Kshitij A. - "The question is bit vague (I guess deliberately) so I believe firstly we shall ask questions and resolve ambiguity. Some initial questions could be :
1) Is this one time activity or something that should be done on continuous basis. If continuous basis then at what frequency.
2) How much staleness is acceptable in SYSTEM Y data
3) Are there any limitation in SYSTEM Y and is it fair to assume that we would need some kind of transformation to bring data into SYSTEM Y schema.
4) What kind of vol"See full answer
"To answer this, I will focus my efforts on explaining the most common type of API used in most modern software development applications - the REST API. For the purpose of simplicity, I will also keep the topics of Authorization and Authentication out of the mix.
In essence, an API is a group of logic that takes in a specific set of inputs and responds with a specific set of outputs. This is analogous to going to a drive-thru and placing an order for a meal.
When you give an API a bunch of"
Pathworks P. - "To answer this, I will focus my efforts on explaining the most common type of API used in most modern software development applications - the REST API. For the purpose of simplicity, I will also keep the topics of Authorization and Authentication out of the mix.
In essence, an API is a group of logic that takes in a specific set of inputs and responds with a specific set of outputs. This is analogous to going to a drive-thru and placing an order for a meal.
When you give an API a bunch of"See full answer
"Scoped out problem constraints - how much data to download, how many computers, and how much time, how you can be detected (decentralized load).
Designed a web crawler."
Faraz A. - "Scoped out problem constraints - how much data to download, how many computers, and how much time, how you can be detected (decentralized load).
Designed a web crawler."See full answer
Engineering Manager
System Design
+1 more
🧠 Want an expert answer to a question? Saving questions lets us know what content to make next.
"Very helpful! But I have a question: the location of drivers can be changing every minute, so won't reading from location cache returning stale data?"
Linying W. - "Very helpful! But I have a question: the location of drivers can be changing every minute, so won't reading from location cache returning stale data?"See full answer
"Understand the business problem: Identify the business problem that the AI data product is intended to solve.
Identify the target audience: Understand who will be using the data and what problem they will be solving for using the data. This will inform the features and functionality that should be included in the product.
Gather and preprocess the data: Collect and preprocess the data that is relevant to the problem that it is being solved for. This will inform the AI algorithm"
M D. - "Understand the business problem: Identify the business problem that the AI data product is intended to solve.
Identify the target audience: Understand who will be using the data and what problem they will be solving for using the data. This will inform the features and functionality that should be included in the product.
Gather and preprocess the data: Collect and preprocess the data that is relevant to the problem that it is being solved for. This will inform the AI algorithm"See full answer
"Load Balancer, Multiple API Servers, game Service, pub-sub system, HTTP Long Polling for status updates, write-through cache. No SQL DB to store user, game, and move information."
gdecuri - "Load Balancer, Multiple API Servers, game Service, pub-sub system, HTTP Long Polling for status updates, write-through cache. No SQL DB to store user, game, and move information."See full answer
"Excellent job Roshan, really liked the explanation. Which software were you using to explain the system design diagrams?"
Krishnan S. - "Excellent job Roshan, really liked the explanation. Which software were you using to explain the system design diagrams?"See full answer
"Inventory Service and Registration service can they be in SYNC always ?? The "experience layer" needs accessed by Hotel owners also ??? How does real time inventory come from all Hotels ???"
Anup S. - "Inventory Service and Registration service can they be in SYNC always ?? The "experience layer" needs accessed by Hotel owners also ??? How does real time inventory come from all Hotels ???"See full answer
"I will start with discussing the Google mission statement, which is to organize the world's information and make it easily accessible to everyone to help them get things done, and make their day a bit easier.
I will think about building something that will leapfrog this mission.
Google has already organized all of the world's public digital information in the current state. There are opportunities to get more information into a digital form. But there are also significant improvements in mak"
Praveen - "I will start with discussing the Google mission statement, which is to organize the world's information and make it easily accessible to everyone to help them get things done, and make their day a bit easier.
I will think about building something that will leapfrog this mission.
Google has already organized all of the world's public digital information in the current state. There are opportunities to get more information into a digital form. But there are also significant improvements in mak"See full answer
"
Functional Requirements
Content Ingestion\:
Ingest news articles from various sources (websites, social media, etc.).
Handle different types of content (text, images, videos).
Content Analysis\:
Extract and preprocess text from articles.
Analyze the content for potential indicators of fake news.
Model Training and Prediction\:
Use machine learning models to classify content as fake or real.
Continuously improve models with new data and f"
Scott S. - "
Functional Requirements
Content Ingestion\:
Ingest news articles from various sources (websites, social media, etc.).
Handle different types of content (text, images, videos).
Content Analysis\:
Extract and preprocess text from articles.
Analyze the content for potential indicators of fake news.
Model Training and Prediction\:
Use machine learning models to classify content as fake or real.
Continuously improve models with new data and f"See full answer