This article walks through how to approach a Web Crawler system design problem in interviews, including scale estimation, architectural choices, and trade-offs. This system design problem comes quite a lot in the interviews - and for a reason - this simple high level problem comes in many different flavours and has the potential to get deeper-and-deeper and it gets harder … [Read more...] about System Design – Building Large Scale Web Crawler
Play Tic Tac Toe With LLM Claude 3.7
I was looking for some projects that would be a good demonstration of how AI can automate or solve problems just like humans. Yesterday, I was playing a tic tac toe game with my wife and I just thought this could be a very good starting point for it. So here I'm sitting at my desk trying to build one. But plain old command line tic-tac-toe is not fun. You should be able to … [Read more...] about Play Tic Tac Toe With LLM Claude 3.7
Is The Cosmos a Vast Computation?
Throughout history, humanity has often looked to its most advanced technologies as metaphors to understand the universe. During the mechanical era, we viewed the cosmos as a giant machine. The advent of clocks led to the notion of a "Clockwork Universe," steadily ticking forward with a predictable rhythm. When steam engines came into play, we imagined our universe as a … [Read more...] about Is The Cosmos a Vast Computation?
Building Semantic Search for E-commerce Using Product Embeddings and OpenSearch
Hey! So, I recently worked on this cool proof of concept (POC) where I tried to combine product embeddings with OpenSearch to enable a k-nearest neighbours (k-NN) search. Basically, I wanted to see if I could make searching through a product catalog smarter by using embeddings (which are essentially vector representations of the product information) instead of just basic text … [Read more...] about Building Semantic Search for E-commerce Using Product Embeddings and OpenSearch
Leader Election with ZooKeeper: Simplifying Distributed Systems Management
Imagine you are in a group project, the task is pretty complex :p and everyone is looking at each other, wondering who is going to take the lead? Does this situation sounds similar to you? I remember we used to play some sort of game like "Rock, Paper, Scissor" or "Eeny, meeny, miny moe" (I never did this but my sister). Similarly, distributed systems have their own ways of … [Read more...] about Leader Election with ZooKeeper: Simplifying Distributed Systems Management
AWS Serverless Event Driven Data Ingestion from Multiple and Diverse Sources
At times there is a requirement to centralize data from different sources. This could be required for various needs, There could be any number of reason behind data ingestion. And in this article I will explore a very cost effective, scalable and completely serverless architecture that could do just that. I have used AWS cloud for the same but I'm sure you can find a … [Read more...] about AWS Serverless Event Driven Data Ingestion from Multiple and Diverse Sources
A Step-by-Step Guide to Deploy a Static Website with CloudFront and S3 Using CDK Behind A Custom Domain
This article is intended for developers who want to deploy a Single Page Application on Cloudfront and make it accessible via a custom domain. I will try to make it systematic so its easier to pick and implement directly. I will try my best to explain why each step is necessary. Sometimes I have found that things don't work and usually there's a puny reason behind it. … [Read more...] about A Step-by-Step Guide to Deploy a Static Website with CloudFront and S3 Using CDK Behind A Custom Domain
Setup Frontend Project Without Needing Backend Using Local Storage (Dependency Injection)
Today the frontend applications has become quite sophisticated. They are capable of handling complex logic to provide a seamless user experience. With the frameworks like React/Angular/Vue the landscape of frontend has changed quite a lot. I remember the days when I used to write my application in pure HTML and use to handle the dynamic interactions with Javascript (using … [Read more...] about Setup Frontend Project Without Needing Backend Using Local Storage (Dependency Injection)
Deploy Lambda Function and API Gateway With Terraform
Recently, I have started learning terraform (not by choice) because one of the clients needs it. Its a neat tool to manage your infra as code, no doubt about it. But if you ask my personal opinion as a dev, I would say go with the native cloud solution. For ex - AWS CDK. Native not only has all the required features to work with your cloud provider but also streamlines and … [Read more...] about Deploy Lambda Function and API Gateway With Terraform
Event Driven Communication Between Domains/Components – Frontend Development
Event Driven Architecture has been around for decades. This is not something new that we discovered recently. But we have definitely seen the rise of this with the emergence of distributed system and need for asynchronous communication. Event Driven Architecture really helps to reduce the complexity involved with distributed system while providing a way to asynchronously … [Read more...] about Event Driven Communication Between Domains/Components – Frontend Development