System design is all about bottlenecks and choosing the right solution to make that bottleneck more efficient and keeping in mind the tradeoffs of each available option

DB scaling:

  • You do not want to scale too early, avoid premature optimization

Load Balancing

  • Load balancers are generally used for client-facing requests (like websites), where the client accesses  a fixed address (“like google.com”).
  • The load balancer (or DNS load balancing) fans out the requests into multiple other machines 
  • Whether or not you use a load balancer on a microservice depends on the microserver itself 
  • If it is a custom “chat microservice”, then that would likely include load-balancing on the web domain (like ‘chat,example.com’) so that a single machine does not get overloaded by client requests. If your microservice is like a set of backend memcache services, then that architecture tends to already be distributed in nature on its own
  • Any service can be scaled, the question is when to scale it and where the load is. If there is high load on the database for instance, then you may not need to scale out the webservers using load balancers. If your web servers are being pounded then you need to distribute that across multiple machines using load balancing. 
  • The primary benefit of microservice is that it decouples load and simplifies architecture towards many smaller independent services, making it less monolithic. This structure in itself can help with scaling
  • To truly scale, most companies will never depend on a single machine but multiple machines to help distribute load. So you have multiple read databases, multiple write databases, multiple caching machines, multiple webservers, multiple email servers, multiple image servers, multiple upload servers, multiple CDNs, etc some of these would be configured using LBs but others like distributed databases could be implemented through partitioning/sharding as well

“What do you mean by ‘the DNS occur on your own personal load balancer’”?

  • You can offten associate a domain name with multiple IPs, which in effect helps to balance load. That acts like a load balancer

Caching 

  • Above application server 
  • Before you hit the database, you hit the cache
  • A seperate server/machine that could be running with other distributed machines; it could be a huge distributed cache
  • Highly scalable, distributed and fast 
  • Caching can take a lot of load out of the database
  • Drawback of memory cache is that it is not persistent; you don’t want to store user info data here because it can be lost at any time => use cache for something that is repetitive i.e a single use of truth. You can use it as a data store for data that is not too valuable; data that you can afford losing. 
  • Drawback: invalid data. I.e user changes user settings and settings in cache is not longer accurate
    • solution : write through cache: when user changes information on their profile, you first change the information on the cache and then the database, but writing can be as slow as if you did not have the cache in the first place
    • Write back cache: you only change the cache you don’t change the database yet. You have an intervals where you update the database based no the cache, possible problem is that you update the cache with new user information and you unexpectedly lose that new information, then you can not retrieve that new information because it was not written to the database
    • Cache limit? LRU, FIFO
    • In a nutshell, it is a key value store
    • Use to reduce load think about it as a dictionary in its simplistic terms; a distibuteed dictionary 

Content Delivery Network (CDN)

  • Good for storing static files i.e images, videos, 
  • Even if someone lives in Korea, they can hit up a server near them to get the data 
  • Distributed network of servers that will hold you files that is global
  • Makes static files easily accessible all over the world even if it is not close to your webserver
  • They essentially cache your data
  • Websites that are media heavy use CDN to hold their image and videos

Populating CDN

  • Pull base model: the initial request entails that the CDN does not have your files so it would pull from your server and give it to the user or whom ever requested it and then cache it to a server that is local to that user who requested saved for future requests around that geographical local location of the initial request
  • Push base model: you upload a set of values and push to various CDNs
  • Trade offs
    • Pull base is easy; host files locally, setup cdn and done
    • push : you have to push the files to CDN

Databases and Indexing 

  • A database is an elegant way to store data in a persistent disk, and you have a database server that has an API that you can interact with to retrieve, modify, place, delete the data
    • Schema design:
      • Understand that every table should have an iD that represents the row uniquely
      • Don’t use memory intensive objects i.e an image; makes it slow in terms of scalability
  • Indencies:
    • Indexing prevents form searching the entire table and makes joint searches a lot faster
    • A binary search tree on the ID or name to name a few examples.
    • Makes easer for searching i.e looking up for “Gabriel”
    • You can have double indices ie name and date
    • Drawbacks: the more indexing the more spacing because you have to create multiple binary search three for each type of indexing
      • Every time you insert something you will have to also update the rest of your indencies
      • I.e log(n) operation for inserting for every indencie that you have

Redundancy and Replication

  • Proper way to scale databases
  • Master-slave replication
    • The master database that you can write to and slave database that you can read from but not write to
    • Slave mimic the master with a little bit of delay to bring in the data from the master
    • Slave replication can be used to back up your data
  • Why is replication needed
    • When you have read heavy scenarios but do not write often i.e youtube where you have more consumers that you do content makers. This is where you would implement master-race
  • Master-Master 
    • Multiple databases that replicate each other and all act as masters
    • You can write and read to either one and each one can have multiple slaves
    • drawback : difficult to visualize, writes are going to be your bottlenecks
    • Not the cleanest implementation

Database Sharding

  • Horizontal partitioning also known as sharding, you balance the rows between the databases. 
  • Vertical partitioning is vertical partitioning 
    • One of the easiest things you can do 
  • What if you have a single table that you cannot split across multiple machines
    • This is where the horizontal splitting comes into play i.e database sharding
    • We can load balance our databases
  • def sharding: instead of having one single database to write on, you want to write it to multiple databases on multiple computers to balance the load. Each of the databases represent a portion of the table
  • With all these databases, how do you choose which database to store the database in?
    • Data partitioning, can be key value pair like a hash map
    • Make sure your sharding scheme is well thought out or it can get complicated when you add more databases
  • Data partitioning is dependent on your goal, you can partition based on location for example

SQL vs NoSQL

  • SQL – relational
  • noSQL – non-relational
  • Non- relational noSQL
    • Are not structured
    • Easier to distribute
    • Have a dynamic schema not fixed
    • In a nutshell it is a key-value dictionary
    • Good for pumping data
    • Not ACID compliant
    • Possible race conditions
    • Hugly distributed
    • Easily scalable by adding more machines
    • Memory based
    • styles 
    • Good write performance
    • Not as consistent
      • memcache 
      • Key-value store
      • Document based database, i.e mongodb
        • JSON key-value store
  • SQL
    • Fixed schema
    • Structured because of the schema
    • Got for joints
    • ACID compliant
      • Automizity, consistency, isolation, durability
      • I.e bank transactions
      • Important for transactional operations
      • consistency 
        • When you write data, you read exactly back the data that you just wrote
        • Weak consistency- write data, but when you read you might no be able to read it. I.e memory cache
        • Eventual consistency – you are guaranteed to be able to read that data back but after a certain time, not guaranteed to immediately be able to read it right after a write
        • Strong consistency
          • Used in for example mySQL
        • Part of a trade of three different things
          • consistency 
          • availability 
          • Partition tolerance
          • You can not have all three
  • White column databases
    • Don’t have to read all the attributes
  • Graph database

API Design 

  • An interface for you client
  • You want to make your system as easy to use
  • What is the interface between the client and server
  • Make it clean, simple, secure
  • Be clear and concise as to what your endpoints do
  • Think bout the flow of your application and what operation you need to do. I.e reading, writing,  and what the inputs and outputs should be
  • Think about pagination
  • Break it down to a couple of items
  • pagination 
    • Can be messed up if you don’t account for new posts or duplicate posts for example in a feed. How to fix this?
      • You can use cursors
        • Generally when you get your set of results you can also get a cursor which is marker for the next set or result you can query for
        • This cursor can be a simple scheme i.e the id of the last item that you got
  • Server dirven API vs server drive API
  • Designing API
    • mobile 
      • You want a server drive API
  • Performance 
  • Having a good API would result in excellent work flow between teams because it allows for scalability between workforces 
  • Makes sure you think about inputs, outputs, and error mechanisms
  • Return JSON and a string for errors?
  • Make sure your API does not have intensive side effects that could affect performances of applications utilizing your API

Mobile system design 

  • Think about how things are wired at the high level.
  • Think about how you could make each page as independent as possible and not rely to much on dependencies between pages. This would allow various teams to work on different pages with their own development workflows. 
  • Helps to have a server powered API
  • Abstract the networking layer
  • Have a few singletons in place
  • Having publish/subscribe events and listeners

Resources:

Systems Design Interview Concepts (for software engineers/full-stack web)

TechInterviewPro

https://www.techseries.dev