The ideal distributed file system would guarantee that all its users have coherent access to a shared set of files and be easily scalable. It would also be fault-tolerant and require minimal human administration. This post will cover how Frangipani approximates this ideal, with a focus on how it provides a consistent view of shared files while maintaining a cache for each user.

mit-6.824 distributed-systems learning-diary

Amazon Aurora is a distributed database service provided by AWS. The paper describes the considerations for being a database for the cloud and details how Aurora's architecture differs from many traditional databases today. This post will explain how traditional databases work and then highlight how Aurora provides great performance through quorum writes and by building a database around the log.

mit-6.824 distributed-systems learning-diary

Many distributed systems today sacrifice stronger consistency guarantees for the sake of greater availability and higher throughput. CRAQ, which stands for Chain Replication with Apportioned Queries, is a system designed to challenge this trade-off. CRAQ's approach differs from existing replication techniques we have seen so far, like in Raft. It improves on the original form of Chain Replication. This post will start by presenting the Chain Replication approach, before describing how CRAQ improves on it.

mit-6.824 distributed-systems learning-diary

Can the coordination of distributed systems be handled by a stand-alone general-purpose service? If so, what should the API of that service look like? In addition, can we improve the performance of a system by N times if we add N times replica servers? This post will focus on answering these questions using the ZooKeeper system as a case study.

mit-6.824 distributed-systems learning-diary

One common pattern in the previous systems we have discussed like MapReduce, GFS, and VMware FT is that they all rely on a single entity to make the key decisions. While this has the advantage of making it easier for the system to come to a decision, the downside of this approach is that the entity is now a single point of failure. In this post, we'll discuss the Raft consensus algorithm, which helps to solve this problem, leading to more fault-tolerant systems.

mit-6.824 distributed-systems learning-diary

This post will contain some examples of good and bad Go code, using them to show common mistakes that can be made when starting to build concurrent programs, and how those can be corrected. It will cover goroutines, mutexes, condition variables, and channels.

mit-6.824 distributed-systems learning-diary

Replication is one way by which applications can be made to be more fault tolerant. Using the VMware FT system as a case study, we'll discuss the different ways in which replication can be implemented, the challenges associated with each approach, and some acceptable tradeoffs that can be made when implementing replication in a system.

mit-6.824 distributed-systems learning-diary

Building distributed storage is a hard problem. In this lecture, we'll examine the Google File System and highlight the challenges involved in building distributed storage. We'll consider the tradeoffs that were made in building this system, and briefly discuss why Google eventually had to build a successor to GFS.

mit-6.824 distributed-systems learning-diary

Concurrency is important in distributed systems, but is often easy to get wrong. In this post, we'll learn about threads and why one needs to be careful when dealing with them. We'll also discuss an alternative to having to manage multiple threads and some downsides of that too. Finally, we'll discuss Remote Procedure Calls(RPC), which are a technique for client-server communication, highlighting their challenges and semantics.

mit-6.824 distributed-systems learning-diary

I started a study group with some friends where we will be going through the MIT. 6.824: Distributed Systems course. Over the next couple of weeks, I intend to upload my notes from studying each week's material. This post contains my notes from the first lecture on the MapReduce paradigm for large-scale data processing.

mit-6.824 distributed-systems learning-diary