Adding replication to the Key-Value Store
During the past few weeks I’ve built a basic key-value store that persists data to disk. It works, but only as a single node, so there is a fixed limit on the amount of data it can store, and the througput it can sustain. We can make the store scale horizontally in two ways: Replication, which keeps additional copies of the data in other nodes, improving throughput. Sharding, which partitions the keys across different nodes, so that we can store more data.
Read more…Notes on Eio
This weekend I took a step back from working on my Key-Value store to make sure I understand the concurrency model of Eio, the OCaml 5 library that I’m using for concurrency and parallelism. I decided to turn my notes into a post to force myself to address any gaps in my understanding. I’m sure that there are several errors, or things to improve, so if you are reading this and find some, please, ping me on Twitter!
Read more…A basic Write Ahead Log
This weekend I decided to add some basic persistence to my Key-Value store. I considered going directly into B-Trees or LSM, but they are quite involved. Moving, forward I want to focus more on the distributed side of the store rather than low level storage details (saving those for later!), so for now I decided to implement something simpler, a basic Write Ahead Log (WAL, for short). What is a Write Ahead Log?
Read more…