Accelerant Learning/Implement RougeDB, a Redis clone from outer space

  • $279 or 6 monthly payments of $49

Implement RougeDB, a Redis clone from outer space

Get your hands dirty with asynchronous and event-driven programming with Rust using Tokio.

About the course

Hello, aspiring Rustaceans! Welcome aboard our exciting journey into the depths of systems programming, networked applications, and databases.

You're about to embark on a quest to build RougeDB - a simplified clone of the fast and feature-rich Redis, all from scratch and entirely in Rust, the language that empowers everyone to build reliable and efficient software.

We'll start simple, with the basics of TCP connections using Rust's standard library. Here, you'll get your hands dirty with socket programming, understanding how data flows between a client and a server. It's the barebones, but it's also the essence of any networked application.

We'll then elevate our database to the next level with async programming using the `tokio` runtime. This will take us into the realm of highly concurrent networked applications, where numerous clients can connect and interact with the server at the same time. You'll learn about Rust's `async/.await` syntax, its async ecosystem, and how `tokio` makes handling high-performance networking a breeze.

As we progress, we'll enhance our implementation iteratively. We'll delve into advanced parsing techniques with the 'syn' crate, crafting a parser for our custom command protocol, and you'll witness the power of Rust's type system in managing complex data transformations.

Building a Redis clone is an ambitious project, but fear not! This tutorial is designed to guide you through the complexities and challenges, breaking them down into digestible, understandable parts. Remember, every journey starts with a single step, and we're here to guide you through each step of the way.

It's going to be challenging, engaging, and, most importantly, fun. So roll up your sleeves, boot up your machines, and let's embark on this thrilling adventure into the cosmos of Rust and Redis!

Prepare for lift-off!

UNDER DEVELOPMENT

Contents

SUBJECT TO CHANGE

Starting Out

Who should do this course
Preview
Course structure
Technical setup
Preview

Introduction

In this course, we're going to build a simplified clone of Redis called RougeDB. We will use Rust as the primary programming language. The goal is not to build a full-fledged, production-ready Redis replacement, but rather to learn Rust and understand the workings of databases and how to create a reliable service using the tokio ecosystem.

What is Redis?
Why Rust?

Part 1: RougeDB v1 - Using the Standard Library

In this section, we're focusing on getting some basic up and working. It'll also be a good time to learn more about the standard library and how rich it actually is. Rust's somewhat (in)famous within its peer languages of forcing programmers to rely heavily on third-party packages (crates). This section aims to challenge that perception.

What you'll learn from Part 1

Parsing commands (v1)

We want to give our database the ability to parse commands from some external source. That source will eventually be the network, but we might start at the command line.

Setting up
Preview
Fixing mistakes with Git
Adding command-line arguments to the client
Accepting multiple commands
Creating commands from a client
How to add a Redis server to your project for testing
Enable users to specify the server to connect to as a command-line argument
Provide usage documentation
How Redis commands are structured
Parse SimpleString messages from a Redis server
Generating a command to that can talk to a real Redis server!
A command parser using methods from Rust's standard library

Data store (v1)

How Redis stores data in memory
Implement the GET and SET commands to store byte slices
Implement the INCR and DECR commands to store integers
Testing the data store

Persistence (v1)

How Redis persists data
Implement whole-database snapshots
Testing the persistence layer

Networking (v1)

How clients connect to Redis over the network
A basic TCP server using the standard library to handle incoming client connections
Add multithreading
Test the server

Part 2: RougeDB 2 - Increasing Performance with Async and Third-Party Crates

This section shows you how to build a high-performance network-facing application. You'll have the full gamut of the 3rd party ecosystem at your disposal.

What you'll learn from Part 2

Parsing commands (v2)

The syn crate and its benefits
Re-write the command parser using syn

Data store (v2)

Locking strategies
Enable multiple readers and writers

Persistence (v2)

The serde and bincode crates and their benefits
Re-write the data store using these crates

Networking (v2)

Introduction to tokio crate and asynchronous Rust
Re-write the server using tokio

Part 3: Building a More Complete System

Creating an extendable database

Web Interface and Metrics

A simple web interface
Emit memory usage metric
Emit concurrent connections metric
Emit command usage metrics

Lua Scripting

Introducing the lua crates
Implement EVAL and provide basic Lua scripting

Error Handling

Recovering from panics
Let it crash?