Preface
- Introduction
- What and Why is Redis?
- The Redis Serialization Protocol (RESP)
- How does RESP Work?
- What is a Parser?
- What is a Token?
- How to use our Tokens: Building a Lexer
- What is a Command
- Defining GET
- Implementing a Parser
- Parser Recap
- Creating Values
- Defining Expressions
- Storing Values
- Evaluating Commands
Introduction
Redis revolutionized the database world by embracing a radical idea:
complexity should live in simplicity. This article is your gateway
into that philosophy. We'll reverse-engineer Redis starting with a simple TCP Socket, then implementing the Redis Serialization Protocol (RESP) and
supporting the evaluation of core commands all with Golang and simple yet incredible solutions.
NOTE: 👋 This article is for anyone curious about Redis! Rather
than overwhelming you with 2,000+ lines of code (just testing the clone), we'll focus on the key algorithms and design ideas that power this clone.
Want to explore the full project? Check the repo here! 🚀
What and Why is Redis?
Before Redis, scaling applications was a paradox. As app usage soared, existing infrastructure crumbled under the weight of simple tasks like managing shopping carts, user sessions, and real-time leaderboards. These often relied on slow disk-bound databases and fragmented caching.
Redis provided the breakthrough. It's a blazingly fast, in-memory database designed for simplicity: atomic operations, a clean protocol (RESP), and pure RAM-speed access. By offloading ephemeral data like sessions and cached queries, Redis became the crucial enabler for systems to scale beyond their limits.
The Redis Serialization Protocol (RESP)
Computers communicate like humans at a conference: they establish connections (TCP handshakes) and agree on a shared language (RESP).
In our conference there's a speaker and all participants are asking questions
in this RESP language that the speaker answers.
Our Redis server is no different than this speaker, and all the
Redis clients are the participants which conversation between
Server and Client we call it a Connection.
How does RESP Work?
Redis works with commands like GET, SET, INCR, EXISTS. Each of these has its own structure, technically known as grammar.
Let's see some examples of this grammar:
As you can see, the GET command is pretty simple, just the prefix GET
followed by a key
argument.
As Redis is a key-value database, when we use the GET command it returns the value associated with that key or nil
(nil means the key doesn't hold any value).
Let's see some similar commands:
SET <key> <value>
GETSET <key> <value>
GETDEL <key>
INCRBY <key> <increment>
INCR <key>
These are some of the Redis commands we are going to support.
You will be amazed by how easily we can extend the available commands with the
help of a
Parser, but we'll get to that later.
Now that we know some commands and their grammar, it's time to serialize a command!
Don't let yourself be intimidated by the Serialization of OK
in fact it is pretty simple.
RESP Element | Explanation |
---|---|
*1 | The ( * ) indicates an Array, in
this case the Command itself is consider as an Array of
words, the 1 after it indicates its length.
|
\r\n |
The \r\n (CRLF) is the protocol's terminator, which always separates its parts.
|
$2 |
The ( $ ) indicates a BulkString
which is a Binary-Safe String, the 2
which indicates its length (determined in characters).
|
\r\n |
We use the terminator (CRLF) to separate $2 from the data it represents ( OK ).
|
OK | OK is the actual data represented by $2 .
|
\r\n | The terminator (CRLF) is also used to indicate the end of the Command. |
If you put it all together the result is:
Let's see one more example to understand this concept.
RESP Element | Explanation |
---|---|
*3 |
The ( *3 ) indicates that the command is composed of 3
elements (SET , userName , Alejandro ).
|
\r\n |
The \r\n (CRLF) is the protocol's terminator, which always separates its parts.
|
$3 |
The ($3 ) represents that the following string literal (SET ) has length of 3 letters.
|
\r\n |
We use the terminator (CRLF) to separate $3 from the data it represents (SET ).
|
SET | SET is the actual data represented by the $3 before.
|
\r\n |
The terminator \r\n (CRLF) separates the actual data
(SET ) from the next length specifier.
|
$8 |
The ($8 ) represents that the following string literal (userName ) has a length of 8 letters.
|
\r\n | (Required separation) |
userName | userName is the actual data represented by the $8 before.
|
\r\n | (Required separation) |
$9 |
The ($9 ) represents that the following string literal (Alejandro ) has a length of 9 letters.
|
\r\n | (Required separation) |
Alejandro | Alejandro is the actual data represented by the $9 before.
|
\r\n |
We also use \r\n to indicate that the entire Command
has ended. |
It might seem a long process but we finally had come to a result.
If you are still confused about how RESP works, I highly recommend you to consult this Redis article about RESP. You can find it here! 🚀
If you're wondering why we're doing all of this, remember that computers
communicate with each other in a shared language.
This language had to be efficient and as expressive as possible, as this is crucial
to optimize the Redis Parser.
But wait, what is a Parser?
What is a Parser?
Definition
: Parsing is the process of structuring a linear
representation in accordance with a given grammar.
WHAT? Even if this definition can sound a little complex, it is, in fact, is
pretty simple.
A Linear Representation can be either a sentence of a language,
a piece of music, or a Redis Command.
We have mention grammar before as the structure of a sentence,
or a command, think of it as a formula to get different sentences.
So, applied to our use case.
Case Definition
: Parsing is the process of structuring a
Redis Command in accordance with a given Command Structure.
This is the theory, but in practice, we'll have an input string that we'll try to parse and the output will be a representation of the Redis Command known as an Abstract Syntax Tree (AST).
But still we haven't implemented anything yet, and we still don't know what is a Token. So...
What is a Token?
We have been using the term words, meaning they are the fundamental unit of information in Redis Commands and in texts in general, but the technical term is tokens, and we will use it from now on.
Now that we understand what a Token is, the next step is to define
the various types or kinds of tokens our parser will recognize. We typically do
this using an Enum
(meaning Enumeration
), which is a type that defines a set of named constants. This Enum will
list all the possible categories of tokens our Redis Parser will encounter,
such as Commands
, Identifiers
, Numbers
,
Arguments
, and all categories our Parser will recognize.
type TokenKind byte
// Check the source code to see all the Tokens!
const (
EOF TokenKind = iota // EOF = End Of File
ILLEGAL
// Special
CRLF
TEXT
// Commands
keyword_beg
IDENT
SET
GET
INCR
INCRBY
...
We first create a byte
representation of our tokens that we
will name TokenKind
.
You might be wondering why we use a byte
type instead of string
or even
int
.
There are two main reasons:
- We will be creating a lot of Tokens, so the storing structure should be as lightweight as possible.
-
Given that we won't have a lot of
TokenKinds
and knowing thatbyte
(an alias foruint8
) has a maximum capacity of 255. So, as we won't have many tokens this make it the perfect fit for ourTokenKind
.
So, let's create a Structure to contain the TokenKinds
and the text they represent.
The Token
struct is composed of:
Literal
: Which is the slice of text of the Token.-
Kind
: Which is the TokenKind representing theLiteral
.
🔥 Fun Fact: Even thought i just said that the Token
struct should be as lightweight as possible, this is not the most lightweight
representation for a Token.
The Golang Parser uses an amazing technique: instead of storing
a piece of text in the struct as we do, they just store the indices from where the piece of text beings and ends, a simple and ingenious trick if
you want to implement it, Check the details here.
But now that we have defined our tokens it's time to use them!
How to use our Tokens: Building a Lexer
To transform raw input (like SET userName Alejandro
) into
tokens, we need a Lexer (also called a Scanner). This component scans the input text and converts it into a structured
stream of tokens for parsing.
Key Lexer Mechanics: For Tokens like userName
we need to track:
pos
: Start position of the current Token.-
readPos
: End position of the current Token. (used for slicing). -
ch
: The current character being analyzed.
These fields enable precise substring extraction (e.g., slicing
input[pos:readPos]
to isolate userName
).
type Lexer struct {
input string // Current command as raw text
pos int
readPos int
ch byte
}
Now that we have defined our Lexer, we have to create it a constructor to make it a reliable API.
// The trimInput function removes the whitespaces to the right of the input string
func trimInput(input string) string {
return strings.TrimRightFunc(input, func(r rune) bool {
return r == ' ' || r == '\t'
})
}
func New(input string) *Lexer {
l := &Lexer{
input: trimInput(input),
pos: 0,
readPos: 0,
}
l.next()
return l
}
We use the trimInput()
function to ensure that the lexer correctly
processes commands without being affected by trailing spaces to the right of the
Command.
The New()
constructor initializes the lexer
by calling next()
, which handles character-by-character
advancement through the input string. Here’s how it works:
func (l *Lexer) next() {
if l.readPos >= len(l.input) {
l.ch = byte(token.EOF)
return
} else {
l.ch = l.input[l.readPos]
}
l.pos = l.readPos
l.readPos++
}
As you can see the next()
method handles two cases:
-
If the
readPos
has exceeded the input's length it setsch
to thetoken.EOF
(End Of File) TokenKind that indicates that the input string has been consumed. -
If the
readPos
has not exceeded the input's length it setsch
to the character at the positionreadPos
(making it the new current character) and then advances bothpos
andreadPos
.
With character advancement handled by next()
, we'll now
implement NextToken()
to scan and return structured tokens from
the input.
If you review the code implementation, you'll notice that I have skipped some code.
This is because the Lexer is used in both the serializer
and
the Parser
. However, the ideal approach is to create
separate implementations for them, since this allows you to leverage the full
potential of RESP.
func (l *Lexer) NextToken() token.Token {
// First Operation
if l.ch == byte(token.EOF) {
return token.New(token.EOF, "")
}
// Second Operation
l.skipWhitespaces()
// Third Operation
if l.ch == '\r' && l.peekChar() == '\n' {
l.next()
return token.New(token.CRLF, "\r\n")
}
...
}
The first thing you can see is that the NextToken()
returns
a token.Token
we have defined in previous section.
Also this method performs three key operations:
-
If
l.ch
istoken.EOF
, this indicates that the input string has been fully consumed. In this case, an emptyToken
with itsKind
set totoken.EOF
is returned. -
Skips any leading white space characters (e.g., spaces, tabs
\t
, newlines\n
). -
Using the
peekChar()
function which returns us the next character without advancingreadPos
, we can check if the current character is\r
and the next character is\n
(forming\r\n
known as CRLF), which acts as aToken
separator, so a CRLF token is returned.
EmptyToken{
Kind: token.EOF,
Literal: "",
}
CRLFToken{
Kind : token.CRLF,
Literal : "\r\n",
}
Now, we need to be able to create Tokens
for like SET
, userName
, and Alejandro
. As you may
have noticed, what these all have in common is that they are composed of letters.
To handle the identification of keywords and identifiers more effectively, let's
create two utility functions.
func isLetter(ch byte) bool {
return ('a' <= ch && ch <= 'z') || ('A' <= ch && ch <= 'Z') || ch == '_'
}
func (l *Lexer) readIdent() string {
pos := l.pos
for isLetter(l.ch) {
l.next()
}
return l.input[pos : l.pos]
}
This is a simplified implementation, if you wanna see it whole i recommend you
to check the implementation details.
For now let's see what is happening here.
-
isLetter
: This function checks if the given character (ch
) is a letter(a-z, A-Z)
or an underscore (_
). -
readIdent
: This method reads an entire identifier (a sequence of letters and underscores) from the input
Now that we have implemented these utility functions, let's finally examine
the second part of the NextToken()
implementation.
func (l *Lexer) NextToken() token.Token {
// ... (previous logic for EOF, whitespace, CRLF)
if isLetter(l.ch) {
t.Literal = l.readIdent()
t.Kind = token.LookupIdent(t.Literal)
return t
}
// ... (Handeling numbers)
t.Literal = ""
t.Kind = token.ILLEGAL
return t
}
NOTE: Implementing Numbers is pretty similar to the characters implementation.
This completes the core logic for identifying keywords and identifiers. If the
lexer doesn't recognize any token (including keywords
, identifiers
or numbers
), it defaults to returning a token.ILLEGAL
. As you may have noticed, we haven't yet defined token.LookupIdent
. Let's do that now.
var keywords = map[string]TokenKind{
"GET": GET,
"GETSET": GETSET,
"GETEX": GETEX,
"GETDEL": GETDEL,
"SET": SET,
"DEL": DEL,
"INCR": INCR,
"INCRBY": INCRBY,
"DECR": DECR,
"DECRBY": DECRBY,
// ... (The rest of the Commands and Keywords)
}
func LookupIdent(ident string) TokenKind {
kw, ok := keywords[ident]
if ok {
return kw
}
return IDENT
}
As you can see this function is pretty straight forward, and with this we have
finished our Lexer.
So, after all of this work we can finally implement a Parser.
What is a Command
Do you remember the Case Definition
we proposed ? Let me give
you a hint.
Case Definition
: Parsing is the process of structuring a
Redis Command in accordance with a given Command Structure.
Structure and Redis Command are highlighted for these a specific reason:
- To create a good Structure we have to define all of the required fields that a Command must have.
- To create a good Redis Command we'll use a function that checks all of the required fields to successfully create a Command.
To achieve the first of these requirements we are going to create a Command Interface which is going to identify all of the possible Commands.
type Node interface {
String() string
TokenLiteral() string
}
type Command interface {
Node
cmdNode()
}
Node
represents any parseable unit (Command
or Expressions
).
-
String
returns a Human-readable representation of the Command. -
TokenLiteral
returns the Literal of the prefixToken
(SET
,GET
,INCR
, etc.)
Command
marks Redis Commands nodes (GET, SET, etc.)
-
The
Node
interface is embedded intoCommand
as Command is also a Node. -
cmdNode()
is an empty method that differentiates Commands from Expression (which we will see soon).
Now that we have created our Command
interface, we can start
implementing Commands.
NOTE: Through the rest of the article we will just implement the
GET
command, but all the Command implementations are similar,
and you can check them here.
So... Let's do it!
Defining GET
GET Command Reminder: The GET Command takes a key
as a parameter and returns the value associated to that key.
To implement the Command GET
we first have to create a Structure which implements the Command
interface, then we will create
a function that takes a Stream of Tokens and returns a GetCommand
.
As you can see the GetCommand
structure stores all the relevant
information to represent the GET
Command Structure.
But it still does not implement the Command
interface.
func (gc *GetCommand) cmdNode() {}
func (gc *GetCommand) TokenLiteral() string {
return gc.Token.Literal
}
func (gc *GetCommand) String() string {
var sb strings.Builder
sb.WriteString(gc.Token.Literal)
sb.WriteByte(' ')
sb.WriteString(gc.Key)
return sb.String()
}
There are a lot of thing that are happening right here so let's break it down!
-
cmdNode
: This is correctly implemented as an empty method to satisfy theCommand
interface, serving its purpose of differentiating Commands from Expressions. -
TokenLiteral
: This method returns the literal value of the command's token (e.g.,GET
), adhering to theNode
interface. -
String
: This method provides a human-readable representation of the GET command (e.g.,GET mykey
), which is crucial for debugging and logging, as required by the Node interface. The use of strings.Builder is a good practice for efficient string concatenation in Go.
Now that our GetCommand
satisfies the Command
interface. It's time to put it to work.
Implementing a Parser
Now that we have create the Command we are going to Parse (GET
) it's time to create the Parser!
The Parser logic is pretty similar to the Token's one, let's see why.
- They both use a two index approach: In the same way that the
lexer
uses apos
andreadPos
the Parser usescurTok
andreadTok
to parse the incoming input string. - They both use special functions: To being able to parse a Command, well create a specific function for each command that tries to parse it or throws an error.
NOTE: I'm going to present you a simplified implementation to avoid boilerplate.
Knowing this, let's define the Parser!
-
*lexer.Lexer
: The lexer contains theNextToken()
method that converts raw input (likeSET foo bar
) into a stream of parsed tokens (likeSET
,foo
,bar
). -
curTok
: Represents the token being processed right now—like the word your eyes are focused on while reading. Determines the parser’s immediate action (e.g., "Is this a SET or GET?"). -
readTok
: Represents the upcoming token, peeked at but not yet consumed. Acts like a "sneak preview" to resolve ambiguities (e.g., seeing if theSET
Command contains an optional argument).
Now that we have our definition of our Parser
, it's time to
initialize it!
func (p *Parser) next() {
p.curTok = p.readTok
p.readTok = p.l.NextToken()
}
func New(l *lexer.Lexer) *Parser {
p := &Parser{l: l}
p.next()
p.next()
return p
}
The initialization of the Parser is pretty simple, let's take a brief look:
-
next()
: This function advances the Tokens by using theNextToken()
method of lexer. -
New()
: This method takes a*lexer.Lexer
as argument and then it initializes thecurTok
andreadTok
by using thenext()
method.
With this we are almost done, we just have to create a function that depending
on the first Token
it selects the according parsing function
.
Before creating the function that parses the GET
command we
have to create a function that Parses all the possible Commands, we will call it
Parse
:
func (p *Parser) Parse() (ast.Command, error) {
// ( 1 ) Checks if the Command starts with a Array Representation
if err := p.parseBegCommand(); err != nil {
return nil, err
}
// ( 2 ) Checks if the Command continues with a BulkString
if err := p.skipBulkString(); err != nil {
return nil, err
}
// ( 3 ) Parses the Command
cmd, err := p.parseCommand()
if err != nil {
return nil, err
}
return cmd, nil
}
This function can seem tough but it just handles these three key parts:
-
p.parseBegCommand()
: Checks if the Command starts with anArray
set of Tokens (e.g.,*2\r\n
) if it does, it advances the Tokens till the next set of Tokens ($3\r\n
). -
p.skipBulkString()
: Checks if the current set of Tokens denotes aBulkString
(e.g,$3\r\n
), advancing the Tokens as in the last point. -
p.parseCommand()
: After all of the executions of the previous functions, this function checks if the currentToken
is a command prefix (e.g,SET
,GET
) executing its corresponding parsing function (defined down bellow).
We do all of this because in this stage we will get the Command in a RESP format:
*2\r\n -- $3\r\nGET\r\n -- $3\r\nkey\r\n
# | | |
# We Check all of this parts individualy
With this we are almost finished with Parsing, we just have to create the p.parseCommand()
and p.parseGetCommand()
. So, let's do it!
The definition of p.parseCommand()
is actually pretty simple.
func (p *Parser) parseCommand() (ast.Command, error) {
switch p.curTok.Kind {
case token.GET:
return p.parseGetCommand()
...
}
return nil, fmt.Errorf(
"command not supported. got=%d (%q)",
p.curTok.Kind, p.curTok.Literal,
)
}
As you can see the p.parseCommand()
returns a ast.Command
and a error
, then using a switch
statement
we determine the corresponding Parsing Function for that respective
Command.
If the Command is not found: we return a "Command not supported"
error.
Knowing this, let's define p.parseGetCommand()
and finish this
whole Parsing section.
func (p *Parser) parseGetCommand() (*ast.GetCommand, error) {
getCmd := &ast.GetCommand{Token: p.curTok}
p.next()
if err := p.checkCRLF(); err != nil {
return nil, err
}
key, err := p.parseIdent()
if err != nil {
return nil, err
}
getCmd.Key = key
return getCmd, nil
}
As you can see the parseGetCommand
function returns the *ast.GetCommand
structure we have defined before and an error
.
There are also some special functions we will now see:
func (p *Parser) checkCRLF() error {
// Checks if the current Token is CRLF
if !p.curTokIs(token.CRLF) {
return fmt.Errorf(
"token expected=%d ('CRLF'). got=%d (%s)",
token.CRLF,
p.curTok.Kind,
p.curTok.Literal,
)
}
p.next()
return nil
}
func (p *Parser) parseIdent() (string, error) {
if err := p.skipBulkString(); err != nil {
return "", err
}
if !p.curTokIs(token.IDENT) {
return "", fmt.Errorf(
"token expected=%d ('IDENT'). got=%d (%q)",
token.IDENT,
p.curTok.Kind,
p.curTok.Literal,
)
}
return p.curTok.Literal, nil
}
These two functions are widely used in the whole Parser, let's explain them quickly:
-
checkCRLF
: Checks if the current Token is\r\n
(CRLF) and advances to the next Token, if not it returns a "token expected" error. -
parseIdent
: It first uses thep.skipBulkString
function (defined bellow) to checks if the following set of Tokens describes aBulkString
and advances the Tokens till the actual text.
Then it checks if the current Token is an Identifier (token.IDENT
) that contains the actual key of theGET
command returning the command if parsed successfully.
Now the only thing left is creating the skipBulkString
function
which you may have seen in the last snippets.
func (p *Parser) skipBulkString() error {
if !p.curTokIs(token.BULKSTRING) {
return p.isNotBulkStringErr()
}
p.next()
if !p.curTokIs(token.NUMBER) {
return p.isNotNumberErr()
}
p.next()
if err := p.checkCRLF(); err != nil {
return err
}
return nil
}
As you can see this functions just returns a error if some of the operations fail. Let's see what it does:
-
Checks if the
curTok
is a$
(BulkString) advancing the Tokens. -
Checks if the
curTok
is aNumber
representing the length of the actual text and then it advances the Tokens with thep.next()
method. -
Checks if the
curTok
is a\r\n
(CRLF) therefore checking if it's a valid BulkString.
This have been a tough section, so let's do a recap.
Parser Recap
So, we have gone a long way, don't worry if you think that you didn't get Parsers, in fact, this is quite a complicated topic.
So let's see how Parsers
work in a more graphical way.
Parsers consume Tokens: To be able to Parse complex Command we
use a curTok
and nextToken
.
As you can see by default we put the curTok
and nextTok
in the first two Tokens of the Command (in this example GET and userName).
We use the content of this Tokens to fill the GetCommand
structure
with its representing data.
You also have to remember that this is the raw representation
of the Command.
In fact the Parser will get something like this.
We need to advance through the Tokens to be able to Parse the whole Command,
so we need a function that advances our 'two-token window' (curTok
and nextTok
).
The next()
method achieves this purpose each time we call it.
After calling next()
:
As you can see after calling next()
now both Tokens have advances
one position.
One we encounter a GET
token, we execute a special function
that parses the required fields of the Command, in this case parseGetCommand()
.
NOTE: As you may have guested all of the words and symbols are Tokens and Parsing is the process of going through them and storing the useful information.
With this, we are almost done with this article, we just need to Evaluate Commands.
Creating Values
This is the part of the article when everything comes together, but before we do that we have to create a way to store the value of our keys. So, let's do it!
All of the Values should have its corresponding kind we will focus on Nil
and
String
value kinds, let's define the value kinds first and then we will store its values.
type valueKind byte
const (
StringKind valueKind = iota
NilKind
// ... The rest of the Value Kinds
)
If you notice this is actually pretty similar to the Token logic, in fact, it is. So, let's break it down.
-
valueKind
: Is anEnum
(Enumeration) of the available Value Kinds. -
StringKind
: Represents theString
Value Kind. -
NilKind
: This special kind signifies the absence of a value if a key does not exists we will return aNilVal
(a value ofNilKind
).
We have mention a Value
a couple of times. So, let's define
them too!
As you can see Values
are pretty similar to Tokens
.
-
Kind()
: This method returns thevalueKind
of the value. -
String()
: This method returns the String representation of the current Value.
As Value
is an interface we have to create its corresponding
struct
in this case String
and NilVal
, this structure is going to be the way we are going to store values.
type String struct {
Value string
}
// Returns the String Value stored
func (s *String) String() string { return s.Value }
func (s *String) Kind() valueKind { return StringKind }
type NilVal struct{}
// Retuns the (nil) String
func (n *NilVal) String() string { return "(nil)" }
func (n *NilVal) Kind() valueKind { return NilKind }
As you can see both String
and NilVal
now
implement the Value
interface.
-
String
: It stores a string in itsValue
field. -
NilVal
: It does not represent any Value asNil
represents the absence of one.
We are almost done with values! Let's just create a helper function to create
new values easily, we will call it NewValue
.
func NewValue(v ast.Expression) Value {
switch v := v.(type) {
case *ast.StringExpr:
return &String{v.String()}
// Creating the rest of the values...
}
return nil
}
Have you notice it? This function takes a Expression
and we
still have not explore what a Expression does, so let's do it.
Also this function uses the Expression type to decide which Value
should be returned, if the Expression is not supported it returns
nil
.
Defining Expressions
Do you remember when we were defining Commands
and we briefly
mentioned Expressions
, this is the time when we define
them!
Commands like GET
receive a key parameter which
can be represented a String
. But commands like SET
can receive as a parameter any value (Number
, String
, Bool
, etc...), we wrap all of these values as Expressions
.
type Node interface {
String() string
TokenLiteral() string
}
type Expression interface {
Node
exprNode()
}
Expression
is also a Node
and it uses the
exprNode()
method to differentiate from Commands.
Now that we have defined Expressions let's implement them!
type StringExpr struct {
Token token.Token
}
func (se *StringExpr) exprNode() {}
func (se *StringExpr) TokenLiteral() string {
return se.Token.Literal
}
func (se *StringExpr) String() string {
return se.Token.Literal
}
In this article we will just implement StringExpr
as we will
just support strings. So, let's break it down quickly.
-
StringExpr
: The string it represents is stored in theToken
field as it contains the whole string. -
exprNode
: This method ensures thatStringExpr
implements theExpression
interface, in the same way asCommand
. -
String
: This method provides a string representation of the actualExpression
.
If you want to know the use of Expressions in Commands i highly recommend you to check out the source code!
Now that we have created our Values
it's time to store them!
Storing Values
We are going to take a simple approach when it comes to storing Values
, we are going to create a concurrency safe map which maps string keys to Value
pointers.
NOTE: This implementation was written before the weak
package was released, so keep in mind that now there is a better way to implement the storage of keys.
Knowing this, let's implement our Storage
.
As you can see the storage is just a map of Values
, but
above it you can see a RWMutex
.
A Mutex (mutual exclusion) is one of the most complex and deep
concepts in all of computing, so let's try to explain them quickly.
The Problem of the Five Philosophers and the Limited Forks:
Imagine five brilliant philosophers sitting around a circular table, ready to enjoy some delicious food. Between each philosopher is a single fork. To eat, a philosopher needs two forks: one from their left and one from their right. The problem? There are only five forks for five philosophers, and each fork is a shared resource.

Now, let's say all five philosophers get hungry at the same time
and each reaches for the fork on their left. They all succeed! But now, they're
stuck. Each philosopher has one fork, but they all need a second fork (from their right), which is currently held by their neighbor. No one can eat, and no one can
release their fork because they're all waiting for another – this is a classic
deadlock
. They're all holding onto a resource
and waiting indefinitely for another resource that's also held.

Here's how a mutex
acts as the polite, orderly "fork manager":
- Acquiring the Mutex (Picking up a Fork): When a philosopher
wants to pick up a fork, they first try to 'lock' its
mutex
. If the mutex is unlocked (meaning the fork is available), they successfully acquire themutex
and pick up the fork. - Mutual Exclusion Ensured: If the
mutex
is already locked (meaning another philosopher is holding that fork), our hungry philosopher must wait until themutex
is released. This ensures that only one philosopher can ever hold a specific fork (resource) at a time. - Releasing the Mutex (Putting down a Fork): Once a philosopher
finishes eating, they 'unlock' the
mutexes
for both fork they were holding. This signals to any waiting philosophers that those forks are now available.
By implementing a strategy where philosophers acquire both necessary fork
mutexes
before attempting to eat (or acquiring them in a specific
order, or acquiring a global eating permission mutex), we prevent
the deadlock. The mutex ensures that critical sections of code (like picking up
a shared fork) are accessed by only one goroutine
(philosopher)
at a time, preventing race conditions and resource contention.
In our Redis clone, the sync.RWMutex
acts precisely like these
fork managers, ensuring that when multiple parts of our program
try to read from or write to our Storage
map simultaneously,
they do so in an orderly, deadlock-free fashion, just like
our philosophers eventually get to enjoy their food!"
Now that we have scratched the surface of Mutexes
we are ready
to finish our Storage
!
Evaluating Commands
There are several ways to evaluate commands, some better than others, but we will take a simple approach using the Golang concurrency features.