Following Thorsten Ball's Writing An Interpreter In Go
I've always been drawn to books as a way to learn, and at some point I found myself wanting a serious project involving parsers — something that wasn't going to feel like a toy, something that would demand real focus over a long period of time. So I did what felt natural: I searched for a book about parsers and Go. Writing An Interpreter In Go by Thorsten Ball was the first result that came up. I picked it up, and about six months later I had built a fully functional programming language from scratch — no libraries, no shortcuts, just Go and a much deeper understanding of how interpreters work under the hood. This is the story of that project: dyxgou/interpreter.
The Monkey Programming Language
The language implemented here is Monkey — a small, expressive language invented by Thorsten Ball which implementes a rich set of features.
IntegerandBooleans.Stringmanipulation- Variables and bindings (
letstatements) - First-class and higher-order functions
- Closures
- Arrays and hash maps
-
A built-in library (e.g.,
len,print,first,last,push,pop)
Here's a taste of what Monkey code looks like:
let fibonacci = fn(x) {
if (x == 0) {
0
} else {
if (x == 1) {
return 1;
} else {
fibonacci(x - 1) + fibonacci(x - 2);
}
}
};
print(fibonacci(10)) // 55 You can find more examples in the example/ folder of the repository.
Architecture: How the Interpreter Works
The interpreter is structured as a classic pipeline — source code enters one end, and computed results come out the other. There are four core components, each with a clearly defined responsibility.
-
The
Lexer(src/lexer)The
Lexeris the very first stage of interpretation. Its job is to read raw source code — and convert it into a stream of. A token is a meaningful unit of the language: a keyword likeletorfn, an operator like+or==,an integer, an identifier, or a delimiter like{and}. For example, the codelet x = 5 + 3;gets transformed into a sequence like:The
Lexerdoesn't care whether the code makes sense — that's someone else's problem. It only cares about recognizing and categorizing individual pieces of text. -
The
AST(src/ast)Before the
Parsercan do anything useful, we need a data structure to represent the shape of the program. That's the Abstract Syntax Tree (AST). It's a tree of nodes where each node represents a syntactic construct: aStatement, anExpression, aFunction, aninfix operation, and so on.For example, the expression
5 + 3 * 2doesn't get treated as a flat list — it gets structured as a tree that encodes operator precedence:InfixExpression(+) ├── IntegerLiteral(5) └── InfixExpression(*) ├── IntegerLiteral(3) └── IntegerLiteral(2)The AST package defines all the
Nodetypes and ensures every node in the tree implements a common interface, making it easy for theParserandEvaluatorto work with them uniformly. -
The
Parser(src/parser)The
Parsertakes theTokenstream from theLexerand builds theAST. This is the most intellectually interesting component of the whole project.The implementation follows a Pratt parser (also called a top-down operator precedence parser), a technique described elegantly in the book. The key insight of a Pratt parser is that each token type can have prefix and infix parse functions associated with it. When the parser encounters a
Token, it calls the appropriate parsing function, which returns anASTnode. This approach handles operator precedence naturally and elegantly, without resorting to complicated grammar rules.The parser also performs syntax validation — if the token stream doesn't form a valid Monkey program, the parser reports errors rather than silently producing garbage.
-
The
Evaluator(src/evaluator)The
Evaluatoris where the magic happens. It walks theASTrecursively — a tree-walking interpreter — and computes the value of everyNodeit visits.-
An
IntegerLiteralnode evaluates to its numeric value. -
An
InfixExpressionnode evaluates both sides and applies the operator. -
An
IfExpressionevaluates the condition, then takes the appropriate branch. -
A
FunctionLiteralcaptures the outer environment. -
A
CallExpressionextends the environment, binds arguments, and evaluates the function body.
The
Evaluatoralso manages theObjectsystem — every value in Monkey (integers, strings, booleans, arrays, hashes, functions, null) is represented as a Go struct implementing a commonObjectinterface. This makes it easy to pass values around and pattern-match on their types. -
An
-
The
REPL(src/repl)All of this comes together in a Read-Eval-Print Loop. The
REPLreads a line ofcode, runs it through the full pipeline (lexer → parser → evaluator), and prints the result. You can launch it with a single make command:git clone https://github.com/dyxgou/interpreter cd interpreter makeOr execute a Monkey source file directly:
-
TestingThis project was built following Test Driven Development (TDD), as guided by the book itself. Every component was written test-first — before implementing a feature, leading to an extensive test suite where every major component — the
Lexer,Parser,AST, andEvaluator— has dedicated tests that verify correct behavior across a wide range of inputs, including edge cases and error conditions.
- Repository: github.com/dyxgou/interpreter
- Book: Writing An Interpreter In Go by Thorsten Ball