These are some of the most influential (mostly due to experience or expertise) and active folks (I actually see them attend events) in the NYC infrastructure scene (that I have a personal connection to). If you're running a dinner or are just looking to meet interesting people in NYC in software infrastructure, consider this list and feel free to...
This is an external post of mine. Click here if you are not redirected.
Being unemployed can be incredibly depressing. So much rejection. Everything seems to be out of your control. Everything except for one thing: what you produce. You might know that repeatedly posting on social media that you are looking for work is ineffective. That it looks (or at least feels) worse each time you say so. But there is at least...
You want to check for strict consistency (linearizability) for your project but you don't want to have to deal with the JVM. Porcupine, used by a number of real-world systems like etcd and TiDB, has you covered! Importantly, neither Jepsen projects nor Porcupine can prove linearizability. They can only help you build confidence that you aren't...
Delta Lake is an open protocol for serverless ACID databases. Due to its simplicity, scalability, and the number of open-source implementations, it's quickly becoming the DuckDB of serverless transactional databases for analytics workloads. Iceberg is a contender too, and is similar in many ways. But since Delta Lake is simpler (simple != better)...
I wrote last month that what you want to do is one of the most useful motivations in life. I want to follow that up by saying that the only thing more important than wanting to do something is to actually do something. The most valuable trait you can develop for yourself is to be consistent. It is absolutely something you can develop. And...
In your professional and personal life, I don't believe there is a stronger motivation than having something in mind and the desire to do it. Yet the natural way to deal with a desire to do something is to justify why it's not possible. "I want to read more books but nobody reads books these days so how could I." "I want to write for a magazine...
Bugs in distributed systems are hard to find, largely because systems interact in chaotic ways. And even once you've found a bug, it can be anywhere from simple to impossible to reproduce it. It's about as far away as you can get from the ideal test environment: property testing a pure function. But what if we could write our code in a way that...
This is an external post of mine. Click here if you are not redirected.
This year has seen a resurgence in really high quality systems programming meetups. Munich Database Meetup, Berlin Systems Group, SF Distributed Systems Meetup, NYC Systems, Bengaluru Systems, to name a few. This post summarizes a bit of disappointing recent tech meetup history, the new trend of excellent systems programming meetups, and ends...
A database does not need a write-ahead log (WAL) to achieve durability. A database can write its long-term data structure durably to disk before returning to a client. Granted, this is a bad idea! And granted, a WAL is critical for durability by design in most databases. But I think it's helpful to understand WALs by understanding what you could...
This is an external post of mine. Click here if you are not redirected.
Some of the most interesting technical blog posts I read come from, and a common reason for posts I write is, confusion. You're at work and you start asking questions that are difficult to answer. You spend a few hours or a day trying to get to the bottom of things. If you ask a question to very experienced and successful developers at work, they...
I've been running software book clubs almost continuously since last summer, about 12 months ago. We read through Designing Data-Intensive Applications, Database Internals, Systems Performance, and we just started Understanding Software Dynamics. The DDIA discussions were in-person in NYC with about 5-8 consistent attendees. The rest have been...
In this post we'll build a database in 400 lines of code with basic support for five standard SQL transaction levels: Read Uncommitted, Read Committed, Repeatable Read, Snapshot Isolation and Serializable. We'll use multi-version concurrency control (MVCC) and optimistic concurrency control (OCC) to accomplish this. The goal isn't to be perfect...
I want to explain why the blogs in My favorite technical blogs are my favorite. That page is solely about non-corporate tech blogs. So this post is too. I'll have to make another list for favorite corporate tech blogs. In short, they: Tackle hard and confusing topics Show working code Make things simpler Write regularly Talk about tradeoffs and...
I started a paper reading club this week at work, focused on databases and distributed systems research. I posted in a general channel about the premise and asked if anyone was interested. I clarified the intended topic and that discussions would be asynchronous over email, run fortnightly. Weekly seemed way too often. A month seemed way too...
This is an external post of mine. Click here if you are not redirected.
Having worked a bit in Zig, Rust, Go and now C, I think there are a few common topics worth having a fresh conversation on: automatic memory management, the standard library, and explicit allocation. Zig is not a mature language. But it has made enough useful choices for a number of companies to invest in it and run it in production. The useful...
A little over a month ago, I joined EnterpriseDB on a distributed Postgres product (PGD). The process of onboarding myself has been pretty similar at each company in the last decade, though I think I've gotten better at it. The process is of course influenced by the team, and my coworkers have been excellent. Still, I wanted to share my thought...
Distributed consensus in transactional databases (e.g. etcd or Cockroach) is a big deal these days. Most often under the hood are variations of log-based Paxos-like algorithms such as MultiPaxos, Viewstamped Replication, or Raft. While there are new variations that come out each year, optimizing for various workloads, these algorithms are fairly...
I spent a week looking at MySQL/MariaDB internals along with ~80 other devs. Although MySQL and MariaDB are mostly the same (more on that later), I focused on MariaDB specifically this week. Before last week I had never built MySQL/MariaDB before. The first day of this hack week, I got MariaDB building locally and made a code tweak so that SELECT...
Over the years, I have repeatedly felt like I missed the timing for a meetup or an IRC group or social media in general. I'd go to a meetup every so often but I'd never make a meaningful connection with people, whereas everyone else knew each other. I'd join an IRC group and have difficulty catching up with what seemed to be the flow of...
I learned this week that you can intercept and redirect Postgres query execution. You can hook into the execution layer so you're given a query plan and you get to decide what to do with it. What rows to return, if any, and where they come from. That's very interesting. So I started writing code to explore execution hooks. However, I got stuck...
With Postgres 12, released in 2019, it became possible to swap out Postgres's storage engine. This is a feature MySQL has supported for a long time. There are at least 8 different built-in engines you can pick from. MyRocks, MySQL on RocksDB, is another popular third-party distribution. I assume there will be a renaissance of Postgres storage...
King and I wrote a blog post about building an event-driven cross-platform IO library that used io_uring on Linux. We sketched out how it works at a high level but I hadn't yet internalized how you actually code with io_uring. So I strapped myself down this week and wrote some benchmarks to build my intuition about io_uring and other IO models. I...
The most popular SQLite and PostgreSQL database drivers in Go are (roughly) 20-76% slower than alternative Go drivers on insert-heavy benchmarks of mine. So if you are bulk-inserting data with Go (and potentially also bulk-retrieving data with Go), you may want to consider the driver carefully. And you may want to consider avoiding database/sql....
How software fails is interesting. But real-world errors can be infrequent to manifest. Fault injection is a formal-sounding term that just means: trying to explicitly trigger errors in the hopes of discovering bad logic, typically during automated tests. Jepsen and ChaosMonkey are two famous examples that help to trigger process and network...
Databases are fun. They sit at the confluence of Computer Science topics that might otherwise not seem practical in life as a developer. For example, every database with a query language is also a programming language implementation of some caliber. That doesn't include all databases though of course; see: RocksDB, FoundationDB, TigerBeetle, etc....
This is a collection of random personal experiences. So if you don't want to read everything, feel free to skip to the end for takeaways. I write because I'd like to see more high-quality meetups. And maybe my little bit of experience will help someone out. 2015: PhiladelphiaI first tried to organize a meetup in Philly in 2015. I was contracting...
Someone on Discord asked about how to learn functional programming. The question and my initial tweet on the subject prompted an interesting discussion with Shriram Krishnamurthi and other folks. So here's a slightly more thought out exploration. And just for backstory sake: I spent a few years a while ago programming in Standard ML and I wrote a...
This is an external post of mine. Click here if you are not redirected.
I knew Zig supported some sort of reflection on types. But I had been confused about how to use it. What's the difference between @typeInfo and @TypeOf? I ignored this aspect of Zig until a problem came up at work where reflection made sense. The situation was parsing and storing parsed fields in a struct. Each field name that is parsed should...
As part of bringing myself up-to-speed after joining TigerBeetle, I wanted some background on how distributed consensus and replicated state machines protocols work. TigerBeetle uses Viewstamped Replication. But I wanted to understand all popular protocols and I decided to start with Raft. We'll implement two key components of Raft in this post...
Originally published on December 5, 2021. If you are an experienced software developer whose only exposure to reddit is dank memes, proggit or even language-specific subreddits like /r/python, you're missing out. What follows are my favorite subreddits in tech. My criteria is that: The subreddit topic is relevant to advancing as a programmer...
Originally published on February 1, 2021. The original version included two books I don't think are actually so worthwhile. This list is down to two. I think that's a good thing actually. These are the books I recommend to developers wanting to improve their skills as professional programmers because of high information density, believable...
At TigerBeetle these last few weeks I've been doing a mix of documenting client libraries, writing sample code for client libraries, and writing integration tests against the sample code. The client library documentation is generated with a Zig script. The sample code is integration tested with a Zig script. A bunch of Zig scripts. It's not the...
Disney was a celebrity by his mid-30s, Disney the company was famous by 1930s. Even though politically the 1930s was considered the decade of Roosevelt (elected President in 1933), culturally the 1930s was considered the decade of Mickey Mouse. Almost every new animation/filmmaking technique they tried, they would experiment with it in shorts...
I wanted to practice making coding videos so I did a four-part series on writing a basic Scheme-like language (minus macros and arrays and tons of stuff). I picked this simple topic because I wanted a low-stakes way to learn what I did not know about making videos. Here was the end result (nothing crazy): $ go build $ cat examples/fib.scm (func...
There are three specific activities I have loved in some product managers I've worked with (and missed in others). tldr; Talk with customers and prospects Develop and share a vision Evangelize Talk with customers and prospectsAs a product manager, your superpower over engineering is to have spent time with customers and prospects. You should have...
In 2022 I finished 20 books spanning 15,801 pages. 3 more than I read in 2021, but about twice the number of pages. 3 fiction and 17 non-fiction. Another ~30 started but not finished. I had a hard time reading books while I was trying to start my own company. But I also discovered audiobooks. I would put on a book and listen while I did my...
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
You acquire a skill or experience through time and effort, then downplay the impact of writing and sharing the learning process. Professionals seem naturally to imagine a high bar for what is worth writing about. I think that's misguided. This article is not criticism of folks with these beliefs, but rather encouragement for folks looking for a...
This is an external post of mine. Click here if you are not redirected.
For my second project while learning Zig, I decided to port an old, minimal SQL database project from Go to Zig. In this post, in ~1700 lines of code (yes, I'm sorry it's bigger than my usual), we'll create a basic embedded SQL database in Zig on top of RocksDB. Other than the RocksDB layer it will not use third-party libraries. The code for this...
I mostly programmed in Go the last few years. So every time I wanted an embedded key-value database, I reached for Cockroach's Pebble. Pebble is great for Go programming but Go does not embed well into other languages. Pebble was inspired by RocksDB (and its predecessor, LevelDB). Both were written in C++ which can more easily be embedded into...
This is an external post of mine. Click here if you are not redirected.
When I wrote the "build a distributed PostgreSQL proof of concept" post I first had to figure out how to use Hashicorp's Raft implementation. There weren't any examples I could find in the Hashicorp repo itself. And the only example I could find was Philip O'Toole's hraftd. It's great! However, I have a hard time following multi-file examples in...
Let's assume you're familiar with basic SQL databases like PostgreSQL and MySQL, and document databases like MongoDB and Elasticsearch. You probably know Redis too. But you're hearing more and more about embedded key-value stores like RocksDB, LevelDB, PebbleDB, and so on. And you're hearing about distributed key-value databases like FoundationDB...
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
In this post we'll build a basic jq clone in Go. It will only be able to pull a single path out of each object it reads. It won't be able to do filters, mapping, etc. $ cat large-file.json | head -n2 | ./jqgo '.repo.url' "https://api.github.com/repos/petroav/6.828" "https://api.github.com/repos/rspt/rspt-theme" We'll start by building a "control"...
This is an external post of mine. Click here if you are not redirected.
What is CockroachDB under the hood? Take a look at its go.mod and notice a number of dependencies that do a lot of work: a PostgreSQL wire protocol implementation, a storage layer, a Raft implementation for distributed consensus. And not part of go.mod but still building on 3rd party code, PostgreSQL's grammar definition. To be absurdly...
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
This is an external interview. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
In this post we'll write a rudimentary document database from scratch in Go. In less than 500 lines of code we'll be able to support the following interactions, inspired by Elasticsearch: $ curl -X POST -H 'Content-Type: application/json' -d '{"name": "Kevin", "age": "45"}' http://localhost:8080/docs...
This is an external post of mine. Click here if you are not redirected.
I've never run my own mail server before. Before today I had no clue how email worked under the hood other than the very few times I've set up mail clients. I've heard no few times how hard it is to send mail from a self-hosted server (because of spam filters). But how hard can it be to hook up DNS to my personal server and receive email to my...
This is an external post of mine. Click here if you are not redirected.
Many "must-read" books are not well-written. I try to read a lot, but I still have a low tolerance for bad writing and bad editing. I write this post both to discourage thoughtless recommendations and to encourage the receivers of bad recommendations. For software developers, Structure and Interpretation of Computer Programs is a prime example....
I spent a few days playing around with bootloaders for the first time. This post builds up to a text editor with a few keyboard shortcuts. I'll be giving a virtual talk based on this work at Hacker Nights on Jan 27. There are a definitely bugs. But it's hard to find intermediate resources for bootloader programming so maybe parts of this will be...
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
Last year (2021) I finished 17 books, a five year low. But that's ok! 4 fiction and 13 non-fiction. Another 30 started but not finished. Non-fictionIt seems I was pretty focused on business history books and history of tech. The 8 non-fiction books I liked the most: Designing Data-Intensive Applications, a must-read for anyone interacting with a...
By the end of this guide we'll have a minimal, working implementation of a small part of Lua from scratch. It will be able to run the following program (among others): function fib(n) if n 2 then return n; end local n1 = fib(n-1); local n2 = fib(n-2); return n1 + n2; end print(fib(30)); This is my second project in Rust and...
This is an external post of mine. Click here if you are not redirected.
All code for this post is available on Github. Let's take a look at how zip files work. Take a small file for example: $ cat hello.text Hello! Let's zip it up. $ zip test.zip hello.text adding: hello.text (stored 0%) $ ls -lah test.zip -rw-r--r-- 1 phil phil 177 Nov 23 23:04 test.zip So a 6 byte text file becomes a 177 byte zip file. That is...
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
Previously in exploring PL/pgSQL: Strings, arrays, recursion and parsing JSON In my last post I walked through the basics of PL/pgSQL, the embedded procedural language inside of PostgreSQL. It covered simple functions, recursions and parsing. But there was something very obviously missing from that post: a working interpreter. So in this...
Next in exploring PL/pgSQL: Implementing a Forth-like interpreter PostgreSQL comes with a builtin imperative programming language called PL/pgSQL. I used to think this language was scary because it has a bit more adornment than your usual language does. But looking deeper, it's actually reasonably pleasant to program in. In this post we'll...
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
V8 is, I'm sure, the most used implementation of JavaScript today. Used in Chrome, (and by extension) Microsoft Edge, Node.js, etc. Safari's JavaScriptCore and Firefox's SpiderMonkey are also contenders for extremely mainstream implementations. But what else is out there? What if I want to embed JavaScript in a C program, or a Go program, or a...
Modern C++ has a lot of cool features. Move semantics means passing around structs in functions is cheap. std::shared_ptr means I don't have to manage any memory; no more new/delete! (But try as I might to understand std::unique_ptr, I'm just not there yet.) The syntax has also gotten some treatment with auto and tuple destructuring. In order to...
Developers often think parser generators are the sole legit way to build programming language frontends, possibly because compiler courses in university teach lex/yacc variants. But do any modern programming languages actually use parser generators anymore? To find out, this post presents a non-definitive survey of the parsing techniques used by...
In a ridiculous attempt to prove an internet wrong about the practicality of Lisp (Common Lisp specifically), I tried to get a simple (but realistic) web app running. After four days and a patch to ABCL I got something working. The code I had in mind would look something like this: (let* ((port 8080) (server (make-server port))) (route...
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
This is an external post of mine. Click here if you are not redirected.
This post goes out to anyone who leads a team: managers, directors, VPs, executives. You need to share organization success stories with your organization on a regular and frequent basis. Talk about sales wins, talk about new services released, talk about the positive impact of a recent organizational change. Just get in front of your entire...
This is an external post of mine. Click here if you are not redirected.
For years I've kept a private list of really cool tech companies in NYC. Now that I'm funemployed it's the perfect time to publish. This list is influenced by 1) my perception of the difficulty of the engineering behind the product and 2) the company's educational and OSS presence. With no further ado and in no particular order, here's my list!...
In this post we'll build a minimal text templating library in Python inspired by Jinja. It will be able to display variables and iterate over arrays. By the end of this article, with around 300 lines of code, we'll be able to create this program: from pytemplate import eval_template template = ''' {% for-in(post, posts) %} {{...
I have never contributed to nginx. My C skills are 1/10. But downloading the source, hacking it up, compiling it, and running it doesn't scare me. This post is to help you overcome your own fears about doing so. Not necessarily because you should be running out-of-tree diffs in production but because I see a lot of developers never even consider...
tldr; reimplement standard library functions in your favorite language without loops. BackgroundFor a few years after college I spent a lot of free time doing projects in Standard ML and Scheme. As a result I got really comfortable doing recursion. The two big reasons for this are 1) neither Standard ML or Scheme have loops and 2) they both have...
It's been a few months since I picked up gosql and I wanted to use it to prototype a SQL interface for data stored in S3. But one missing critical feature in gosql is LIMIT and OFFSET support. This post walks through the few key changes to gosql to support LIMIT and OFFSET. You can find this commit in full on Github. This post builds on top of...
This year I finished 47 books, up from last year but not a personal best. The breakdown was 17 non-fiction and 30 fiction. Another 20-30 remain started but unfinished this year. Non-fictionThe 8 non-fiction books I most recommend are: Fashionapolis: The Price of Fast Fashion and the Future of Clothes (Must read) Effective Python: 90 Specific Ways...
In this post we'll get a basic semgrep environment set up in Docker running some custom rules against our code. Existing lintersLinters like pylint for Python or eslint for JavaScript are great for general, broad language standards. But what about common nits in code review like using print statements instead of a logger, or using a defer...
In this post we'll stumble toward a working emulator for a barebones C program compiled for linux/AMD64. The approach will be slightly more so based on observation than by following a spec; a great way to quickly become familiar with a topic, and a bad way to guarantee correctness. The goal: $ cat tests/simple.c int main() { return 4; } $ gcc...
Ambitious companies form management teams at every level above you, sometimes including you. Management teams meet periodically and have private chat rooms. They discuss customers, product and organizational direction. Sometimes discussions are well documented and periodically public. Sometimes decisions are poorly telegraphed out. Management...
Incredibly, Standard ML implementations are still actively developed. MLton, Poly/ML, MLKit, SML# and SML/NJ are the most prominent. Discussion on the future direction of Standard ML remains healthy as well. And somehow OCaml's lesser known cousin still beats out OCaml for multicore threading support (in Poly/ML). While MLton hasn't merged with...
When I first started programming, especially when asked for code samples, my comments lacked purpose and would often duplicate in English what the code clearly indicated. I knew that "commenting is good" but as a beginner I had no further insight. Over time with the help of books like Clean Code, I grew disdainful of comments. Good code should be...
In this post we'll write a Python to C compiler in Python. This is especially easy to do since Python has a builtin parser library and because a number of CPython internals are exposed for extension writers. By the end of this post, in a few hundred lines of Python, we'll be able to compile and run the following program: $ cat...
This post is a recipe for setting up a minimal Kubernetes cluster on Fedora without requiring virtualization or a container registry. These two features make the system cloud-agnostic and the cluster entirely self-contained. The post will end with us running a simple Flask app from a local container. This setup is primarily useful for simple CI...
DBCore can now generate a TypeScript/React CRUD UI that is automatically hooked up to the generated REST API (in Go). The UI has full support for login, viewing (and filtering), editing, and creating database entities. PostgreSQL, SQLite and MySQL are supported. How to use?The goal of this project is primarily to provide as much useful...
I recently published an alpha version of a code generation tool, DBCore, that reads a database schema from PostgreSQL or MySQL and generates an entire Go API with CRUD operations, pagination, filtering, and authentication. But more than just generating code like xo/xo or gnorm, DBCore defines a standard REST API that can be implemented in any...
I hated writing documentation before working on features. But after a while I realized I couldn't communicate well enough, even with folks I had a good connection with. It took me a number of mistaken deliveries to get the message. Sketches and mockupsDesigners solve this by producing low-fidelity sketches early on in the process, iterating on...
Previously in database basics: 1. SELECT, INSERT, CREATE and a REPL 2. binary expressions and WHERE filters 3. indexes In this post, we'll extend gosql to implement the database/sql driver interface. This will allow us to interact with gosql the same way we would interact with any other database. Here is an example familiar...
Previously in database basics: 1. SELECT, INSERT, CREATE and a REPL 2. binary expressions and WHERE filters Next in database basics: 4. a database/sql driver In this post, we extend gosql to support indexes. We focus on the addition of PRIMARY KEY constraints on table creation and some easy optimizations during SELECT...
Previously in database basics: 1. SELECT, INSERT, CREATE and a REPL Next in database basics: 3. indexes 4. a database/sql driver In this post, we'll extend gosql to support binary expressions and very simple filtering on SELECT results via WHERE. We'll introduce a general mechanism for interpreting an expression on a row...
The only time I've been able to seriously, rapidly improve my ability to speak a foreign language was through intensive language courses in college. I was forced to actively speak, read, and write Chinese for 6-8 hours a week (1-2 hours every day). Then study another 5-10 hours a week in preparation for the active sessions. I went three semesters...
The first few paragraphs cover what I was looking for and what I considered. Then the review. Why the Surface Book 2I used a Macbook throughout my professional career until I had the choice a few years ago when I started my current job. Here, I ran Gentoo, then FreeBSD, then Arch, and now Windows 10 on the Dell XPS 15. I enjoy Windows and I think...
Next in database basics: 2. binary expressions and WHERE filters 3. indexes 4. a database/sql driver In this series we'll write a rudimentary database from scratch in Go. Project source code is available on Github. In this first post we'll build enough of a parser to run some simple CREATE, INSERT, and SELECT queries. Then we'll...
There's a style of Java that is a joy to write. This post will cover how to set up a basic PostgreSQL-integrated REST API using Jersey and JOOQ in a style not dissimilar to Flask and SQLAlchemy in Python. In particular, we'll try to avoid as much runtime reflection/class-loading as possible. This will make the application less flexible but easier...
Previously in compiler basics: 1. lisp to assembly 2. user-defined functions and variables 3. LLVM 4. LLVM conditionals and compiling fibonacci 5. LLVM system calls This post upgrades the ulisp x86 backend from using a limited set of registers (with no spilling support) to solely using the stack to pass values between...
The quickest way to cut through confusion or disagreement among otherwise amiable and honest folks is to ask questions. Ask early so you don't waste time. But it's not enough to just ask clarifying questions because the answers won't always be clear. Sounds like Human Interaction 101, and maybe it is. These techniques show up more when discussing...
After spending some time at work on tooling for keeping documentation in sync with Go struct definitions I had enough exposure to Go's built-in parsing package that next steps were clear: write an interpreter. It's a great way to get more comfortable with a language's AST. In this post we'll use the Go parser package to interpret the AST directly...
Kubernetes is easy to use after some exposure; it's pretty convenient too. But it is super hard to set up. eksctl is a good tool for folks who don't want to spend hours/days/weeks debugging VPC configuration in 1000s of lines of CloudFormation. None of the other tools seem to be that much easier to use (kops, kubeadm, etc.). But even with EKS and...
This post covers building and testing a minimal, but still useful, C project. We'll use Google's gtest and CMake for testing C code. This will serve as a foundation for some upcoming posts/projects on programming Linux, userland networking and interpreters. The first version of this post only included one module to test. The...
Previously in emulator basics: 1. a stack and register machine In this post we'll extend x86e to support the exit and write Linux system calls, or syscalls. A syscall is a function handled by the kernel that allows the process to interact with data outside of its memory. The SYSCALL instruction takes arguments in the same order that the...
Previously in compiler basics: 1. lisp to assembly 2. user-defined functions and variables 3. LLVM 4. LLVM conditionals and compiling fibonacci Next in compiler basics: 5. an x86 upgrade In this post we'll extend the ulisp compiler's LLVM backend to support printing integers to stdout. Exit code limitationsUntil...
Better yet, take a look at this post walking through emulating x86 ELF binaries in Go: Emulating linux/AMD64 userland: interpreting an ELF binary Next up in emulator basics: 2. system calls In this post we'll create a small virtual machine in JavaScript and use it to run a simple C program compiled with GCC for an x86_64 (or...
In this post we'll explore what tail calls are, why they are useful, and how they can be eliminated in an interpreter, a compiler targeting C++, and a compiler targeting LLVM IR. Tail callsA tail call is a function call made at the end of a block that returns the value of the call (some languages do not force this return requirement). Here are a...
Previously in compiler basics: 1. lisp to assembly 2. user-defined functions and variables 3. LLVM Next in compiler basics: 5. LLVM system calls 6. an x86 upgrade In this post we'll extend the compiler's LLVM backend to support compiling conditionals such that we can support an implementation of the fibonacci...
Responsibility is only possible by granting ownership and setting expectations. If you don't turn over ownership, don't expect folks to take responsibility. When you grant ownership and set expectations, you'll be astounded what folks will accomplish without you. I am astounded.
In addition to providing a static type system and compiler for a superset of JavaScript, TypeScript makes much of its functionality available programmatically. In this post we'll use the TypeScript compiler API to build an interpreter. We'll build off of a TypeScript wiki article and cover a few areas that were confusing to me as I built out a...
Say we have some HTML: html> body> h1>Hello world!h1> body> html> And say we'd like to be able to render this page in a web browser. If the server is hosted locally we may want to enter localhost:9000/hello-world.html in the address bar, hit enter, make a request (done by the browser), receive a response (sent by some server), and render...
Let's say we want to implement a simple list filtering language so we can enter a.b = 12 and return only results in a list where the a column is an object that contains a field b that is set to the value 12. What would a filter(jsonPath, equals, listOfObjects) function look like? If we only needed to support object lookup, we might implement...
Previously in compiler basics: 1. lisp to assembly 2. user-defined functions and variables Next in compiler basics: 4. LLVM conditionals and compiling fibonacci 5. LLVM system calls 6. an x86 upgrade In this post we'll extend the compiler to emit LLVM IR as an option instead of x86 assembly. All source code is...
tldr; I'm working on a AOT-compiled Javascript implementation called jsc. Many dynamically typed programming languages have implementations that compile to native binaries: Python: Cython Common Lisp: SBCL Scheme: Chicken Scheme The benefits of compiling dynamically typed languages are similar to those of compiling statically typed languages:...
I saw a post on dev.to that talks about dysfunctional teams. This is a response that focuses specifically on how to prevent burnout from overworking. This is aimed at senior/lead engineers and engineering/project managers -- because everyone in a leadership role is responsible for the health of the team and the company. In an otherwise good...
Previously in compiler basics: 1. lisp to assembly Next in compiler basics: 3. LLVM 4. LLVM conditionals and compiling fibonacci 5. LLVM system calls 6. an x86 upgrade In this post we'll extend the compiler to support defining functions and variables. Additionally, we'll require the program's entrypoint to be...
It has been six years since I last used Windows for any remotely serious software development. I've used Ubuntu, Arch, or FreeBSD since. But eventually I spent so much time working around common workplace tasks that I decided to put Windows 10 Pro on my work laptop. Windows Subsystem for LinuxIntroduced in 2016, this technology allows Windows to...
Two frustrating things that can happen in an organization are 1) big changes and 2) changes that aren’t clearly associated with a known problem. It’s even worse in that order. These situations tend to happen when a problem remain unaddressed for too long. These situations tend to happen when there is not a strong enough emphasis on respect for...
Next in compiler basics: 2. user-defined functions and variables 3. LLVM 4. LLVM conditionals and compiling fibonacci 5. LLVM system calls 6. an x86 upgrade In this post we'll write a simple compiler in Javascript (on Node) without any third-party libraries. Our goal is to take an input program like (+ 1 (+ 2 3)) and...
I’ve lived in NYC for the past year — moved here after years in Philly and after growing up in a rural community a few hours west of there. My wife is South Korean and last week concluded my second trip to the suburbs of Seoul to visit her family. We finished up that trip with a week in Tokyo. Long a mecha and Godzilla fan, I was struck by a city...
The last time I read for fun was in elementary school. Since college I knew I must read more, but I never forced myself to build the habit. Then three years ago I spent time around my brother and a coworker who were avid readers. This "peer pressure" helped me get started. Since I started, I've seen concrete improvements in vocabulary. I find...
It can be difficult to disassociate the idea that dynamically typed programming languages are tied to byte-code interpreters (e.g. YARV Ruby, CPython, V8, Zend Engine, etc.). But for many languages, a compiled implementation also exists. Cython, Chicken Scheme and SBCL are good examples. In this post I will briefly describe how I built a compiler...
btest is a minimal, language-agnostic test runner originally written for testing compilers. Brian, an ex- co-worker from Linode, wrote the first implementation in Crystal (a compiled language clone of Ruby) for testing bshift, a compiler project. The tool accomplished exactly what I needed for my own language project, BSDScheme, and had very few...
There is a common struggle in the writing and maintenance of documentation, checklists, emails, guides, etc. Each provides immense value; a document may be the key to an important process. The goal is to remove barriers -- to encourage understanding and correct application of what has been noted -- without requiring a change in the character of...
Writing a JSON parser is one of the easiest ways to get familiar with parsing techniques. The format is extremely simple. It's defined recursively so you get a slight challenge compared to, say, parsing Brainfuck; and you probably already use JSON. Aside from that last point, parsing S-expressions for Scheme might be an even simpler task. If...
I've been using FreeBSD as my daily driver at work since December. I've successfully done my job and I've learned a hell of a lot forcing myself on CURRENT... But there's been a number of issues with it that have made it difficult to keep using, so I replaced it with Arch Linux yesterday and I no longer have those issues. This is not the first...
Score: 4.5 / 5Paul Graham and his editor(s) are excellent. His prose is light and easy to follow. The only awkward component of the book's organization is that he tends to use a concept one section before explicitly introducing and defining that concept. I'm not sure yet if this is a good or bad thing. As a learning resourceAmong books...
If you've only vaguely heard of Lisp before or studied Scheme in school, Common Lisp is nothing like what you'd expect. While functional programming is all the rage in Scheme, Common Lisp was "expressly designed to be a real-world engineering language rather than a theoretically 'pure' language" (Practical Common Lisp). Furthermore, SBCL -- a...
This is an external post of mine. Click here if you are not redirected.
I started working on BSDScheme last October, inspired to get back into language implementation after my coworker built bshift, a compiler for a C-like language. BSDScheme is an interpreter for a (currently small subset of) Scheme written in D. It implements a few substantial primitive functions (in under 1000 LoC!). It uses the same test...
I became a FreeBSD user over 2 years ago when I wanted to see what all the fuss was about. I swapped my y410p dual-booting Windows / Ubuntu with FreeBSD running Gnome 3. I learned a lot during the transition and came to appreciate FreeBSD as a user. I soon began running FreeBSD as my OS of choice on cloud servers I managed. So naturally, when I...
Racket is an impressive language and ecosystem. Compared to Python, Racket (an evolution of Scheme R5RS is three years younger. It is as concise and expressive as Python but with much more reasonable syntax and semantics. Racket is also faster in many cases due in part to: JIT compilation on x86 platforms support for both concurrency and...