openmymind.net

In the last post, we looked at some of Zig' weirder syntax. Specifically, this line of code: var gpa = std.heap.GeneralPurposeAllocator(.{}){}; While you'll commonly run into the above when looking at Zig code or going through Zig learning resources, some people pointed out that the code could be improved by doing: var gpa:...

One of the first pieces of Zig code that you're likely to see, and write, is this beginner-unfriendly line: var gpa = std.heap.GeneralPurposeAllocator(.{}){}; While we can reason that we're creating an allocator, the (.{}){} syntax can seem a bit much. This is a combination of three separate language features: generics, anonymous struct literals...

Now that we're more familiar with epoll and kqueue individually, it's time to bring everything together. We'll begin by looking at the possible interaction between evented I/O and threads and then look at writing a basic cross-platform abstraction over the platform-specific epoll and kqueue. Evented I/O + Multithreading We began our journey with...

kqueue is a BSD/MacOS alternative over poll. In most ways, kqueue is similar to the Linux-specific epoll, which itself is important, but important, incremental upgrade to poll. Because kqueue has a single function it superficially looks like poll. But, as we'll soon see, that single function can behave in two different ways, making its API and...

In the last two parts we introduced the poll system call and, with it, patterns around evented I/O. poll has the advantage of being simple to use and supported on most platforms. However, as we saw, we need to iterate through all monitored sockets to figure out which is ready. Another awkwardness with poll's API is associating...

In the previous part we introduced non-blocking sockets and used them, along with the poll system call, to maximize the efficiency of our server. Rather than having a thread-per-connection, waiting on data, a single thread can now manage multiple client connections. But this performance leap doesn't come for free: our code has gotten more...

One of the reasons we introduced multithreading was to get around that fact that our read and, to a lesser extent, accept and write, block. In our initial single-threaded implementation, rather than pushing our server to its limits, we spent a lot of time idle, waiting for data to come in. Multithreading helped to unblock the main thread so that...

We finished Part 1 with a simple single-threaded server, which we could describe as: Create our socket Bind it to an address Put it in "server" mode (i.e. call listen on it) Accept a connection Application logic involving reading/writing to the socket Close the connection Goto step 4 While this approach is useful for getting...

Before we look at making our server multi-threaded, and then move to polling, there are two optimization techniques worth exploring. You might think that we should finalize our code before applying optimizations, but I think optimizations in general can teach us things to look out for / consider, and it's particularly true in both these cases. In...

This part isn't Zig-specific. If you're familiar with length-prefixed messages and binary encoding vs text encoding, you can probably skip it. When thinking about communication between systems using TCP, we generally think about it in terms of messages: an HTTP response, a database record, etc. The implementation though comes down to writing and...

In this series we're going to look at building a TCP server in Zig. We're going to start with a simple single-threaded server so that we can focus on basics. In following parts, we'll make our server multi-threaded and then introduce polling (poll, epoll and kqueue). We begin with a little program that compiles and runs but doesn't do much: const...

In Basic MetaProgramming in Zig we saw how std.meta.hasFn uses @typeInfo to determine if the type is a struct, union or enum. In this post, we'll take expand that introduction and use std.builtin.Type to create our own type. Spoiler: as far as I know, you cannot (yet) define methods on comptime-generated structs, unions or enums. Only fields....

Zig doesn't have a simple way to create closures because closures require allocation, and Zig has a "no hidden allocation" policy. The values that you want to capture need to live somewhere (i.e. the heap), but there's no way for Zig (the language) to create the necessary heap memory. However, outside of the language, either in the standard...

If you've used Zig for a bit, you've probably come across the @memcpy builtin. It copies bytes from one region of memory to another. For example, if we wanted to concat two arrays, we could write a little helper: fn concat(comptime T: type, allocator: std.mem.Allocator, arr1: []const T, arr2: []const T) ![]T { var combined = try...

In Zig an array always has a compile-time length. The length of the array is part of the type, so a [4]u32 is a different type than a [5]u32. But in real-life code, the length that we need is often known only at runtime so we rely on dynamic allocations via allocator.alloc. In some cases the length isn't even known until after we're done adding...

In our last blog post, we saw how builtins like @hasDecl and functions like std.meta.hasMethod can be used to inspect a type to determine its capabilities. Zig's standard library makes use of these in a few place to allow developers to opt-into specific behavior. In particular, both std.fmt and std.json provide developers the ability to define...

While I've written a lot about Zig, I've avoided talking about Zig's meta programming capabilities which, in Zig, generally falls under the "comptime" umbrella. The idea behind "comptime" is to allow Zig code to be run at compile time in order to generate code. It's often said that an advantage of Zig's comptime is that it's just Zig code, as...

Previously On... In Part 1 we saw how Zig's StringHashMap and AutoHashMap are wrappers around a HashMap. HashMap works across different key types by requiring a Context. Here's the built-in context that StringHashMap uses: pub const StringContext = struct { pub fn hash(_: StringContext, s: []const u8) u64 { return std.hash.Wyhash.hash(0,...

I frequently run into a silly compilation error which, embarrassingly, always takes me a couple of seconds to decipher. This most commonly happens when I'm writing tests. Here's a simple example: fn add(values: []i64) i64 { var total: i64 = 0; for (values) |v| { total += v; } return total; } test "add" { const actual = add(&.{1, 2,...

Depending on what you're used to, Zig's built-in test runner might seem a little bare. For example, you might prefer a more verbose output, maybe showing your slowest tests. You might need control over the output in order to integrate it into a larger build process or want to have global setup and teardown functions. Maybe you just want to be...

In the Coding in Zig section of my Learning Zig series, an invalid snippet was recently pointed out to me. The relevant part was: if (try stdin.readUntilDelimiterOrEof(&buf, '\n')) |line| { var name = line; if (builtin.os.tag == .windows) { name = std.mem.trimRight(u8, name, "\r"); } if (name.len == 0) { break; } try...

A bug was recently reported in pg.zig which was the result of a dangling pointer to an ArenaAllocator (1). This amused me since (a) I write a lot about dangling pointers (b) I write a bit about ArenaAllocators and (c) it isn't the first time I've messed this up. (1) Actually, multiple bugs were reported (and fixed), but lets not dwell on that....

In Zig's discord server, I see a steady stream of developers new to Zig struggling with parsing JSON. I like helping with this problem because you can learn a lot about Zig through it. A typical, but incorrect, first attempt looks something like: const std = @import("std"); const Allocator = std.mem.Allocator; const Config = struct { db_path:...

Let's say we wanted to write an HTTP server library for Zig. At the core of this library, we might have a pool of threads to handle requests. Keeping things simple, it might look something like: fn run(worker: *Worker) void { while (queue.pop()) |conn| { const action = worker.route(conn.req.url); action(conn.req, conn.res) catch { //...

As you learn Zig, you'll see examples of memory being allocated and through the use of defer, freed. Often, these allocations and deallocations are wrapped in init and deinit functions. But whatever specific implementation is used, the point is to show a common pattern which is suitable in simple cases. It isn't too much of a leap to take such...

I'm working on an application that needs the ability to schedule tasks. Many applications have a similar need, but requirements can vary greatly. Advanced cases might require persistence and distribution, typically depending on external systems (like a database or queue) to do much of the heavy lifting. My needs are simpler: I don't have a huge...

You'll often find yourself wanting to know the address of a newly created value which gets returned from a function. This can happen for a number of reasons, but the most common is creating bidirectional references. For example, if we create a pool of objects, we often want those objects to reference the pool. You might end up with something...

One consequence of programming without a garbage collector is that you're generally more aware of every memory allocation your program makes. Ideally, this increased awareness results in more prudent memory use. Developers often lament that while hardware has gotten faster, computers feel more sluggish. I don't know how true this is, and if it is...

If you're writing a Zig library, you might find yourself wishing to expose a compile-time configuration option to application developers. One of the reasons you might want to do this for performance reasons, preferring to do something at compile-time versus runtime. Consider this example using my PostgreSQL library: var result = try...

I recently wrote a Prometheus client library for Zig. I had to write a custom hash map context (what Zig calls the eql and hash functions) which led to my last couple blog posts exploring Zig's hash maps in more details. The last topic covered was the getOrPut method which returns a pointer to the key and value arrays of where the entry is or...

In part 1 we explored how the six HashMap variants relate to each other and what each offered to developers. We largely focused on defining and initializing HashMaps for various data type and utilizing custom hash and eql functions for types not supported by the StringHashMap or AutoHashMap. In this part we'll focus on keys and values, how...

This blog posts assumes that you're comfortable with Zig's implementation of Generics. Like most hash map implementations, Zig's std.HashMap relies on 2 functions, hash(key: K) u64 and eql(key_a: K, key_b: K) bool. The hash function takes a key and returns an unsigned 64 bit integer, known as the hash code. The same key always returns the same...

img{border: 1px solid black;margin:0 auto 40px;display:block} I wouldn't mind this huge ad so much if the pancake mix was made with lentil flour...on second thought, that doesn't sound good: And if I complain about not being able to sort by price/quality (or weight), I assume some people will point out that choice adds complexity. But it's hard...

You're probably familiar with Zig's GeneralPurposeAllocator, ArenaAllocator and FixedBufferAllocator, but Zig's standard library has another allocator you should be keeping at the ready: std.heap.MemoryPool. What makes the MemoryPool allocator special is that it can only create one type. As compensation for this limitation, it's very fast when...

A variable associates memory with a specific type. The compiler uses this information to generate the correct instructions (or to tell us that our code is invalid). For example, given this code: const std = @import("std"); const User = struct { id: u32, name: []const u8, }; pub fn main() !void { const user1 = User{.id = 1, .name =...

Maybe I'm wrong, but I believe the canonical way to read a file, line by line, in Zig is: const std = @import("std"); pub fn main() !void { var gpa = std.heap.GeneralPurposeAllocator(.{}){}; const allocator = gpa.allocator(); var file = try std.fs.cwd().openFile("data.txt", .{}); defer file.close(); // Things are _a lot_ slower if we...

When parsing JSON using one of Zig's std.json.parseFrom* functions, you'll have to deal with the return type: an std.json.Parsed(T): const file = try std.fs.openFileAbsolute(file_path, .{}); defer file.close(); var buffered = std.io.bufferedReader(file.reader()); var reader = std.json.reader(allocator, buffered.reader()); defer reader.deinit();...

Deploying code to production is either the funnest part of my day or the most stressful. Where I land on that spectrum largely comes down to how much control I have over the process. Specifically, I like being able to deploy code when I want to deploy it. Process, and people, that don't get this, don't understand software development (and I'm not...

If you're picking up Zig, it won't be long before you realize there's no syntactical sugar for creating interfaces. But you'll probably notice interface-like things, such as std.mem.Allocator. This is because Zig doesn't have a simple mechanism for creating interfaces (e.g. interface and implements keywords), but the language itself can be used...

Language Overview - Part 1 Intro Style Guide Language Overview - Part 2 This part continues where the previous left off: familiarizing ourselves with the language. We'll explore Zig's control flow and types beyond structures. Together with the first part, we'll have covered most of the language's syntax allowing us to tackle more of the...

Language Overview - Part 2 Intro Pointers Style Guide In this short part, we'll cover two coding rules enforced by the compiler as well as the standard library's naming convention. Unused Variables Zig does not allow variables to go unused. The following gives two compile-time errors: const std = @import("std"); pub fn main() void {...

Style Guide Intro Stack Memory Pointers Zig doesn't include a garbage collector. The burden of managing memory is on you, the developer. It's a big responsibility as it has a direct impact on the performance, stability and security of your application. We'll begin by talking about pointers, which is an important topic to discuss in and of...

Pointers Intro Heap Memory & Allocators Stack Memory Diving into pointers provided insight into the relationship between variables, data and memory. So we're getting a sense of what the memory looks like, but we've yet to talk about how data and, by extension, memory is managed. For short lived and simple scripts, this likely doesn't...

Stack Memory Intro Generics Heap Memory and Allocators Everything we've seen so far has been constrained by requiring an upfront size. Arrays always have a compile-time known length (in fact, the length is part of the type). All of our strings have been string literals, which have a compile-time known length. Furthermore, the two types...

Heap Memory & Allocators Intro Coding In Zig Generics In the previous part we built a bare-boned dynamic array called IntList. The goal of the data structure was to store a dynamic number of values. Although the algorithm we used would work for any type of data, our implementation was tied to i64 values. Enter generics, the goal of which...

Generics Intro Conclusion Coding In Zig With much of the language now covered, we're going to wrap things up by revisiting a few topics and looking at a few more practical aspects of using Zig. In doing so, we're going to introduce more of the standard library and present less trivial code snippets. Dangling Pointers We begin by...

Coding In Zig Intro Conclusion Some readers might recognize me as the author of various "The Little $TECH Book" and wonder why this isn't called "The Little Zig Book". The truth is I'm not sure Zig fits in a "The Little" format. Part of the challenge is that Zig's complexity and learning curve will vary greatly depending on your own...