Exploring parsing APIs: the cost of recursion

Especially in high-level languages, inlining is most useful when it causes: Optimizing the callee’s body based on the arguments passed. Optimizing the call site based on the callee’s return value. Let’s look at some examples. Example: avoiding redundant bounds checks Suppose we have a library for decoding some format of binary files with...

In the previous post we looked at three different parsing APIs, and compared them for runtime and the use cases they support. In this post we’ll add a lexer (or “tokenizer”), with two APIs, and for each lexer API see how the parsers from the previous post perform when combined with a lexer. What is a lexer? A lexer is very similar to the event...

Consider a simplified and enhanced version of JSON, with these changes: Numbers are 64-bit unsigned integers. Strings cannot have control and escape characters. Single-line comments are allowed, with the usual syntax: // ... . When parsing a language like this, a common first step if to define an “abstract syntax tree” (AST), with only the...

The main use case of resumable exceptions would be collecting a bunch of errors (instead of bailing out after the first one) to log or show to the user, or actually recovering and continuing from the point of error detection, rather than in a call site. Why not design the code to allow error recovery, instead of using a language feature? There...

Code is tree structured, but manipulated as a sequence of characters. Most language tools1 need to convert these sequence of characters to the tree form as the first thing to be able to do anything. When the program is being edited, the tree structure is often broken, and often to the point where the tool cannot operate. For example: An opening...