Exploring parsing APIs: adding a lexer

Especially in high-level languages, inlining is most useful when it causes: Optimizing the callee’s body based on the arguments passed. Optimizing the call site based on the callee’s return value. Let’s look at some examples. Example: avoiding redundant bounds checks Suppose we have a library for decoding some format of binary files with...

In the first post of this series we looked at a few different ways of parsing a simple JSON-like language. In the second post we implemented a few lexers, and looked at the performance when the parsers from the first post are combined with the lexers in the second post. One of the surprising results in these posts is that our recursive descent...

Consider a simplified and enhanced version of JSON, with these changes: Numbers are 64-bit unsigned integers. Strings cannot have control and escape characters. Single-line comments are allowed, with the usual syntax: // ... . When parsing a language like this, a common first step if to define an “abstract syntax tree” (AST), with only the...

The main use case of resumable exceptions would be collecting a bunch of errors (instead of bailing out after the first one) to log or show to the user, or actually recovering and continuing from the point of error detection, rather than in a call site. Why not design the code to allow error recovery, instead of using a language feature? There...

Code is tree structured, but manipulated as a sequence of characters. Most language tools1 need to convert these sequence of characters to the tree form as the first thing to be able to do anything. When the program is being edited, the tree structure is often broken, and often to the point where the tool cannot operate. For example: An opening...