Marijn pointed out to me that our current setup should avoid the worst of the versioning problems I was afraid of. In the snapshot, we package up a copy of the compiler along with its associated libraries, and use this compiler to produce the new compiler. The new compiler can then compilers its own target libraries, thus avoiding the need to...

In the last few posts I’ve been discussing various options for regions. I’ve come to see region support as a kind of continuum, where the current system of reference modes lies at one end and a full-blown region system with explicit parameterized types and user-defined memory pools lies at the other. In between there are various options. To...

My current implementation of the auto-serialization code generator requires full type information. This is a drag. First, macros and syntax extension currently run before the type checker, so requiring full type information prevents the auto-serialization code from being implemented in the compiler, as it should be. At first I wanted to change...

Cross-crate inlining has come a long way and is now basically functional (I have yet to write a comprehensive test suite, so I’m sure it will fail when exercising various corners of the language). Just for fun, I did some preliminary micro-benchmarks. The results are not that surprising: removing method call overhead makes programs run faster!...

Yesterday we had a hackathon/meeting to discuss the overarching design of Servo, the project to build a next-generation rendering engine. We didn’t ultimately do much hacking (though we did a little), but mostly we tried to hammer out the big picture so that we can actually get to writing code. I wanted to try and write up what I understood as...

pcwalton and I (but mostly pcwalton) have been hard at work implementing regions in Rust. We are hoping to use regions to avoid a lot of memory allocation overhead in the compiler—the idea is to use memory pools (a.k.a. arenas) so that we can cheaply allocate the data needed to process a given function and then release it all in one shot. It is...

First off, I want to welcome Brian Anderson to the Rust blog-o-sphere (which so far consists primarily of myself). His first post does a great job of explaining how to use the new for syntax that was recently added to Rust: this syntax allows for break, ret, and cont from within user-defined loops, which is very nice. Reading some of the Hacker...

On the rust-dev mailing list, someone pointed out another “BitC retrospective” post by Jonathon Shapiro concerning typeclasses. The Rust object system provides interesting solutions to some of the problems he raises. We also manage to combine traditional class-oriented OOP with Haskell’s type classes in a way that feels seamless to me. I thought...

One of the questions in our object system is what precisely how “declared” we want things to be when it comes to interfaces and implementations. In a discussion on IRC, graydon suggested it’d be nice to have terms like “duck-typing” defined more precisely in a Rust syntax, and he is correct. So here is my effort. The current setup Currently,...

I’d like to propose a term for code that has been “over-DRY’d” (dessicated?). I occasionally run across some method which just seems horribly complex. Reading it closer, it usually turns out that what happened is that two or three independent operations got collected into one subroutine. Perhaps they started out as doing almost the same...

For a long time, it was considered fairly obvious, I think, that syntax didn’t really matter. It was just the surface skin over the underlying ideas. In recent times, though, the prevailing wisdom has reversed, and it is now quite common to hear people talk about how “syntax matters”. While I don’t exactly disagree, I think that the importance...

We’ve been discussing a lot about how to manage vectors and strings in Rust. Graydon sent out an excellent proposal which allows for a great number of use cases to be elegant handled. However, I find the syntax somewhat misleading. I’ve proposed one alternative on the mailing list, but I now find I don’t like it, so I thought I’d brainstorm a...

After my recent dalliance in Matters of a Truly Trivial Nature, I’d like to return to Matters Most Deep and Profound. I’m running up against an interesting question with regions that has to do with the nature of function types like fn(&int): up until now, I’ve assumed that this refers to a function that takes an integer pointer in some region...

I want to do an introduction to the regions system I’ve been working on. This is work-in-progress, so some of the details are likely to change. Also, I’m going to try some new terminology on for size: although it has a long history in the literature, I think the term “region” is not particularly accurate, so I am going to use the term...

I’ve been making a point of reading academic papers on the train as I ride home. It’s so easy to get behind with the sheer quantity of work that is being produced. Anyway, it occurred to me that I ought to try and summarize the papers I read on this blog so that I can I remember my reactions to them. I’ll start with “Permission Regions for...

I’m still thinking about vector and string types in Rust and I think I’ve decided what I feel is the best approach. I thought I’d summarize it here and make the case for it. If you don’t know what I’m talking about, see this post for more background. I’ll forward this to the mailing list as well; I’m sorry if it seems like I’m harping on this...

I’ve been working for the last few days on the proper safety conditions for borrowing. I am coming into a situation where I am not sure what would be the best approach. The question boils down to how coarse-grained and approximate our algorithm ought to be: in particular, ought it to be flow sensitive? But let me back up a bit, first, and...

I implemented a simple, non-flow-sensitive version of the reference checker which I described in my previous post. Of course it does not accept the Rust codebase; however, the lack of flow-sensitivity is not the problem, but rather our extensive use of unique vectors. I thought I’d write a post first showing the problem that you run into and...

As you loyal readers know, I am on a quest to make the Rust type system more orthogonal with respect to the kind of pointer in use, by which I mean that I want to have the three pointer sigils (@, &, and ~) indicate where memory is located and the other types indicate what value is to be found at that memory. Right now there are a few cases...

Yesterday I wrote about my scheme for paring down our set of function types to one type, fn:kind(S) -> T. When I finished writing the post, I was feeling somewhat uncertain about the merits of the idea, but I’m feeling somewhat better about it today. I really like the idea that top-level items have the type fn:kind(S) -> T and that you...

I wanted to bring together the various ideas around vectors and function types into one post. The goals of these changes are to achieve orthogonality of the pointer types, so that leading &, @, and ~ sigils are the only way to indicate the kind of pointer that is in use; to help pare down on the proliferation of subtle variantions on types, such...

I am dissatisfied with how mutability is treated in the Rust type system. The current system is that a type is not prefixed mutable; rather, lvalues are. That is, a type T is defined like so: T = [M T] | @ M T | & M T | { (M f : T)* } | int | uint | ... M = mut | const | (imm) Note that there is no type mut int (a mutable integer). ...

Currently, Rust has an effect system but refuses to admit it. In an effort to broaden the set of things that can be safely done in the face of extant aliases into the heap, I have been experimenting with a lightweight extension to Rust’s system. So far I think it is promising but also no magic bullet. Background For those who aren’t familiar...

I have been thinking a bit more about the approach to mutability I recently discussed. It seemed a bit too good to be true (too clean) and I think I’ve realized a problem. The problem derives from my definition of types: T = Q U Q = mut | const | imm U = [T] | @T | &T | { (f : T)* } | X | int | uint | ... The interesting case is...

OK, I’ve been thinking more about the mutability issue and I think I have found a formulation that I am happy with. The basic idea is that we refactor types like so: T = M T | X | @T | ~T | [T] | {(f:T)*} | int | uint | ... M = mut | const | imm This no doubt looks similar to some of my other permutations. The key difference is...

This is a proposal to unify the mechanics of alt and destructuring assignment. It was born out of discussion between erickt, pcwalton, and I amidst various bugs in the bug tracker but I wanted to float it around to a larger audience. I’d like to discuss this on Tuesday, because one of the logical next steps for the regions work is to begin...

Last Thursday and Friday I had the good fortune of presenting a paper of mine at HotPar 2012. The paper is called Parallel Closures: A New Twist on an Old Idea; it basically describes what has evolved to become the PJs (Parallel JavaScript) model, though it does so in the context of a static checker built in Java. I really enjoyed the workshop:...

This is a draft of (the first section of) a new Rust tutorial on borrowed pointers (the official name for “regions”). Comments welcome. UPDATE: I added a section “Why borrowed?” Borrowed pointers Borrowed pointers are one of the more flexible and powerful tools available in Rust. A borrowed pointer can be used to point anywhere: into the shared...

I am trying to mop up some of the remaining work for regions now. One of the big remaining areas is dealing with function and iface types. This proposal is certainly influenced by my previous proposals. However, we have backed away from the idea of dynamically-sized types for vectors and so I will do the same here. The design My current design...

I had a very interesting discussion with Sriram and Terrence (of Kilim and ANTLR fame, respectively—two smart dudes) yesterday. One of the things we talked about was adapting shared-memory data structures like concurrent hash maps into an actor setting. One thing we’ve found when working on Servo is that the temptation to cheat is enormous. Most...

Here is a (much) more complete draft of the tutorial on borrowed pointers. It is becoming more in-depth than I intended. I hope to later extract a much shorter subset. But I thought I’d post what I’ve got so far. Borrowed pointers Borrowed pointers are one of the more flexible and powerful tools available in Rust. A borrowed pointer can be used...

One thing I didn’t make clear regarding my last post: I am not especially satisfied with the way the “tutorial” was turning out. I use scare quotes here because I think it resembles a reference manual more than a tutorial. Nonetheless I think there are some sections that are quite good; and a document like it probably ought to exist. So I...

Here is my latest stab at a tutorial on borrowed pointers. I know, I know, enough with the borrowed pointer tutorials already! Hopefully this will be my last post in this vein for a while. I am much happier with this version. It is still too long to serve as a chapter in the general Rust tutorial, but I think it’s more approachable than the...

I have been working on a change to the definition of mutability in Rust. This is a much smaller change than my previous thought experiments, which were aimed at achieving better parameterization (those are still percolating; I think the best approach is a modified version of the latest proposal where not all types have mutability but type...

I’ve been slowly learning how type inference works in SpiderMonkey. As I understand it, SpiderMonkey’s type inference scheme is the brain child of one Brian “Hack-it”, coder extraordinaire. You may have seen a recent PLDI publication on the topic. You may, like me, have read that publication. You may, also like me, have walked away thinking,...

One of the things that is sometimes frustrating in Rust is the inability to define a type that indicates some subset of enum variants. For example, it is very common to have a pattern like this: match an_expr { expr_call(*) => process_call(an_expr) ... } fn process_call(a_call_expr: @ast::expr) { ... } But as you can see, the type of...

We need to clarify our story on rvalue lifetimes. This is related to issue #3387 and also various recent and not-so-recent discussions on IRC. The basic question is how long an rvalue lives when the program creates pointers into it. To understand the rough issues, first consider this program: let x = foo(); match x { Some(ref y) =>...

Well, I have not done too well with my goal of reading a research paper a day on the train (actually my initial goal was two papers, but seeing as how I’ve failed so spectacularly, I’ve dialed it back some). However, I’ve decided to give it another go. I’ve bought a printer now so I can print papers out (double-sided, no less!) at home (I had...

I have been trying to come up with a reasonable set of rules for deciding when a pattern binding ought to be a move and when it ought to be a copy and utterly failing. Simultaneously, pcwalton, brson, and I kind of simultaneously arrived at an alternate design that tries to simplify the copy/move distinction. I think that it also solves the...

Currently, the Rust compiler accepts all manner of trait, impl, and bound declarations. In fact, it accepts plenty of declarations that later phases of the compiler are not sophisticated enough to handle. In other words, the syntax is writing checks the semantics can’t cash. (An aside: I just love saying that phrase for some perverse reason. I...

I was thinking more about type classes as I walked down the street. In my prior post I wrote that the rules I proposed resulted in a system where traits loosely fit the following Haskell template: class C self a ... z | self -> a ... z where ... However, I gave two caveats. The first was that due to subtyping we cannot say that one type...

So, the condition that was supposed to ensure termination in my previous post is most certainly wrong. The idea was to prevent tautological impls like the following: impl A: Foo { ... } Such impls, given a naive algorithm, would loop infinitely trying to decide if a type T implemented Foo. You can imagine: it would ask, “does T implement Foo?...

I have started work on implementing Rivertrail, Intel’s proposal for data parallelism in JS. I am excited about this project, it seems like it’s going to be great fun. The initial version that we produce is going to be focused on Intel’s specification, but I hope we can eventually combine it with the more general stuff I’ve been doing as part...

In this post I propose an extension of Rust’s purity rules. The short version is that pure functions would be allowed to mutate data owned by their &mut parameters. This extends the current Rust purity rules which allow pure functions to invoke impure closures so long as they are an argument to the function. The principle is the same: pure...

My big goal for 0.5 is to straighten out our function types (yet again). I’ve been tossing the design for these over in my head since the summer and I wanted to lay out my plan. This is a variation of something that Ben Blum and I sketched out on a whiteboard. Closure type The closure type will be described something like so. Beware, it’s got...

I am considering whether we should add a way to borrow something but retain uniqueness. This would address a shortcoming of the borrowing system that has been bothering me for some time, and it would enable a few patterns that are difficult or awkward today. The Problem I described the problem in this paper review I wrote, but I will repeat it...

I can’t believe I’m saying this, but I’ve started to think that Parallel JS (nee Rivertrail) should not demand pure callbacks to functions like map() and so forth. Rather it should just accept arbitrary functions. Previously, I thought that it was important that ParallelArray methods should only accept functions which, at least in a perfect...

I’ve been thinking of a radical change we could make to the treatment of mutability and borrowed pointers in Rust. The goal is to eliminate all of the error messages about “aliasable, mutable data” that the borrow checker currently issues. The idea is somewhat inspired by writing a recent paper on Rust’s current system—writing a paper on...

The blog has been silent for a while. The reason is that I’ve been hard at work on Parallel JS. It’s come a long way: in fact, the goal is to land an initial version in the next couple weeks! One of the very exciting developments has been that we switched to a self-hosting implementation. Self-hosting is a very cool new direction for the...

I mentioned in my previous post that we are using a single primitive parallel operation to implement PJs. It turns out that I am not very satisfied with what we currently have and I have been thinking about some alternatives. In this post I’ll describe briefly how things are setup, what problems I see, and then sketch out how I think we could...

I’ve been thinking for a while that our lifetime notation has too many defaults which can be more confusing than helpful. A recent spate of e-mails on rust-dev brought this back to my mind. I’ve been wanting to take a look at these defaults for a while, so I thought I’d write up a quick exploration of the “syntactic space”. A warning: this is...

One of the interesting questions with respect to Parallel JS is what the semantics ought to be if you attempt a parallel operation with a kernel function that has side-effects. There are basically three reasonable options: Deterministic results where possible: The function behaves “as if” it executed sequentially, executing the kernel from 0 to...

In my last post, I made the case against having a deterministic semantics. I’ve gotten a fair amount of feedback saying that, for a Web API, introducing nondeterminism is a very risky idea. Certainly the arguments are strong. Therefore, I want to take a moment and make the case for determinism. Why determinism? All things being equal, it’s...

In a previous post I outlined some of the options for updating our lifetime syntax. I want to revist those examples after having given the matter more thought, and also after some discussions in the comments and on IRC. My newest proposal is that we use <> to designate lifetime parameters on types and we lean on semantic analysis (the resolve...

In Rust today, there is a special for syntax designed to support interruptible loops. Since we introduced it, this has proven to be a remarkable success. However, I think we can improve it very slightly. Current for protocol The current “for protocol” is best explained by giving an example of how to implement it for slices: fn each(v: &[E], f:...

Rust features destructors and, as of this moment, they are simply not sound with respect to many other features of the language, such as borrowed and managed pointers. The problem is that destructors are granted unlimited access to arbitrary data, but the type system and runtime do not take that into account. I propose to fix this by limiting...

One of the things that I’ve been working on for some time now is the proper integration of C functions. As with virtually every other facet of the design of Rust, we’ve been slowly moving from a model where Rust tried to hide low-level details for you to one where Rust offers tight control over what’s going on, with the type system intervening...

Lately, I’ve been thinking about the ParallelJS API that we want to expose. In particular, I’ve been considering offering methods on the normal array type for basic parallel operations. I think this opens up some interesting doors. Note: To give credit where credit is due, I should note that a lot of the ideas in this post originate with other...

The first version of our work on ParallelJS has just been promoted to mozilla-central and thus will soon be appearing in a Nightly Firefox build near you. I find this pretty exciting. In honor of the occassion, I wanted to take a moment to step back and look both at what has landed now, what we expect to land soon, and the overall trajectory we...

I am going to write a series of blog posts giving a tour of the current Parallel JS implementation in SpiderMonkey. These posts are intended to serve partly as documentation for the code. The plan is to begin high level and work my way down to the nitty gritty details, so here we go! I will start my discussion at the level of the intrinsic...