profile picture

Jasper rough cut: September 2026

October 28, 2026 - jasper crystal compilers ai rough-cut

I want to write the Jasper compiler in Jasper. In September I made a list of what Jasper could not do yet, and 42 merged branches put Jasper closer to that objective. Plus, release builds with LTO now make programs that run in 45% less time. It only took $1,398 and about 2 hours a day of my time.

Notes from building Jasper, one month at a time

New here? The earlier posts explain what Jasper is and how I got here:

This is a rough cut and, like before, some of these decisions may change.

Rough cut: September 2026

August was quiet, and it gave me time to plan what comes next.

The current compiler (Green Jasper) is written in Crystal. The compiler written in Jasper will be Red Jasper. To find what Red Jasper needs, I searched the code in src/. For example, about 490 lines have array or hash literals, and about 200 lines are accessors.

From those counts I built a list of priorities. Some items became work right away, and others needed research first.

As I shared in the August update, the goal is no longer parity with Crystal. Jasper needs enough to write the compiler, and then to build applications and maintain the compiler in the same language.

It was the busiest month so far, with 459 commits.

Blocks: the signature decides

In August, blocks were only inline. The compiler pasted the block body at each yield. This month added break and next, and blocks that you can store and run later.

In Crystal, a method captures its block when the body uses the block parameter (Eg. stores it or calls block.call). To know how a method uses its block, you must read the body of that method. I wanted to avoid that.

In Jasper, the signature decides:

A method with a named block that uses yield does not compile. The signature also tells the caller if break works. It works in an inline block, and not in a stored one.

The use case I had in mind is a spec runner. Jasper does not have one yet, but stored blocks are one of the pieces it needs. it must keep each block and run it later:

class Runner
  @examples : Array(Proc(Void)) = [] of Proc(Void)

  def it(name : String, &block : -> Void) : Void
    @examples << block
  end

  def run : Void
    @examples.each { |example| example.call }
  end
end

runner = Runner.new
runner.it("adds") do
  LibC.printf("1 + 1 = %d\n", 1 + 1)
end
runner.run

A Proc is a full value. You can keep it in a local, an instance variable, an array or a nilable. Outside of the & in a method's parameter list, you write its type as Proc(Void) or Proc(Int32, Bool). The short arrow form (-> Void) only works after &.

Oh, about Void: in Jasper, Void and Nil are not the same. I covered this briefly in a Crystal forum post, but not in this series.

Void means there is no value. Nil is a real value, and a method that returns Nil must return it. The compiler refuses three things:

def done : Void
end

done_value = done
error: expression has no value
  done_value = done
               ^^^^

Blocks follow the same rule. A -> Void block gives no value, so the it blocks in the runner do not need to end in nil. A -> Nil block must end in nil.

Note: because of this difference, the runner above does not compile in Crystal. Crystal turns Proc(Void) into Proc(Nil), and then refuses the instance variable (must be Array(Proc(Nil)), not Array(Proc(Nil))). It is one of the few places where Jasper accepts code that Crystal rejects, and the crosscheck lists it as a known deviation.

What a stored block can see

There are no closures yet. A stored block can read its own parameters and locals, and top-level methods. It cannot read the locals around it, or self:

expected = 2
runner.it("adds") do
  x = 1 + 1
  x == expected
end
error: stored block reads local `expected`, stored blocks cannot see the locals around them
    x == expected
         ^^^^^^^^
  note: captured.jasper:4:25: block stored here
      def it(name : String, &block : -> Void) : Void
                            ^^^^^^^^^^^^^^^^

Closures need an environment that lives with the block, with ARC rules for each captured value. That is the next stage, and I will build it when a real program needs it.

The rules from August still apply. A block argument cannot shadow a local, and a subclass cannot override a method that takes an inline block.

The rules for jumps are strict too. break and next now work in inline blocks. The compiler still refuses return inside a block, because a break with a local gives the same result and is easier to follow.

Release builds and LTO

Since I now run more code in Jasper, I started to look at some performance numbers. Don't worry, I am not getting obsessed with it. These are only the first benchmarks, but they showed a few gaps.

Until now, Jasper had one kind of build. Each unit (the per-method units from July) compiles with myc-qbe, and cc links them. It is quick, but no optimizer looks across units.

A spike compared five builds. On binary_trees, the default build ran in 2.23s. The same program with myc-llvm and link-time optimization (LTO) ran in 1.23s, 45% less time.

Jasper now has --release, like Crystal.

How Crystal does it

Crystal's --release means -O3 --single-module: it puts the whole program in a single LLVM module. The optimizer sees everything, but the cache works per module, and there is only one. So any change, even a one-line edit, compiles the whole program again.

How Jasper does it

Jasper keeps the units. In a release build, myc-llvm writes each unit as LLVM bitcode, and the linker (lld) optimizes across all of them. The cache still works per unit, so a one-line edit rebuilds one unit.

The spike measured both on hash_keys (119 units), after a one-line edit:

Both run about 29% faster than per-unit builds without LTO.

Note: the single module in this spike is the Crystal approach applied to Jasper code, not Crystal itself. On binary_trees, both builds ran almost the same (1.25s for one module, 1.23s for LTO). But these are small programs. I do not know yet if, on a larger program, LLVM optimizes bitcode units at link time as well as one big module. I will run another benchmark when Jasper has bigger programs to measure.

A patch to myc

myc could not write bitcode objects. Also, without --final, myc marks every function noinline, so LTO had nothing to gain.

I first used a patched version of myc. Since kostya merged the LTO patch upstream, Jasper builds myc from upstream again.

This is the second time myc saved me a lot of work. Switching to LLVM for the release build took two flags (--final and --llvm-bitcode-obj), and the program ran in 45% less time.

Two release modes

But LTO is not free. The linker optimizes the whole program on every build, with or without a cache.

The bigger the program, the slower it gets. In a benchmark on Linux, the link took 99ms at 123 units and 6.3 seconds at 1,203 units.

At 1,203 units, in milliseconds (binary size in KB):

BuildRebuild, no changeRebuild, one editRun timeBinary
Default (myc-qbe)1331363,924232
--release-fast1291433,002226
--release (LTO)6,3885,867535112
ThinLTO with cache (not used)16712,373536331

ThinLTO is made for incremental builds, and its cache makes a rebuild with no changes cheap. But after an edit to one file, the rebuild took twice as long as with full LTO, and the binary is bigger than the unoptimized one.

I think a fast edit loop (watch, save and rebuild) and a fast final build are different enough to need two modes. --release gives the fastest program, and --release-fast keeps the link cheap for when you rebuild often.

These are early optimizations. Nothing about the approach is final yet.

Note: Jasper may not use all of what myc offers yet. myc can inline a function from another unit, but only when it has that function's body. Jasper compiles each unit alone, so myc never sees the other bodies, and every call to another unit stays a real call. myc has a --header option that collects those bodies into one header, so each unit can inline across units without LTO. This could give faster programs in --release-fast and in the default build, but I have not figured out all the details to make it work yet.

Everyday syntax: types everywhere

Jasper is strict about types. I stated that previously, and now the rule also covers literals, instance variables and constants.

Array and hash literals need of, at every level:

numbers = [1, 2, 3] of Int32
counts  = {"a" => 1, "b" => 2} of String => Int32
rows    = [[1, 2] of Int32, [3] of Int32] of Array(Int32)

If you try to compile without types, you get a compiler error:

error: a literal needs a type: write `of Int32`
  numbers = [1, 2, 3]
                     ^

Instance variables must also have type declarations:

class Token
  getter kind : String
  getter start : Int32
  property? skipped : Bool = false
  @count : Int32 = 0

  def initialize(@kind : String, @start : Int32)
  end
end

Without the type, the build fails:

error: `getter` needs a type: `getter count : T`
  getter count = 0
         ^^^^^

A breaking change: Jasper used to create the instance variable the first time initialize assigned it. Now you need to declare it in the class body, or as a parameter of initialize (Eg. @kind : String).

Constants also require a type:

ERR : Sink::Fd = Sink::Fd.stderr

This was recently introduced in Crystal (PR #16951).

What else arrived

Refused: string interpolation

Jasper compiled "x=#{x}" and printed x=#{x}, while Crystal prints x=42. Real interpolation needs IO and to_s, and I have not decided how those should work yet.

So for now, #{ is a compile error:

error: string interpolation is not supported yet, write `\#{` for a literal `#{`
  LibC.printf("x=#{x}\n")
                 ^^

The compiler code changes too

Red Jasper must be able to read the current compiler code, so that code must only use syntax that Jasper supports.

One example is case. In Crystal, a short when branch often fits on one line with then. Jasper has no then keyword (and no ;), so it cannot read this code from src/:

case type
when VoidType then "void"
when NilType  then "nil"
else               type_code(type)
end

Every branch in src/ now puts its body on the next line:

case type
when VoidType
  "void"
when NilType
  "nil"
else
  type_code(type)
end

This helped keep the compiler simpler. There is one less optional form to maintain, or to learn when you read the code.

No macros: expanders

Crystal gets getter and property from macros. Jasper has no macros, and I do not plan to add them, as I don't want to implement an interpreter of Jasper along with the compiler.

So how will Jasper generate code at compile time?

I have not decided what it will look like yet. I only did some research, and compared a few approaches, including two kinds of macros (simple templates and Crystal's approach). The research recommends two steps:

  1. Now, add one expansion pass to the compiler, with the expanders built in.
  2. Later, add expander programs. An expander is an ordinary Jasper program. The compiler builds it once, sends it each call it must expand, and reads back the Jasper code it generates.

Crystal already does something similar with the run macro. It builds a helper program, caches it and reads its output.

The difference is how the cache notices a change. Crystal compares the file timestamps of the helper and the files it requires. Jasper's cache uses a hash of each unit's content, so only a real change to the expander would rebuild it.

I have not settled on the DX (developer experience) yet, so for now only four accessors (getter, getter?, property and property?) are built into the compiler (the ones the compiler uses most, getter! comes later).

The same work added an abstract Object class, now the root above Reference and Value. The compiler builds it in, and copies its methods into each class that calls them.

Linting the compiler code

In August I added Ameba to lint the compiler code. I use its default rules, with no custom settings. The only configuration says which folders to check.

The linter has two jobs:

In September I started to address the six whole files that were excluded from the complexity rule. Ameba never checked new methods in those files, so the agents kept adding complex code there. In two weeks, the methods over the limit grew from 15 to 20.

The fix moved the baseline from files to methods. Each old method now has its own disable comment, and Ameba checks new code in the same file like code anywhere else. New code never joins the baseline, so the list can only get shorter.

This is now part of the lint task (mise lint), which is one of the gates a change must pass before the agent can merge it.

Then I started to clear the list. The token switches in the lexer and parser were the first. scan_token in the lexer scored 49, and parse_prefix scored 24. The fix moved the facts about each token into config/ast.yml (the same generator from July) and brought every method under 12. Now 15 methods are left.

There is no bug-free software

Many of this month's bugs gave a wrong answer instead of a crash:

Some bugs were in myc, not in Jasper:

I reported one myc bug this month, a hang with recursion across files, and kostya fixed it the same day. The three above are not reported yet. The agents found them and wrote down what they saw, but before I send them to kostya, I want a small reproduction case for each one, checked by hand. A ticket written by an LLM and not checked by a person only costs the maintainer time.

I logged a few questions but have not decided them yet:

Steering the agents

While I wrote this post, I asked myself two questions. Am I micromanaging the agents? And do they build things before the research is done?

To check, I reviewed my sessions, the commits and the notes the agents keep when I correct them.

The short answer is that I stay close to the design, and away from the implementation.

Most of my messages go to brainstorming and spec sessions. About a quarter of them are short answers like "option A" or "yes". The brainstorming skill asks one question at a time, so each short answer is a design decision. When a session runs a plan, I send a few messages, and the agents dispatch 13 to 36 subagents. In the implementation sessions, only 4 of my 19 messages corrected something.

The pushback happens in design. For example, when a design created a missing parent namespace on its own:

Jasper should not create the missing parent. Explicit over implicit/automatic things that can lead to confusions.

The agents did not build without a spec either. Every implementation session started from a plan, and every plan came from a spec.

When the agent merged too early

In one session, an agent merged two pieces of work to main before I confirmed them. One was the spike for the myc --header option. The other was the reproduction case for the myc recursion hang. The agent merged them three times, and I asked it to revert them and "keep it in a branch until validated and confirmed".

The agent did what AGENTS.md tells it to do: when the three gates pass, it merges the branch with --no-ff. The project settings also allow git merge without asking me first.

The problem was in my instructions. They did not say that a spike is different from a regular feature. A spike is something we explore, and its result may never reach main.

A clearer instruction is not enough to fix that, because an agent can ignore it. The setup of the project needs a guardrail that checks the merge itself. For example, a git hook that refuses to merge a spike branch into main until I approve it.

Stack and numbers

Stack

Code

Lines include comments and blank lines.

Note: commits are counted by author date (when the work happened). Commits are not a useful metric on their own, but some folks asked me how big this thing is getting (don't worry, it is not going to be 1 million lines of code).

Tokens & cost