> No metaprogramming: send, method_missing, define_method (dynamic)
> No threads: Thread, Mutex (Fiber is supported)
Speaking as someone who has written a lot of Ruby code over the years, utilizing every single one of these features of Ruby, I have to say this is the version of Ruby I've evolved to want: simpler and easier to understand but with the aesthetic beauty of Ruby intact.
IMO this more limited variant of Ruby is more practical now that we have extremely productive code generation tools in the form of LLMs. A lot of meta-programming ostensibly exists to make developers more productive by reducing the amount of boilerplate code that has to be written, but that seems no longer necessary now that developers aren't writing code.
I think some of the limitations can still be implemented (definitely Threads and Mutex), and I'd prefer it to compile to LLVM-IR or something, not C, but overall I think it is great to see Matz playing around with AOT compiling.
This is a problem I see with many ruby projects. How would I reword this?
Well, first thing, after stating what spinel is, I would show a simple example. Ideally a standalone .rb file or something like that, that can be downloaded (or whatever other format). Yes, the README shows this, but believe it or not, I have realised that I am usually below average when trying to understand something that is now. I even manage to make copy/paste mistakes. This is why I think one or two standalone as-is examples would be best.
And then I would explain use cases.
The current structure of the document is strange. Has that been written with AI? If AI replaces the human individual, why is it then expected that real people should read that? So many questions here ...
Also, I would really like for the ruby ecosystem to not be split up into different entities. I understand that mruby does not have the same goals as MRI ruby, but still, there is fragmentation. Now there is spinel - how does it relate to other parts of ruby? Why are truffleruby and jruby separate? (I know why, so I am not objecting to the rationale; I am pointing out that for a USER it would be better if things would be more unified here in general.)
Ruby really needs to focus on its inner core. The base should be solid. Even more so when it is harder to attract genuinely new developers.
The lack of eval/meta-programming fallbacks is a shame though, but I guess they kept the focus on a small, performant subset.
It would be nice to have gems compiled with this AOT compiler that can interact well with MRI.
When it comes to packaging/bundling more standard ruby (including gems) we'll still need tebako, kompo, ocran – and then there's a bunch of older projects that did similar things too like ruby-packer, traveling ruby, jruby warbler etc.
It's nice to have another option, but still, I'm hoping for a more definitive solution with better developer UX.
- No eval: eval, instance_eval, class_eval
- No metaprogramming: send, method_missing, define_method (dynamic)
- No threads: Thread, Mutex (Fiber is supported)
- No encoding: assumes UTF-8/ASCII
- No general lambda calculus: deeply nested -> x { } with [] calls
Assuming UTF-8/ASCII isn’t, IMO, a huge limitation, but some of the others likely are, for quite a few programs. Removing them also will require serious work, I think.
Compilers code was never pretty, but even by those standard, I feel like it is a very-very hard to maintain code by humans.
It’s named after his new cat, which is named after a cat in Card Captor Sakura, which is the partner to another character named Ruby.
My thesis work (back when EcmaScript 5 was new) was an AOT JS compiler, it worked but there was limitations with regards to input data that made me abandon it after that since JS developers overall didn't seem to aware of how to restrict oneself properly (JSON.parse is inherently unknown, today with TypeScript it's probably more feasible).
The limitations are clear also, the general lambda calculus points to limits in the type-inference system (there's plenty of good papers from f.ex. Matt Might on the subject) as well as the Shed-skin Python people.
eval, send, method_missing, define_method , as a non-rubyist how common are these in real-world code? And how is untyped parsing done (ie JSON ingestion?).
Spinel compiles Ruby source code into standalone native executables. It performs whole-program type inference and generates optimized C code, achieving significant speedups over CRuby.
Spinel is self-hosting: the compiler backend is written in Ruby and compiles itself into a native binary.
Ruby (.rb)
|
v
spinel_parse Parse with Prism (libprism), serialize AST
| (C binary, or CRuby + Prism gem as fallback)
v
AST text file
|
v
spinel_codegen Type inference + C code generation
| (self-hosted native binary)
v
C source (.c)
|
v
cc -O2 -Ilib -lm Standard C compiler + runtime header
|
v
Native binary Standalone, no runtime dependencies
# Fetch libprism sources (from the prism gem on rubygems.org):
make deps
# Build everything:
make
# Write a Ruby program:
cat > hello.rb <<'RUBY'
def fib(n)
if n < 2
n
else
fib(n - 1) + fib(n - 2)
end
end
puts fib(34)
RUBY
# Compile and run:
./spinel hello.rb
./hello # prints 5702887 (instantly)
./spinel app.rb # compiles to ./app
./spinel app.rb -o myapp # compiles to ./myapp
./spinel app.rb -c # generates app.c only
./spinel app.rb -S # prints C to stdout
Spinel compiles its own backend. The bootstrap chain:
CRuby + spinel_parse.rb → AST
CRuby + spinel_codegen.rb → gen1.c → bin1
bin1 + AST → gen2.c → bin2
bin2 + AST → gen3.c
gen2.c == gen3.c (bootstrap loop closed)
74 tests pass. 55 benchmarks pass.
Geometric mean: ~11.6x faster than miniruby (Ruby 4.1.0dev) across
the 28 benchmarks below. Baseline is the latest CRuby miniruby build
(without bundled gems), which is considerably faster than the system
ruby (3.2.3); Spinel's advantage is correspondingly smaller but still
substantial on computation-heavy workloads.
| Benchmark | Spinel | miniruby | Speedup |
|---|---|---|---|
| life (Conway's GoL) | 20 ms | 1,733 ms | 86.7x |
| ackermann | 5 ms | 374 ms | 74.8x |
| mandelbrot | 25 ms | 1,453 ms | 58.1x |
| fib (recursive) | 17 ms | 581 ms | 34.2x |
| nqueens | 10 ms | 304 ms | 30.4x |
| tarai | 16 ms | 461 ms | 28.8x |
| tak | 22 ms | 532 ms | 24.2x |
| matmul | 13 ms | 313 ms | 24.1x |
| sudoku | 6 ms | 102 ms | 17.0x |
| partial_sums | 93 ms | 1,498 ms | 16.1x |
| fannkuch | 2 ms | 19 ms | 9.5x |
| sieve | 39 ms | 332 ms | 8.5x |
| fasta (DNA seq gen) | 3 ms | 21 ms | 7.0x |
| Benchmark | Spinel | miniruby | Speedup |
|---|---|---|---|
| rbtree (red-black tree) | 24 ms | 543 ms | 22.6x |
| splay tree | 14 ms | 195 ms | 13.9x |
| huffman (encoding) | 6 ms | 59 ms | 9.8x |
| so_lists | 76 ms | 410 ms | 5.4x |
| binary_trees | 11 ms | 40 ms | 3.6x |
| linked_list | 136 ms | 388 ms | 2.9x |
| gcbench | 1,845 ms | 3,641 ms | 2.0x |
| Benchmark | Spinel | miniruby | Speedup |
|---|---|---|---|
| json_parse | 39 ms | 394 ms | 10.1x |
| bigint_fib (1000 digits) | 2 ms | 16 ms | 8.0x |
| ao_render (ray tracer) | 417 ms | 3,334 ms | 8.0x |
| pidigits (bigint) | 2 ms | 13 ms | 6.5x |
| str_concat | 2 ms | 13 ms | 6.5x |
| template engine | 152 ms | 936 ms | 6.2x |
| csv_process | 234 ms | 860 ms | 3.7x |
| io_wordcount | 33 ms | 97 ms | 2.9x |
Core: Classes, inheritance, super, include (mixin), attr_accessor,
Struct.new, alias, module constants, open classes for built-in types.
Control Flow: if/elsif/else, unless, case/when,
case/in (pattern matching), while, until, loop, for..in
(range and array), break, next, return, catch/throw,
&. (safe navigation).
Blocks: yield, block_given?, &block, proc {}, Proc.new,
lambda -> x { }, method(:name). Block methods: each,
each_with_index, map, select, reject, reduce, sort_by,
any?, all?, none?, times, upto, downto.
Exceptions: begin/rescue/ensure/retry, raise,
custom exception classes.
Types: Integer, Float, String (immutable + mutable), Array, Hash,
Range, Time, StringIO, File, Regexp, Bigint (auto-promoted), Fiber.
Polymorphic values via tagged unions. Nullable object types (T?)
for self-referential data structures (linked lists, trees).
Global Variables: $name compiled to static C variables with
type-mismatch detection at compile time.
Strings: << automatically promotes to mutable strings (sp_String)
for O(n) in-place append. +, interpolation, tr, ljust/rjust/center,
and all standard methods work on both. Character comparisons like
s[i] == "c" are optimized to direct char array access (zero allocation).
Chained concatenation (a + b + c + d) collapses to a single malloc
via sp_str_concat4 / sp_str_concat_arr -- N-1 fewer allocations.
Loop-local str.split(sep) reuses the same sp_StrArray across
iterations (csv_process: 4 M allocations eliminated).
Regexp: Built-in NFA regexp engine (no external dependency).
=~, $1-$9, match?, gsub(/re/, str), sub(/re/, str),
scan(/re/), split(/re/).
Bigint: Arbitrary precision integers via mruby-bigint. Auto-promoted
from loop multiplication patterns (e.g. q = q * k). Linked as static
library -- only included when used.
Fiber: Cooperative concurrency via ucontext_t. Fiber.new,
Fiber#resume, Fiber.yield with value passing. Captures free
variables via heap-promoted cells.
Memory: Mark-and-sweep GC with size-segregated free lists, non-recursive marking, and sticky mark bits. Small classes (≤8 scalar fields, no inheritance, no mutation through parameters) are automatically stack-allocated as value types -- 1M allocations of a 5-field class drop from 85 ms to 2 ms. Programs using only value types emit no GC runtime at all.
Symbols: Separate sp_sym type, distinct from strings (:a != "a").
Symbol literals are interned at compile time (SPS_name constants);
String#to_sym uses a dynamic pool only when needed. Symbol-keyed
hashes ({a: 1}) use a dedicated sp_SymIntHash that stores
sp_sym (integer) keys directly rather than strings -- no strcmp, no
dynamic string allocation.
I/O: puts, print, printf, p, gets, ARGV, ENV[],
File.read/write/open (with blocks), system(), backtick.
Whole-program type inference drives several compile-time optimizations:
N = 100) are
inlined at use sites instead of going through cst_N runtime lookup.while i < arr.length evaluates
arr.length once before the loop; while i < str.length hoists
strlen. Mutation of the receiver inside the body (e.g. arr.push)
correctly disables the hoist.static inline so gcc can inline them at call sites.a + b + c + d compiles to a
single sp_str_concat4 / sp_str_concat_arr call -- one malloc
instead of N-1 intermediate strings.x = x * y or fibonacci-style
c = a + b self-referential addition auto-promote to bigint.to_s: divide-and-conquer O(n log²n) via mruby-bigint's
mpz_get_str instead of naive O(n²)."literal".to_sym resolves to a
compile-time SPS_<name> constant; the runtime dynamic pool is
only emitted when dynamic interning is actually used.strlen caching in sub_range: when a string's length is
hoisted, str[i] accesses use sp_str_sub_range_len to skip the
internal strlen call.fields = line.split(",") inside a loop reuses
the existing sp_StrArray rather than allocating a new one.-ffunction-sections -fdata-sections and linked with --gc-sections; each unused
runtime function is stripped from the final binary.parse_id_list byte walk: the AST-field list parser (called
~120 K times during self-compile) walks bytes manually via
s.bytes[i] instead of s.split(","), dropping N+1 allocations
per call to 2.-Werror so regressions surface immediately.spinel One-command wrapper script (POSIX shell)
spinel_parse.c C frontend: libprism → text AST (1,061 lines)
spinel_codegen.rb Compiler backend: AST → C code (21,109 lines)
lib/sp_runtime.h Runtime library header (581 lines)
lib/sp_bigint.c Arbitrary precision integers (5,394 lines)
lib/regexp/ Built-in regexp engine (1,759 lines)
test/ 74 feature tests
benchmark/ 55 benchmarks
Makefile Build automation
The compiler backend (spinel_codegen.rb) is written in a Ruby subset
that Spinel itself can compile: classes, def, attr_accessor,
if/case/while, each/map/select, yield, begin/rescue,
String/Array/Hash operations, File I/O.
No metaprogramming, no eval, no require in the backend.
The runtime (lib/sp_runtime.h) contains GC, array/hash/string
implementations, and all runtime support as a single header file.
Generated C includes this header, and the linker pulls only the
needed parts from libspinel_rt.a (bigint + regexp engine).
The parser has two implementations:
Both produce identical AST output. The spinel wrapper prefers the
C binary if available. require_relative is resolved at parse time
by inlining the referenced file.
make deps # fetch libprism into vendor/prism (one-time)
make # build parser + regexp library + bootstrap compiler
make test # run 74 feature tests (requires bootstrap)
make bench # run 55 benchmarks (requires bootstrap)
make bootstrap # rebuild compiler from source
sudo make install # install to /usr/local (spinel in PATH)
make clean # remove build artifacts
Override install prefix: make install PREFIX=$HOME/.local
Prism is the Ruby parser used by
spinel_parse. make deps downloads the prism gem tarball from
rubygems.org and extracts its C sources to vendor/prism. If you
already have the prism gem installed, the build auto-detects it; you
can also point at a custom location with PRISM_DIR=/path/to/prism.
CRuby is needed only for the initial bootstrap. After make, the
entire pipeline runs without Ruby.
eval, instance_eval, class_evalsend, method_missing, define_method (dynamic)Thread, Mutex (Fiber is supported)-> x { } with [] callsSpinel was originally implemented in C (18K lines, branch c-version),
then rewritten in Ruby (branch ruby-v1), and finally rewritten in a
self-hosting Ruby subset (current master).
MIT License. See LICENSE.
The revenge of Rational Unified Process, Enterprise Architect and many other tools.
Instead of UML diagrams it is markdown files.
For Ruby and c development it’s the best llm we got right now, the others lag behind by a lot sadly.
With openAI it’s so bad for my usecase it’s basically unusable.
We talk a lot about AI building programs from soup to nuts. But I think people overlook the more likely scenario. AI will turn 10x programmers into 100x programmers. Or in Matz’s case maybe 100x programmers into 500x programmers.
Quite a lot, that's what allows you to build something like Rails with magic sprinkled all around. I'm not 100% sure, but probably the untyped JSON ingestion example uses those.
Remove that, and you have a very compact and readable language that is less strongly typed than Crystal but less metaprogrammable than official Ruby. So I think it has quite a lot of potential but time will tell.
This depends on the individual writing code. Some use it more than others.
I can only give my use case.
.send() I use a lot. I feel that it is simple to understand - you simply invoke a specific method here. Of course people can just use .method_name() instead (usually without the () in ruby), but sometimes you may autogenerate methods and then need to call something dynamically.
.define_method() I use sometimes, when I batch create methods. For instance I use the HTML colour names, steelblue, darkgreen and so forth, and often I then batch-generate the methods for this, e. g. via the correct RGB code. And similar use cases. But, from about 50 of my main projects in ruby, at best only ... 20 or so use it, whereas about 40 may use .send (or, both a bit lower than that).
eval() I try to avoid; in a few cases I use them or the variants. For instance, in a simple but stupid calculator, I use eval() to calculate the expression (I sanitize it before). It's not ideal but simple. I use instance_eval and class_eval more often, usually for aliases (my brain is bad so I need aliases to remember, and sometimes it helps to think properly about a problem).
method_missing I almost never use anymore. There are a few use cases when it is nice to have, but I found that whenever I would use it, the code became more complex and harder to understand, and I kind of got tired of that. So I try to avoid it. It is not always possible to avoid it, but I try to avoid it when possible.
So, to answer your second question, to me personally I would only think of .send() as very important; the others are sometimes but not that important to me. Real-world code may differ, the rails ecosystem is super-weird to me. They even came up with HashWithIndifferentAccess, and while I understand why they came up with it, it also shows a lack of UNDERSTANDING. This is a really big problem with the rails ecosystem - many rails people really did not or do not know ruby. It is strange.
"untyped parsing" I don't understand why that would ever be a problem. I guess only people whose brain is tied to types think about this as a problem. Types are not a problem to me. I know others disagree but it really is not a problem anywhere. It's interesting to see that some people can only operate when there is a type system in place. Usually in ruby you check for behaviour and capabilities, or, if you are lazy, like me, you use .is_a?() which I also do since it is so simple. I actually often prefer it over .respond_to?() as it is shorter to type. And often the checks I use are simple, e. g. "object, are you a string, hash or array" - that covers perhaps 95% of my use cases already. I would not know why types are needed here or fit in anywhere. They may give additional security (perhaps) but they are not necessary IMO.
This is for a limited subset of Ruby - almost no popular Ruby gems would run under it. It's more like PreScheme [1] (ie. a subset of a language oriented at C compilation).
I don't think these compete in the same niches right now. Full Ruby almost certainly requires a JIT.
It's a very pragmatic design: Uses Prism - parsing Ruby is almost harder than the actual translation - and generates C. Basic Ruby semantics are not all that hard to implement.
On the other extreme, I have a long-languishing, buggy, pure-Ruby AOT compiler for Ruby, and I made things massively harder for myself (on purpose) by insisting on it being written to be self-hosting, and using its own parser. It'll get there one day (maybe...).
But one of the things I learned early on from that is that you can half-ass the first 80% and a lot of Ruby code will run. The "second 80%" are largely in things Matz has omitted from this (and from Mruby), like encodings, and all kinds of fringe features (I wish Ruby would deprecate some of them - there are quite a few things in Ruby I've never, ever seen in the wild).
> eval, send, method_missing, define_method , as a non-rubyist how common are these in real-world code? And how is untyped parsing done (ie JSON ingestion?).
They are pervasive. The limitations are similar to those of mruby, though, which has its uses.
Supporting send, method_missing, and define_method is pretty easy.
Supporting eval() is a massive, massive pain, but with the giant caveat that a huge proportion of eval() use in Ruby can be statically reduced to the block version of instance_eval, which can be AOT compiled relatively easily. E.g. if you can statically determine the string eval() is called with, or can split it up, as a lot of the uses are unnecessary or workaround for relatively simple introspection that you can statically check for and handle. For my own compiler, if/when I get to a point where that is a blocking issue, that's my intended first step.
Matz: gem env|info and find should do