Disclosures

🧠 This post is 100% human-written. Claude was used for feedback and to assist with the linker symbol diagram. Cursor was used for feedback and to ensure examples were compilable.

The author of this post is deeply interested in the topic of life-before-main: he is the author of the ctor crate, and the creator of the linktime project that we’ll be using in the examples below.

Every Rust binary has one thing in common: fn main(). If you come from the C world, that might be more familiar as int main(argc, argv). Some platforms might obfuscate it a bit more, but under the hood, every binary has an entrypoint.

We’re going to discuss what happens before main and what interesting things we can do there. In addition, we’ll be showing some novel techniques for mutable data that aren’t in common use in the Rust ecosystem today.

This post is a deep dive into some technical details of how Rust source becomes a Rust binary. Some background knowledge may be helpful to the reader, including:

• References in Rust
• Unsafe Rust

What might not be familiar to most developers is how you get into the main function. You see, under the hood for every language is the runtime. C has one: the C runtime that you might recognize as libc. Rust also has its own runtime: the Rust standard library. And because C is the lingua franca of runtimes for most executable code 1, Rust builds its own runtime atop of C’s, effectively building its own higher-level abstraction encapsulating C’s.

A runtime is a bit fuzzy to define. It’s both the executable code that lives on disk and compilable headers and libraries used at compile time. But the purpose of a runtime is always the same: integrating developer code with the platform’s operating system.

There’s an entire ecosystem of processing that happens before the function you declared as main starts up. C uses this to configure allocation, file access, thread-local storage and other C runtime services. Rust uses this time to configure parts of its own language and runtime. Specifically, Rust has infrastructure to handle panics and unwinding. Rust also needs to translate the C-style program arguments 2 into its own std::env::args interface. The machinery for all this is visible in the Rust compiler project.

Runtimes make use of this pre-main phase because it guarantees (1) running before user code, and (2) a single-threaded, highly-consistent and predictably-ordered environment, which allow for reliable and deterministic initialization.

By not taking advantage of this environment, you are missing out on a very useful bootstrapping phase. We’ll see later on in this post how we can build some useful primitives making use of life before main.

Entry Points

A binary starts when the operating system’s loader 3 - the part of the OS that loads the binary into memory and sets up the environment - hands off control. The runtime is responsible for accepting the hand-off from the loader. There’s a platform-specific hook on every OS that accepts the hand-off - to some extent this is the real main. On Linux, the entry point is stored in the e_entry field of the ELF header, and by default, the linker places the address of a symbol named _start there. A similar hook exists on Windows, and boots the executable in a function named _WinMainCRTStartup. At this point the C runtime has a chance to configure itself, and the way that all runtimes do this is via initialization functions.

In early iterations of runtimes, bootstrapping was a static tree of function calls: initialize file I/O, initialize the allocator, etc. As runtimes became more complex, this tree of function calls became more complex, and binary sizes increased to absorb more C runtime functionality that they may or may not need.

Over time, linkers developed the ability to discard unused code before even writing the binary to disk (including unused parts of the C runtime), and with that came a need for a replacement for the static init call trees.

The most popular method 4 of declaring init code came from GCC: __attribute__((constructor)). The way this worked was to place a list of init functions into a contiguous chunk of the binary on disk. When the C runtime started, it could walk through each of these functions and call them, allowing various bits of the C runtime to request initialization without strongly coupling subsystems, and allowing the linker to jettison unused subsystems, init code and all.

Eventually the need for constructor ordering became important enough that constructors could be given a priority and run in a specific order, allowing the runtime to initialize subsystems before and after each other. E.g., the memory allocation (malloc) subsystem might be needed for buffered file I/O.

On most platforms 5, the linker was called in to do the priority work: each platform ended up with a way to prioritize the order in which data gets written to sections, which allowed for the C runtime to end up with a well-ordered list of function pointers 6.

We can even build an example of this by hand in Rust using the #[unsafe(link_section = "...")] attribute (try it in the Rust Playground):

/// Linux example: the modern glibc runtime uses `.init_array` to hold function
/// pointers, and a numeric suffix allows them to be ordered. Note that priorities
/// less than or equal to 100 are reserved for the runtime itself, so any code that
/// wants to use the C runtime must use a priority of 101 or higher.

// On Linux, `.init_array` holds _function pointers_, not functions.
// We can convert a function to a function pointer with one of the below
// blocks which is equivalent to this:
//
// #[used] // <-- without this, Rust might decide the init function is unused and remove it
// #[unsafe(link_section = ".init_array.NNNNN")] // <-- the section where we place the function pointer
// static INIT_ARRAY_FN_PTR: extern "C" fn() 
//     = function; // <-- the function pointer data: we assign the function to it
//
// extern "C" fn function() { ... } // <-- the function itself

#[used]
#[unsafe(link_section = ".init_array.101")]
static INIT_FN_FIRST: extern "C" fn() = const {
    extern "C" fn init() {
        println!("Initializing (first!)");
    }
    init
};

#[used]
#[unsafe(link_section = ".init_array.201")]
static INIT_FN_SECOND: extern "C" fn() = const {
    extern "C" fn init() {
        println!("Initializing (second!)");
    }
    init
};

fn main() {
    println!("Main!")
}

linktime: ctor, link-section and more

The examples in this post will work on Linux and various BSDs, but are not designed to be cross-platform examples. For example, macOS has start and stop symbols, but they are named differently 7. Windows does not support start and stop symbols, but has a set of rules for sorting sections that is effectively equivalent.

Because platforms are so widely variable, we’ll be introducing the ctor and link-section crates (from the linktime project) as a way to abstract away platform-specific differences and hide the general complexity of linker work.

The excellent inventory and linkme are two other very popular crates built on the same principles, but have limitations 8 that make them less suitable for the examples in this post.

If you’d like to learn more, the link-section crate contains a detailed report on platform-specific behaviour.

The ctor crate is designed to handle all of the boilerplate of registering constructors in a cross-platform way. This allows us to simplify our examples above to:

use ctor::ctor;

#[ctor(unsafe, priority = 101)]
fn init1() {
    println!("Initializing (first)!");
}

#[ctor(unsafe, priority = 201)]
fn init2() {
    println!("Initializing (second)!");
}

fn main() {
    println!("Main!")
}

Note that neither example explicitly calls the init functions. The linker organized them in a way that the C runtime called them for us!

Sections and Linker Scripts

The process in which constructors are linked isn’t mysterious, though. In fact, compilers allow you to name the location in the binary (on most platforms called a “section”) you want to place any of your data and/or code. And by extension, and as we saw above, Rust allows this as well. The challenge, as we will see, is making use of this organizational feature.

Linkers have been the key to C’s ability to target any form of binary for some time. Most linkers allow for developers to provide linker scripts - text files that live alongside your source code (which is compiled to object files) and instruct the linker on how those object files are assembled. Using a linker script, a single C file might become a Linux executable, or a block of raw assembly that lives in the boot sector of a hard drive.

Linker scripts also allow for defining virtual symbols - that is, symbols that don’t exist in any source file but can be used by C code to access pointers to the underlying data in the loaded binary.

Linker scripts are a complex topic and beyond the scope of this post, but we can easily find examples of them in the wild:

// Adapted from https://wiki.osdev.org/Linker_Scripts
SECTIONS
{
  .text.start (_KERNEL_BASE_) : {
    startup.o( .text )
  }

  .text : ALIGN(CONSTANT(MAXPAGESIZE)) {
_TEXT_START_ = .;
    *(.text)
_TEXT_END_ = .;
  }

  .data : ALIGN(CONSTANT(MAXPAGESIZE)) {
_DATA_START_ = .;
    *(.data)
_DATA_END_ = .;
  }
}

In the above example, the virtual symbols _TEXT_START_ and _TEXT_END_ are explicitly defined to point to the beginning and end of the .text section, respectively. The period in _TEXT_START_ = .; is a special syntax that refers to a location counter that resolves roughly to the current output address in the binary.

Linker Symbols

This trips up most developers that encounter it for the first time, but the linker is setting the address of the start and end symbols, and therefore where the static with the same name is placed, and not setting the value of symbols that are pointers. That is to say: the start and stop symbols aren’t a *const Type. The start and stop symbols carry no data themselves and are used for their addresses only! The section consists of the range of data between the start (inclusive) and stop (exclusive) symbols.

Specifying start and end symbols for every section can be complex and tedious in linker scripts, so many linkers 9 eventually gained a feature where they could automatically define symbols bounding all sections in the executable. E.g., for GNU toolchains, a section named MY_SECTION will automatically have symbols __start_MY_SECTION and __stop_MY_SECTION defined. macOS has a similar pattern where it synthesizes a section$start and section$end symbol for each section.

In the GNU linker, those sections not explicitly defined in the linker script are called “orphan sections” 10. One important thing to note: if (and only if!) a section’s name is compatible with a C symbol name, the linker will automatically define a _start- and _stop-prefixed symbol for the section. In the example you’ll see below, the section name our_strings that we used works, but if we had chosen our.strings or .our_strings it would not have!

You’ll see in the example below that the start and stop symbols are MaybeUninit<()>. The boundary symbols contain no data, and only their address is significant.

The ideal Rust type for these would be an “opaque external type” (this would be implemented by the extern_types feature). As these are not currently implemented in Stable Rust, MaybeUninit is a stand-in. It signifies to the compiler that the data is uninitialized, and generally not safe to read via reference. Since taking a &raw const pointer to a static item is always valid, however, we can still safely capture its address without ever reading its value.

Try it in the Rust Playground:

use std::mem::MaybeUninit;

#[used]
#[unsafe(link_section = "our_strings")]
static FIRST_STRING: &'static str = "Hello, ";

#[used]
#[unsafe(link_section = "our_strings")]
static SECOND_STRING: &'static str = "world!";

// Note: these are not pointers. Instead, the linker has placed
// the boundary symbols STATIC_STRING_START and STATIC_STRING_END at
// the start and end of the section!
unsafe extern "C" {
    #[link_name = "__start_our_strings"]
    static STATIC_STRING_START: MaybeUninit<()>;
    #[link_name = "__stop_our_strings"]
    static STATIC_STRING_END: MaybeUninit<()>;
}

fn main() {
    let strings: &'static [&'static str] = unsafe {
        // SAFETY: get the addresses of the start and end symbols without
        // reading them.
        let start = &raw const STATIC_STRING_START as *const &'static str;
        let end = &raw const STATIC_STRING_END as *const &'static str;
        std::slice::from_raw_parts(start, end.offset_from(start) as usize)
    };
    
    // "Hello, world!"
    println!("String: {}", strings.join(""));
}

The link-section crate is designed to abstract away the details of these linker sections and convert them into traditional Rust slices with all standard slice operations available. We can use it to simplify the example above to:

use link_section::{in_section, section};

#[section(typed)]
static OUR_STRINGS: link_section::TypedSection<&'static str>;

#[in_section(OUR_STRINGS)]
static FIRST_STRING: &'static str = "Hello, ";

#[in_section(OUR_STRINGS)]
static SECOND_STRING: &'static str = "world!";

fn main() {
    println!("String: {}", OUR_STRINGS.join(""));
}

In these examples we’re submitting items to the link section in a single module within a single crate, but that’s not a requirement. In fact, the power of link sections is that you can submit items to a link section from any crate that contributes code to a binary - the linker will gather them all together just before writing the final binary.

Dependency Injection

The registration pattern we’re about to build is Dependency Injection by another name. This is a well-known pattern: frameworks like Dagger and Spring are built on the same principle that consumers of registration data should not be coupled to the providers of that data. A provider registers data at its definition site, a consumer simply reads the registry.

What’s somewhat different with linker sections versus traditional DI is that in DI the framework often needs to walk the module graph or scan loaded classes at startup to discover both providers and consumer sites. With linker sections, this magic is handled when the binary is written. The linker is the one that gathers all of the provider data and makes it trivially available to the consumer.

The example below uses a link_section::section to register CLI subcommands and is an instance of this pattern. More complex projects like Turbopack use this pattern to register string-pool constants, and the registration machinery used for serialization/deserialization and turbotask incremental compilation functions. A hypothetical webserver could make use of this pattern to register routes and middleware that are automatically collected at build time. The core mechanism is the same: the contributors place data into a shared registration system from any crate in the dependency tree, and the consumer reads the collected data without having to know where it was provided from.

Using Sections for Registration

One advantage we have in doing work before main is that it is well-behaved. No threads are running unless we start them. This means we are able to avoid the complexity of locks and other synchronization primitives in many cases, and that we can explicitly split our writable and immutable phase of our data’s lifecycle clearly: before and after main. And because of that, accessing data in the running program can become both simpler and more efficient by avoiding the need to acquire and release locks.

First, we’ll define our subcommand, a const constructor function, and a #[section] to collect them:

use std::collections::VecDeque;
use std::path::Path;

use link_section::{in_section, section};

struct CliSubcommand {
    is_default: bool,
    name: &'static str,
    description: &'static str,
    f: fn(&Path, &[String]),
}

impl CliSubcommand {
    const fn new(name: &'static str,
                 description: &'static str, 
                 f: fn(&Path, &[String])) -> Self {
        Self { is_default: false, name, description, f }
    }

    const fn new_default(name: &'static str,
                        description: &'static str, 
                        f: fn(&Path, &[String])) -> Self {
        Self { is_default: true, name, description, f }
    }
}

#[section(typed)]
static CLI_SUBCOMMANDS: link_section::TypedSection<CliSubcommand>;

Then we’ll register subcommands - these can live anywhere in your code:

mod list {
    #[in_section(CLI_SUBCOMMANDS)]
    static CLI_SUBCOMMAND_LIST: CliSubcommand = 
        CliSubcommand::new("list", "List all items", |_exe, _args| {
            println!("Listing all items");
        });
}

mod add {
    #[in_section(CLI_SUBCOMMANDS)]
    static CLI_SUBCOMMAND_ADD: CliSubcommand = 
        CliSubcommand::new("add", "Add a new item", |_exe, _args| {
            println!("Adding a new item");
        });
}

mod help {
    #[in_section(CLI_SUBCOMMANDS)]
    static CLI_SUBCOMMAND_HELP: CliSubcommand = 
        CliSubcommand::new_default("help", "Show help", |exe, _args| {
            println!("Usage: {} <subcommand> [options]", exe.display());
            println!();
            println!("Subcommands:");
            for subcommand in CLI_SUBCOMMANDS {
                println!("  {}: {}", subcommand.name, subcommand.description);
            }
        });
}

And then in our main function we can dynamically dispatch to any registered subcommand without ever having to know what they are or where they live. It only needs to be able to see the CLI_SUBCOMMANDS section definition:

fn main() {
    let mut args: VecDeque<String> = std::env::args().collect();
    let exe = args.pop_front().expect("No executable name provided");
    let exe = Path::new(&exe);
    let subcommand_name = args.pop_front().unwrap_or_default();
    let rest: Vec<String> = args.into();

    // Try to find the subcommand by name
    for cmd in CLI_SUBCOMMANDS {
        if cmd.name == subcommand_name {
            (cmd.f)(exe, &rest);
            return;
        }
    }
    // If no subcommand was found, fall back to the default subcommand
    for cmd in CLI_SUBCOMMANDS {
        if cmd.is_default {
            (cmd.f)(exe, &rest);
            return;
        }
    }
}

Running the code above works as you’d expect:

$ ./cli
Usage: ./cli <subcommand> [options]

Subcommands:
  list: List all items
  add: Add a new item
  help: Show help

$ ./cli list
Listing all items

Beyond Immutable Data

This section deals with some more advanced topics. Familiarity with Rust Atomics and Locks, or at least reading the first chapter on the basics of Rust concurrency, is recommended!

The example above assumes that the linked data is immutable. But that’s only half the power of using linker organization for data. Mutability in global static data is a common problem with well-known solutions in standard Rust. We could potentially use Rust’s built-in tools for interior mutability like mutexes, or atomic types, for example. Each of those comes with some runtime cost. If they are “uncontended” they aren’t expensive, but they are not necessarily free. 11

But what if we want to minimize the overhead of runtime data access? Immutable data is trivial: Rust allows safe concurrent access to immutable data by default 12. Rust has strict requirements for mutable data, however. There are two requirements to safely mutate data: (1) the modifications must be done in a thread-safe manner, and (2), there must never be more than one reference to the data if a mutable reference exists.

At the beginning of this post, we mentioned that life-before-main is a useful place to bootstrap because no threads are running unless we start them. And the solution to (1) is trivial if the data is currently accessible to a single thread only! We don’t need to do anything atomically; we only need to ensure that all of the changes we make to that data “happen before” any reads to the data. In a single-threaded environment, “happens before” is automatic 13. This means that we can mutate data in a link-section before main and it will be safe to access, lock free, from any thread after main.

The resolution for (2) is similar: as long as we only ever take a mutable reference (and only a mutable reference!) before main, there will never be more than one reference to the data when a mutable reference exists.

The pre-main environment satisfies both (1) and (2), without needing to reach for locks or other synchronization primitives.

There’s also one additional gotcha with linker sections we need to be very careful with: the slice that contains all of the items in a section is an alias to the static item that lives in the section. The rules about aliasing apply to both the slice and the static item, and you must ensure that static items are placed in UnsafeCell to safely mutate them from the slice 14. Rust does not allow a static item to be modified through other means. With static items that aren’t wrapped in an UnsafeCell, LLVM may consider itself free to cache, reorder or otherwise make assumptions about the data itself. UnsafeCell itself is not Sync, so you’ll need to add your own wrapper types on top of this!

Note that in the example below, we’re now using MaybeUninit<SyncUnsafeCell<...>> for the boundary symbols and SyncUnsafeCell<...> for the items.

Because we’re planning on sorting the slice, we need to tell Rust that the slice items are not immutable so the data doesn’t end up in read-only memory. By using a type that includes UnsafeCell - a semantic signal that Rust uses to indicate interior mutability - the Rust compiler will then know to place that data in a part of the binary that can be written.

On some platforms (Windows in particular), omitting this from the data items will cause segmentation faults when trying to sort the slice. On other platforms (AIX for example), section mutability is part of a section’s identifier, so the boundary symbols’ mutability needs to match the section’s mutability!

Let’s walk through an example of how we might otherwise do something like this. We’re going to build a string interning pool defined entirely at link-time, and add a wrinkle: we want to be able to sort the slice of interned strings at runtime so we can quickly intern a string by value if needed via binary search (try it in the Rust Playground):

use std::cell::UnsafeCell;
use std::mem::MaybeUninit;
#[cfg(debug_assertions)]
use std::sync::atomic::{AtomicBool, Ordering};

/// Nightly Rust offers a built-in `SyncUnsafeCell`. This is a minimal
/// reimplementation of that: 
/// <https://doc.rust-lang.org/std/cell/struct.SyncUnsafeCell.html>
#[repr(transparent)]
struct SyncUnsafeCell<T: ?Sized>(UnsafeCell<T>);
// SAFETY: safety burden of UnsafeCell is placed entirely on the user
unsafe impl<T: ?Sized + Sync> Sync for SyncUnsafeCell<T> {}

macro_rules! intern_string {
    ($name:ident, $string:literal) => {
        #[allow(unused)]
        const $name: &'static str = const {
            // This is not a common pattern, but it's entirely valid
            // to nest static items inside of const blocks. 
            // You can think of this as a way to hide the symbols
            // in a completely anonymous namespace.
            const VALUE: &str = $string;
            // Safety note: this static must _never_ be used. This is
            // purely a submission to the linker and _any_ access to it
            // may be UB.
            #[used]
            #[unsafe(link_section = "our_strings")]
            static ITEM: SyncUnsafeCell<&'static str> = 
                SyncUnsafeCell(UnsafeCell::new(VALUE));
            VALUE
        };
    };
}

intern_string!(WORLD, "world");
intern_string!(EXCLAMATION, "!");
intern_string!(HELLO, "hello");
intern_string!(FROM, "from");
intern_string!(RUST, "Rust");

unsafe extern "C" {
    #[link_name = "__start_our_strings"]
    static STATIC_STRING_START: MaybeUninit<SyncUnsafeCell<()>>;
    #[link_name = "__stop_our_strings"]
    static STATIC_STRING_END: MaybeUninit<SyncUnsafeCell<()>>;
}

/// Debug check to make sure the slice is sorted once and only once. This _could_
/// be enabled in release mode without any major performance impact, but we have enough
/// guarantees in place. Note that atomic access _does_ establish some memory ordering
/// guarantees, but the soundness guarantees are upheld with or without this atomic check.
#[cfg(debug_assertions)]
static SLICE_IS_SORTED: AtomicBool = AtomicBool::new(false);

// Implementation note: this function must not be called before `SORT_STRINGS_CTOR` has
// run.
fn interned_strings() -> &'static [&'static str] {
    // We use Acquire/Release pairing as a double-initialization check
    #[cfg(debug_assertions)]
    debug_assert!(SLICE_IS_SORTED.load(Ordering::Acquire), "Oh no! Slice was not sorted!");

    // SAFETY: we are calling this after main and we can guarantee that no
    // mutable reference is still alive. Since we know that no other code
    // is running before main, and that `SORT_STRINGS_CTOR` will run before main,
    // we can guarantee creating these slices is safe as 1) the sort "happens-before"
    // any access and 2) the mutable reference has been closed before any read-reference
    // access (satisfying aliasing XOR mutability requirement).
    let strings: &'static [&'static str] = unsafe {
        let start = &raw const STATIC_STRING_START as *const &'static str;
        let end = &raw const STATIC_STRING_END as *const &'static str;
        std::slice::from_raw_parts(start, end.offset_from(start) as usize)
    };
    strings
}

// Implementation note: this function assumes the slice has been sorted. See
// the guarantee above on `interned_strings` for reasoning.
fn maybe_intern_string(s: impl AsRef<str>) -> Option<&'static str> {
    let s = s.as_ref();
    let strings = interned_strings();
    strings.binary_search(&s).ok().map(|index| strings[index])
}

// SAFETY: We use the reserved `.init_array.0` priority because we do not
// access any C runtime functions (sort_unstable does not allocate) and we
// want to run before all other code. `.init_array.101` would work in our
// case, but this prevents other early-init code from accidentally running
// in the wrong order. `SLICE_IS_SORTED` is a debug check to make sure that
// doesn't happen. Note that all early-init code is tagged with `unsafe` so
// it always needs to be aware of safety guarantees of all APIs it touches.
#[used]
#[unsafe(link_section = ".init_array.0")]
static SORT_STRINGS_CTOR: extern "C" fn() = const {
    extern "C" fn sort_strings() {
        // We use Acquire/Release pairing as a double-initialization check
        #[cfg(debug_assertions)]
        debug_assert!(!SLICE_IS_SORTED.load(Ordering::Acquire), "Oh no! Sorted twice?!?");
        // SAFETY: we are calling this before main and we can guarantee that no
        // reference from `interned_strings` exists yet because we know no other
        // threads will be running, and we're not calling `interned_strings` yet.
        let strings: &mut [&'static str] = unsafe {
            // SAFETY: the bounds markers are not mutable, but we can safely
            // cast them to mutable pointers because we know the data behind
            // them is stored within `UnsafeCell` which is Rust's way of 
            // giving us interior mutability.
            let start = &raw const STATIC_STRING_START as *mut &'static str;
            let end = &raw const STATIC_STRING_END as *mut &'static str;
            std::slice::from_raw_parts_mut(start, end.offset_from(start) as usize)
        };
        strings.sort_unstable();
        #[cfg(debug_assertions)]
        SLICE_IS_SORTED.store(true, Ordering::Release);
    }
    sort_strings
};

fn main() {
    for (i, s) in interned_strings().iter().enumerate() {
        println!("[{i}]: {s}");
    }

    println!("{}, {}{}", HELLO, WORLD, EXCLAMATION);
    println!(
        "{}, {}{}",
        maybe_intern_string("hello").unwrap(),
        maybe_intern_string("world").unwrap(),
        maybe_intern_string("!").unwrap()
    );
}

The above example is pretty heavy (and a bit thicker thanks to the generous commentary), but it’s a good example of how much boilerplate crates like ctor and link-section can save you.

The equivalent using those crates can make use of the TypedMutableSection and a ctor to ensure the items are sorted before main. Note that the requirements for TypedMutableSection are that the items must be const - the reason is that the mutable section uses a similar style of code to the manually-implemented example above.

//! String interning pool using `ctor` and `link-section`.
use ctor::ctor;
use link_section::{in_section, section};

#[section(mutable)]
static INTERNED_STRINGS: link_section::TypedMutableSection<&'static str>;

#[in_section(INTERNED_STRINGS)]
const WORLD: &'static str = "world";

#[in_section(INTERNED_STRINGS)]
const EXCLAMATION: &'static str = "!";

#[in_section(INTERNED_STRINGS)]
const HELLO: &'static str = "hello";

#[in_section(INTERNED_STRINGS)]
const FROM: &'static str = "from";

#[in_section(INTERNED_STRINGS)]
const RUST: &'static str = "Rust";

#[ctor(unsafe)]
fn sort_strings() {
    let strings: &mut [&'static str] = unsafe { INTERNED_STRINGS.as_mut_slice() };
    strings.sort_unstable();
}

fn maybe_intern_string(s: impl AsRef<str>) -> Option<&'static str> {
    let s = s.as_ref();
    let strings = INTERNED_STRINGS.as_slice();
    strings.binary_search(&s).ok().map(|index| strings[index])
}

fn main() {
    for (i, s) in INTERNED_STRINGS.iter().enumerate() {
        println!("[{i}]: {s}");
    }

    println!("{}, {}{}", HELLO, WORLD, EXCLAMATION);
    println!(
        "{}, {}{}",
        maybe_intern_string("hello").unwrap(),
        maybe_intern_string("world").unwrap(),
        maybe_intern_string("!").unwrap()
    );
}

This particular example isn’t impossible without link sections, of course. What we get from the patterns we’ve discussed in this post are three things: (1) the guaranteed aggregation of tagged items, with all data pre-allocated and contiguous in memory; (2) the ability to distribute registrations anywhere in the code; and (3) a guaranteed count of the items in the section.

One major benefit that falls out of the three advantages above is that link sections require no allocations. If we were to rewrite this without link sections we’d be allocating a HashMap, Vec or other data structure, and potentially resizing it a number of times as we gather items (because we don’t actually know how many items we’ll have until runtime!).

The second major benefit that falls out is the Inversion of Control. The dependency graph for a traditional “collection” approach looks like below, with shared types deeply nested in the dependency graph, modules depending on that shared types module, and then a collector module that depends on all those modules to collect their types:

Shared Types

Module A

Module B

Module C

Collector

The change doesn’t seem large, but there is a large impact: the collector can now live anywhere and no longer needs to care what modules are contributing data:

(anywhere!)

Shared Types

Collector

Module A

Module B

Module C

And of course, we aren’t just limited to slices: you’ll find that there are analogues to many data-structures with link-time support in the scattered-collect crate:

• Scattered*Slice: Various Vec-like structures that provide slices (and optionally sorting).
• ScatteredMap/ScatteredSet: An analogue to HashMap/HashSet that provides hashed key-to-value lookup with some minimal pre-main initialization.

But Seriously: When Not to Use This

Link-time computation is fun and powerful, and it’s not always the right tool for the job. There is often a non-link-time equivalent: manually collecting data in crates that have visibility into each crate that wishes to contribute data. This can be inconvenient at times - instead of the contributors seeing a single contribution point “upstream” in a core crate, a “collector” crate with lots of crate references is required to collect them all.

Dead-code elimination becomes challenging: the link-section crate (and the linkme equivalent) both decorate all items using #[used], so the linker is disallowed from pruning unused data. Figuring out how to make link-time collection and dead-code elimination work well together is a complex problem beyond the scope of this post. For smaller bits of data like interned string atoms this might not be a problem, but if a program wants to intern larger chunks of data like chunks of raw JSON/JavaScript, or extensive data structures, this may add up to a lot of dead-code that may be difficult to identify.

Pre-main constructor functions have limitations: they cannot panic 15, Rust does not guarantee that all stdlib functions are available, and the order that the initialization functions are called within a given priority level is not guaranteed and highly platform-dependent. With careful planning, these limitations may be worked around but life-before-main may not be correct for subtle and difficult-to-debug reasons.

At this time, Miri is not fully compatible with all pre-main constructors and link-section constructions: it has a very basic view of pre-main execution, and does not model link sections at all. This may improve over time, but as of the time of writing, LLVM sanitizers (ASan, TSan, and others) are the recommended way to test for undefined behaviour.

The Inversion of Control pattern also has a cost: it makes it potentially harder to audit all the places that contribute data to a link section.

In reality, many widely-deployed and heavily-used Rust programs already rely on pre-main functionality: the ctor, link-section, inventory and linkme crates are used by many downstream crates today.

Briefly, on WASM

The examples above omitted a fairly important platform, though for good reason. WASM does not currently support linker sections natively because of an inconvenient choice many years back (51088 and 52353 for more details). Instead of allowing the #[link_section] annotation to place items in a true code section, the items are placed in a WASM custom section which is inaccessible to the WASM code itself!

The linktime crates do support WASM and have an emulation workaround that makes the approaches work for WASM binaries, but the author of this post hopes to make a suggestion in the near future on how proper WASM support could be added!

Conclusions

You can do a lot before main, and the benefits of doing so are pretty significant for certain cases. It’s a highly-ordered, highly-controllable environment that lets you more confidently do a lot of work without locks, atomics and other synchronization primitives. Link sections give you arbitrary aggregation and co-location of related data across your whole binary without awkward crate dependency order. In a lot of cases you can even avoid allocations completely which helps keep you away from one of the worst allocator sins: churn of allocations leading to fragmentation.

For further reading, check out the various crates discussed in this post:

• ctor: Module initialization functions that run before main
• dtor: Not discussed in this post, but the shutdown analogue to ctor
• link-section: Linker-managed typed (slices) and untyped sections, with mutability support.
• scattered-collect: Linker-managed higher-level collections: slices, sorted slices, maps

Thanks

Thanks to my lovely wife Mia, Benjamin Woodruff and Luke Sandberg, as well as @ssokolow for their feedback and review. This post would not have been possible without their help.

1. Go is a notable exception in that it avoids the C runtime on some platforms, only using libc as an ABI stability boundary on platforms that require it e.g., Apple’s libSystem.dylib and OpenBSD’s libc. ↩

2. On Windows, these are DOS-style (which were in turn derived from CP/M-style) arguments. ↩

3. Before the loader runs, the program is just some bytes on a disk and the loader (which can be the kernel itself or a user-space system component like ld.so on Linux) maps those bytes into memory and hands off control. ↩

4. The most popular method… in the humble author’s opinion. ↩

5. macOS does not support this. The C runtime does its own initialization and then just runs every user constructor function in the order the linker saw them. ↩

6. AIX has a special symbol naming convention for constructor functions: the sinit prefix, followed by a hexadecimal priority value. ↩

7. This will be discussed later in the post, but macOS synthesizes a section$start and section$end symbol for each section instead of a __start_ and __stop_ symbol. ↩

8. linkme creates distributed slices, but does not currently support WASM, and does not support mutable section data required to sort a section. inventory supports WASM, but requires a ctor-like function per item in the section. ↩

9. The Windows linker does not support this feature, but instead defines an overall sort order for symbols that is effectively equivalent. ↩

10. Orphan sections have a complicated algorithm for placement. ↩

11. For example, an atomic value must always be re-read, and that may incur use of CPU cache which is pretty numerous these days, but definitely not infinite. ↩

12. As long as it’s Sync. Sync means that it’s safe to share a reference to the data between threads. ↩

13. This is a complex topic, and Rust Atomics and Locks is the best resource for learning more. Starting a new thread means that all the previous writes “happen before” anything on the new thread, but we’ll leave the proof of this for the reader (or possibly a future post). ↩

14. Even taking a reference to a mutable static is disallowed by default in Rust 2024! ↩

15. Well, more accurately they can panic but they shouldn’t panic. In fact you’ll get a double panic: thread '<unnamed>' panicked at ...: pre-main panic! and then immediately thread '<unnamed>' panicked at ...: panic in a function that cannot unwind. ↩