A few years ago, a paper came out that blew our minds. The idea was that you can decode what someone is looking at just from their brain activity.

Reconstructing seen images from brain activity — seen image (left) and reconstruction (right) for each (MindEye, decoded from fMRI)

It’s wild and shows just a glimmer of what a telepathic future would be like. Unfortunately, it requires an MRI machine, which sadly can’t be worn on the head.

In fact, the first bottleneck to the whole field of mind interfacing is the hardware. There are currently two extremes: drill a hole through your skull and stick electrodes in your brain, or record blurry-at-best images of brain activity outside the head with EEG.

We’ve been building a new type of hardware that requires no drilling, and gives you MRI-level detail of the brain.

It’s based on ultrasound. It exploits a connection between your vascular system and your neurons — when neurons fire, more blood is delivered to the neurons. We send ultrasound waves through the skull, and they scatter off red blood cells. We can then form maps of blood flow and volume throughout the brain.

Ultrasound propagating through the human head.

We think there are two requirements in a general-purpose mind interface. The first is that it has to be able to see a large part of the brain. Even with 1000 electrodes, you capture at most 0.001% of the brain. This is great for a narrow task like controlling a cursor. But thoughts are distributed all over the brain.

The second requirement is detail, or resolution. Modalities like EEG and MEG have great field of view, but capture blurry images of brain activity. This is fundamental, it’s due to the way electric and magnetic fields propagate, and this is not solved by scaling to millions of sensors.

Neurovascular ultrasound — like MRI — hits both of these requirements. The physics allows for recording a million independent pixels throughout the brain, at less than a millimeter each. It’s produced wonderful results in the last few years when the skull is removed. But the challenge is doing it with the skull intact.

First light

Today, we’re sharing a milestone: the most detailed vascular image of a living human brain (to our knowledge), captured with ultrasound through the skull.

The reconstructed vascular volume of a living human brain, imaged through the intact skull

We can see the large vessels, the pial arteries, and the arterioles. It’s the world’s first 3D image of ultrasound localization microscopy in a human brain through a skull, and achieves a resolution that’s 100 times greater volumetrically than comparable CT.1

We know that there will be many applications of transcranial microbubble imaging beyond what we’re working on, and we’re therefore open sourcing the entire pipeline along with the dataset. Conditions like stroke, Alzheimer’s, traumatic brain injury each leave vascular signatures at scales CT and MRI can’t resolve, and we expect imaging at this resolution to reach them.

Microbubble processing pipeline

Microbubbles let us beat the diffraction limit. Ultrasound normally can't separate two objects closer than about a wavelength — anything finer collapses into a single blob.

A single microbubble blurs into a wavelength-wide spot, but a sub-pixel fit pins its center far below the diffraction limit

The trick is concentration. Inject the bubbles sparsely enough that their blobs don't overlap, and you can pinpoint the center of each one far more precisely than the wavelength itself. As bubbles flow through the vasculature, we accumulate millions of these positions and stack them into a single image with detail finer than the wavelength.

Raw ultrasound resolves only a few wavelength-wide blobs; localizing each bubble's center recovers the vessels threading beneath them

The bubbles themselves are pockets of sulfur hexafluoride encapsulated in lipid shells. They're an FDA-approved contrast agent, and we infuse them continuously over a 4-minute acquisition. The gas has an acoustic impedance far from that of tissue, so sound reflects sharply at each bubble's surface — which strengthens the signal on top of enabling super-resolution.

Bubble centers are linked frame-to-frame into tracks, shown here in 3D. Their direction and speed trace blood flow through the living microvasculature.

Toward contrast-free neurovascular imaging

Our contrast-enhanced results are a step in the journey. They give us a confident picture of the vascular detail that’s achievable through an intact skull. The real destination is contrast-free neurovascular imaging of the brain.

Two trends give us confidence we’ll get there. The first is hardware. Ultrasound machines used to cost over $100,000 and require a cart full of electronics. Thanks to companies like Butterfly, they’re now about the price and size of a smartphone, and they keep getting better.

The second is data. Contrast-free imaging is harder. Red blood cells scatter far less than microbubbles, so the signal is weaker. But that signal isn’t lost. Today’s methods just don’t pull it out. A standard ultrasound probe receives terabytes of data per hour, but the typical processing pipeline compresses this down to just 0.1% of the original. It’s built on hand-engineered features, and it reminds us of early computer vision. We believe end-to-end machine learning, trained on large enough datasets, will recover far more signal than current methods can see.

That’s why we’re currently collecting what we believe is the world’s largest dataset of neurovascular ultrasound. We’re excited to share what comes next.

Notes

1. Note though that this is using the super-resolution trick, which is only available to the contrast version of neurovascular ultrasound. ↩