Visual Analysis of Binary Files

I've also built a visualizer into ClickHouse, so we can analyze its binary: https://play.clickhouse.com/binary?user=binary - which code takes more space, what dependencies are too big, or what template instantiations are bloated.

Pretty neat. It tangentially reminds me of when we used to be able to hear HDDs clatter away, or waaaaay back when we used be able to hear EM interference from whatever was going on in a computer.

There was an episode of Cowboy Bebop, where a mechanic held a wrench against an engine cowling to see how much it vibrated (likely a real thing people do with real engines?) and, of course, the computer sounds in the Enterprise-D, along with blinkenlights.

All this seems like the same kind of "translate data into a form humans can perceive." I'm not sure what immediate value this utility has, but I like it.

I made my own binary visualizer when I was learning C. I had a for-loop which generated a texture by just setting the pixels based on the bytes in the file.

I had two modes, monochrome (the bits) and grayscale (the bytes).

Anyway funny thing happened, I had a buffer overflow and my program ended up visualizing its own memory.

Actually I had the same error another time, I was trying to make a simple Terraria clone (like 2D Minecraft) and the terrain ended up rendering my mouse coordinates interpreted as booleans or something.

C is fun!

Discussion from a few days ago that had links to more visualization tools: https://news.ycombinator.com/item?id=49423878

This brings back memories! In 93 I wrote something similar (simpler) for MSDOS, to inspect and extract pieces of binary files, GFV (still downloadable from the Hornet archive somewhere in here https://www.hornet.org/cgi-bin/ha_search.cgi?opt_s=5:Iguana). You could set the stride and show 1/n bytes, so it was relatively easy to find textures from game files and memory dumps of running games.

Not exactly the same but https://runhexed.com allows you to create binary visualizations online

“You get used to it; I don’t even see the code.

“All I see is blonde, brunette, redhead…”

https://youtu.be/fBDifUjNzbQ?is=7d43YcQZ8PVhFX11

Does the uniformity indicate that the files are space-inefficient, and would benefit from compression?

It looks neat but I have no idea how the clustering works or why uniform colors are assigned to large areas, e.g. the color blue in the pdf example.

I made my own binary visualizer when I was learning C. I had a for-loop which generated a texture by just setting the pixels based on the bytes in the file.

I had two modes, monochrome (the bits) and grayscale (the bytes).

Anyway funny thing happened, I had a buffer overflow and my program ended up visualizing its own memory.

C is fun!

Discussion from a few days ago that had links to more visualization tools: https://news.ycombinator.com/item?id=49423878

Not exactly the same but https://runhexed.com allows you to create binary visualizations online

“You get used to it; I don’t even see the code.

“All I see is blonde, brunette, redhead…”

https://youtu.be/fBDifUjNzbQ?is=7d43YcQZ8PVhFX11

Pretty neat. It tangentially reminds me of when we used to be able to hear HDDs clatter away, or waaaaay back when we used be able to hear EM interference from whatever was going on in a computer.

All this seems like the same kind of "translate data into a form humans can perceive." I'm not sure what immediate value this utility has, but I like it.

Does the uniformity indicate that the files are space-inefficient, and would benefit from compression?

I forget where I read this (maybe PG's blog?) but I read some early 2000s startup had some kind of mechanical device that responded to website visitors, so everyone in the office could hear the traffic in realtime.

I also heard stories of professors who would know a program had a bug based on the sounds the computer was making. (Back when they were slightly slower!)

The wrench thing is real. My approach to a DIY mechanic’s stethoscope is to stick the handle of a screwdriver against my ear and the tip on whatever I want to listen to. Great for figuring out what part of the machine a noise is coming from.

Binvis has different color modes. One of them shows entropy. Compressed or encrypted data shows as very bright in the entropy mode, and repetitive data as dark.

It looks neat but I have no idea how the clustering works or why uniform colors are assigned to large areas, e.g. the color blue in the pdf example.

open one and inspect the two ways to view it, one is horizontal scan, the other is a peano curve which leads to natural square-ish areas.

I also heard stories of professors who would know a program had a bug based on the sounds the computer was making. (Back when they were slightly slower!)

Binvis has different color modes. One of them shows entropy. Compressed or encrypted data shows as very bright in the entropy mode, and repetitive data as dark.

I’ve heard this about mechanical electronic signage used for team dashboards 30-40 years ago.

You’d sit in the office with your back to it and the clicky clacky sounds would be going every so often when the content updated. but people would know the specific clicky clacky sound that required them to turn around and read the sign based on their job function.

I’ve also heard of day traders mapping stock tickers or other data into sound and having headphones on all day with it

open one and inspect the two ways to view it, one is horizontal scan, the other is a peano curve which leads to natural square-ish areas.

Interesting I thought it was a hilbert curve? (I don't know much about either of those though.)

I can only see the color visualization and the hex view. Maybe I can't see the curve due to lack of WebGL?