Linux on the Sega 32X. Who needs hardware synchronization primitives anyway?

Interesting, I hadn't heard of Petersen's algorithm before. There's a similar algorithm by Lamport (of course): https://lamport.azurewebsites.net/pubs/fast-mutex.pdf .

After testing the waters with a Linux for Jaguar port; I basically set myself to get similar results on the 32X add-on for the Sega Genesis. Turns out that, even without hardware synchronization primitives, you can get smp-ready Linux: https://github.com/cakehonolulu/linux_ports

Neat to see this on HN after discussing this a bit with you on the Emudev discord (I'm Mask of Destiny there). I'm curious if you tested this on hardware. I think I mentioned this previously, but my understanding is that the SH-2s are incapable of writing to the cartridge area [0]. This would seem to preclude using the cart as RAM even with the extended "SSF2" mapper. Did you find that to be not the case or was this only tested in an emulator?

[0] - See this Spritesmind forum thread https://web.archive.org/web/20190209161834/http://gendev.spr... (Internet Archive link because the live site is mostly unusable due to the scraper deluge)

For those unfamiliar with the Hitachi SuperH architecture, imagine THUMB. ARM even had to license some of the patents from Hitchai when they developed THUMB. SuperH architecture is not the same thing as THUMB, but it has a lot in common, mainly use of 16-bit instructions, and instructions that only specify two registers rather than three.

But it also has some RISC jank of the era, such as branch delay slots.

This is exciting! I didn’t see anything about I/O, but could the serial port be used for terminal access (or to drive keyboard input for the console). Lots of untapped potential.

With the Sega CD you’d get more RAM and a second, faster 68k

These recent posts remind me of an earlier time in the 2000s when people were trying to boot Linux from anything computer-like

Congrats. Any idea why one cpu reports 10 bogomips and the other 20?

Thank you very much! Is the SSFv2 mapper supporting memory writes in the MiSTer FPGA, too? (Megadrive and/or 32x cores). Also, now that there's a SH3 core already available ([1] for replicating the Cave CV1000 system), it could be mind blowing having a system with 512MB of RAM on a SH3 at 100MHz running Linux.

[1] https://github.com/ika-musume/HS3

[0] - See this Spritesmind forum thread https://web.archive.org/web/20190209161834/http://gendev.spr... (Internet Archive link because the live site is mostly unusable due to the scraper deluge)

Hi! Nice to see you here.

Yes so, this was tested within a software context; the ASIC that does the bus arbitration on the side of the 32X has some of the required pins as N/C. They do a much more interesting deep-dive on the link you added.

I initially thought the 68000 side /UWR & /LWR pins were wired to the complimentary 315-5818 ones (Since there wasn't any apparent difference between the pinouts of both).

This makes the setup a bit different since you'd not be addressing the cart's PSRAM, but instead, the 32X's SDRAM. With XIP (Like on the Jaguar port) you could probably pull it off (Not sure if you'd be able to cram the entire Busybox environment but maybe a small program could do it).

Good thing is, this is actual Linux (And not uCLinux) so there's still stuff that can be disabled further if for some reason it can't fit still on the available memory.

But it also has some RISC jank of the era, such as branch delay slots.

> 16-bit instructions, and instructions that only specify two registers rather than three

These are not strange design features. I'm surprised they were patented? For example, most non-SIMD x86 instructions are two-operand. The trade-off between two-operand and three-operand has been known for a long time. (Shorter instruction encodings, but sometimes you need more instructions)

I actually didn't know the THUMB thing; pretty interesting stuff...

The branch delay slots/load delay slots thing was also something that plagued some MIPS processors back then too; 5th generation of consoles (At least, from Sony, Nintendo and Sega) had either all or a subset of those quirks on their flagship machines.

These recent posts remind me of an earlier time in the 2000s when people were trying to boot Linux from anything computer-like

This is exciting! I didn’t see anything about I/O, but could the serial port be used for terminal access (or to drive keyboard input for the console). Lots of untapped potential.

With the Sega CD you’d get more RAM and a second, faster 68k

It's kinda like "Running DOOM on x"

Hi! I'm basically forwarding UART data from the SH2s over to the M68K using a simple communication channel. Since the SH2s don't really have "direct" access to the Genesis's I/O port (They have TTL UART which caps at 4800 bps) I need this precise schema to have output. Technically speaking it should be simple enough to wire RXD too so you can interact with it. I saw some keyboard prototypes for the Genesis online (Which seem to use the Joypad ports) but I figured no one would have them so why bother writing a driver for it (Which again, would add more complexity to the M68K dispatcher; which does UART and acts as an INTC "master" to forwards IPIs between the 2 SH2s).

As to the Sega CD; it'd probably be cool but this is already super-slow as it is (There's lots of bus contention already from 2 x SH2s and 1 x 68000) so yeah.

Maybe a cool project would be SMP for the 2 68000s (Sega CD + Genesis) but I'm not even sure if there's any form of SMP available for arch/m68k in Linux (If both 68000s are wired you could maybe get away-ish with some simple bringup but can't really say for sure).

I actually didn't know the THUMB thing; pretty interesting stuff...

> 16-bit instructions, and instructions that only specify two registers rather than three

It's kinda like "Running DOOM on x"

Hi! Nice to see you here.

I initially thought the 68000 side /UWR & /LWR pins were wired to the complimentary 315-5818 ones (Since there wasn't any apparent difference between the pinouts of both).

Good thing is, this is actual Linux (And not uCLinux) so there's still stuff that can be disabled further if for some reason it can't fit still on the available memory.

I was actually thinking, if memory is the bottleneck; I could always go with the Saturn instead (Or find a way to have the 32X catridge writes go through…; but I’m no EE unfortunately).

I do recall having a RAM expansion cartridge in mine that could possibly be used to fit more stuff if the main 2MBs it had were too little to hold Linux.

You got me thinking…

[1] https://github.com/ika-musume/HS3

Hi!

Honestly, no idea if Krikkz's Extended SSFv2 is even implemented on MiSTer. I'd have to check but considering the rarity of it I'd be surprised if it was there.

Porting Linux to SH3 is surely easier than what it has been for SH2 (SH7604, more precisely); and considering it has 512MB of RAM you probably can just get away with a non-stripped to the bones Linux image (And initramfs, of course). It surely sounds like a good playground.

Congrats. Any idea why one cpu reports 10 bogomips and the other 20?

That was something I also noticed; I can't say for sure (Since I haven't yet checked) but I feel like I may have done something incorrect when wiring the FRT (The timer) to the secondary, smp-enabled SH2.

The BogoMIPS value is derived from a calibration that Linux does against the timer of choice so it could very well be a reason why.

I was actually thinking, if memory is the bottleneck; I could always go with the Saturn instead (Or find a way to have the 32X catridge writes go through…; but I’m no EE unfortunately).

I do recall having a RAM expansion cartridge in mine that could possibly be used to fit more stuff if the main 2MBs it had were too little to hold Linux.

You got me thinking…

The BogoMIPS value is derived from a calibration that Linux does against the timer of choice so it could very well be a reason why.

Hi!

Honestly, no idea if Krikkz's Extended SSFv2 is even implemented on MiSTer. I'd have to check but considering the rarity of it I'd be surprised if it was there.

As to the Sega CD; it'd probably be cool but this is already super-slow as it is (There's lots of bus contention already from 2 x SH2s and 1 x 68000) so yeah.

For what it's worth, the Sub 68K (mostly*) does not share memory with the main 68K. This makes SMP impossible, but on the bright side there's no contention. Sadly, there's also no way for the faster Sub 68K to directly access the cartridge so I think it would be difficult to run Linux on that CPU at all unless you can cram everything into 768KB (program RAM + word RAM)

* Word RAM can be swapped between the 68Ks, or can be operated in a split mode where each 68K gets half. There's no mechanism to allow both 68K's to access the same bank simultaneously though.

As a semi silly (But also possibly pragmatic, if we want to do this for more than simple 'fun') question... I wonder what it would take for projects like this, to take the video output and turn it into a (possibly not-visually-usable) jankhacked data output stream that could then be decoded?

I mean, there's a lot of hand-waving there (i.e. it would only be output which could complicate ease of using it practically) OTOH it might free up resources?

(I'm not experienced in this hardware level so I apologize if this idea is absolutely stupid/impractical.)

* Word RAM can be swapped between the 68Ks, or can be operated in a split mode where each 68K gets half. There's no mechanism to allow both 68K's to access the same bank simultaneously though.

Ah, that'd be it then; it would've been such a cool experiment.

Regardless, I don't really know if there's the notion of an SMP-enabled m68k board based on Linux so I would not really know where to start.

For SuperH, I've gotten very lucky because there's a single CPU subtype (J2, https://j-core.org/) which has SMP tidbits that helped me understand what to look for and where in terms of wiring the 32X's CPU subtype.

I mean, there's a lot of hand-waving there (i.e. it would only be output which could complicate ease of using it practically) OTOH it might free up resources?

(I'm not experienced in this hardware level so I apologize if this idea is absolutely stupid/impractical.)

Not a silly question actually.

There's really clever ways you can get down to scratch that space-saving itch; I'm not knowledgeable about it either (I'm just really bad when it comes to analog signals) but if you really, really needed to shave off even more stuff; I reckon that the video console driver could possibly be reduced to something far more esoteric (That'd need signal processing to get tangible output of). But at that point, just use UART (Which is simple-enough already and should not really starve you out of much memory).

Ah, that'd be it then; it would've been such a cool experiment.

Regardless, I don't really know if there's the notion of an SMP-enabled m68k board based on Linux so I would not really know where to start.

Not a silly question actually.

Well, when I say resources in this case, I mean UART bandwidth and the overhead of synchronizing that for output.... not so much memory itself (well, unless the overhead of the hackery for whatever 32x does as far as VRAM is worse, not sure if it's another tile engine or one just bangs bits...)

Interesting, I hadn't heard of Petersen's algorithm before. There's a similar algorithm by Lamport (of course): https://lamport.azurewebsites.net/pubs/fast-mutex.pdf .

Preface

I couldn't start this entry in any other way possible but by thanking everyone who enjoyed the Linux on Jaguar post.

Also special props to the linuxmd project again.

It was a very fun experience and we've even made it into a few news outlets:

I also posted it on Hacker News and it recieved some good interest (And nice questions!).

So, yes; thanks everyone again!

Why is this (Linux) trend continuing?

I have an easy answer to this question, it's mainly to improve my board bringup skills.

I've been working professionally on the Firmware/Operating Systems/Embedded area for well over 2 years now (Professionally-speaking) and one of the usual suspects you see when trying to land a new job is "the ability to handle board bringups from the ground up".

I know it doesn't sound too fascinating but for me, other than job hunting (Which is not my bread and butter anyway), bringing up a piece of silicon and having it run something you made/helped making is super fun. You get to experiment with many of the different layers that make the result up: Emulators, Technical documentation, Experimentation, Investigation... you know; the usual engineering stuff.

Not that reading early-90s scanned datasheets is my passion (There's some documents that are plain images embedded onto a PDF) but there's something to it that makes me come back every time.

I actually started doing this in '13/'14. I was 12/13 at that time and I vividly recall my parents buying me my first phone. It was a MediaTek-based 6589 quad-core processor with a gig of RAM. Powerful stuff eh!

At that time (Not much soon after), Google released Android 5.0 Lollipop; but the vendor of the phone decided that it would not update it to that version. And I was really sad... so I had to get my hands dirty.

I'll sum up the story a bit because it's a bit large, but the gist of it is that I asked the company to comply with GPL and release the Linux kernel source code for the phone; for which they opposed, but as a community we got together and they released it (Along some other phone's trees).

Picture me surprised when the source code was half-assed and the board configuration didn't boot on the phone. Fortunately at that time, I found a leak of the full-source of MediaTek ALPS targetting a variation of the 6589 that seemed promising enough (Had the full Little-Kernel bootloader, Baseband, Preloader, Linux kernel and AOSP) for me to check.

I spent a few months trying to reverse-engineer parts of the zImage that was bundled on the stock boot image provided by the phone vendor to fill parts of the MediaTek code (Mostly related to lcm, battery, chipset quirks and whatnot) and after some good amount of digging and experimenting (And good chats with the cool people over at xda-developers) I had Android 5.0 running on my phone; mind you, it wasn't perfect. No Bluetooth, no HWComposer (Hardware acceleration), no Media engines... but seeing the pure, unadultered new Material Design on my phone blew my mind.

Android 5.0 Lollipop

I should probably try to make a separate post of this story... someday.

The mighty Sega 32X

Released in November 1994 in America/Europe (December 1994 on Japan); the Sega 32X was an add-on that promised to be an intermediate step between the 16-bit era Genesis and the soon-to-be 32-bit Sega Saturn. I'm not a good writer so the story that led up to that particular moment (For SEGA, that is) is a bit tumultuous (SEGA Japan vs SEGA of America) so I'll leave it as a task for the reader.

Anyhow, prior to the 32X, SEGA launched a CD add-on; the Sega CD (Or Mega CD, if you are european/japanese). It plugged into the expansion slot of the Genesis and it enabled CD-quality audio streaming (And well, game/game asset streaming). It was a machine on it's own (It had a higher clocked 68000 in comparison with the Genesis). There were 2 separate models, the first "generation", which was front-loading...:

Sega CD 1 Sega [Mega]CD Model 1 with Mega Drive/Genesis Model 1

...and the second, top-loading one:

Sega CD 2 Sega [Mega]CD Model 2 with Mega Drive/Genesis Model 2

But this post is about the 32X... so here it is:

Sega 32X

The actual addon was plugged into the cartridge slot of the Genesis (It had a separate power brick and video output, later of which was "combined" on-top of the Genesis video out timing/video/color channel signals).

Sega 32X & Genesis Model 2

If you've ever had the pleasure of building the Tower of Power, you'll know that having 3 separate power bricks, plus the cables to get stereo output out of the 32X, the "patch" cables and the many missing tidbits I'm probably forgetting now, wasn't anything short of fancy.

But what did all of that power?

32X Hardware specifications

Admittedly, the addon is far more powerful than the base console:

Component(s)	Sega Genesis/Mega Drive	Sega 32X
Main CPU	1 x Motorola 68000 @ 7.6 MHz	2 x Hitachi SuperH SH2s (SH7604) @ 23 MHz
Bits	16-bits	32-bits
Max. on-screen colors	64	32768
System RAM	64KB	+256KB SDRAM (Along Genesis's 64K)
Video RAM	64KB	256KB (Double-buffered)
Graphics	Sprite-based	Software-rendered 3D (Flat shaded, goraud shaded, simple texture-mapped polygons)

As you can see, quite the difference. I've not even listed all the differences, but the main ones I can think are those.

Unfortunately for SEGA, the launch happened almost at the same time as Sony's PlayStation 1; which was primed to overcome the gaming industry with a much simpler architecture. It also didn't help in any way that the SEGA Saturn was close to launch date too (So consumers were puzzled as to which one to get). All in true SEGA fashion!

Anyway, if we got Linux running on the Jaguar, we should be able to do similarly for the 32X; right?

jmp _linux, the rendezvouz

Getting Linux to run on the SH2s (Running as in, having it print anything meaningful over UART) required some brainstorm.

We'd need to set up 3 different CPUs, the 2 x SH2s and the 68000. Fortunately there's readily-available examples for that on Github so I picked what seemed the most popular 32X Hello World example: Chilly Willy's Sega MD/CD/32X devkit.

The basic example sets up a communication channel between the 32X(s) and the 68000 (So you can ask the 68000 do some stuff the 32X can't) through specific memory-mapped IO addresses; prepares the SH2s (Sets up the caches, uploads the SH2 code from the cart directly off of 68000 machine code...) and makes the primary SH2 jump to it; whilst the secondary gets parked.

What I did was, expand this so that I could have a simple state machine that pushed characters to the UART on the Genesis (Since the 32X doesn't have direct access to it); but there was a problem...

256K + 64K

The biggest hurdle when porting Linux to a new (And newer-ish, in terms of when it launched) platform is mostly about doing the wiring to get it up and running; that's usually the case if you have the memory to even load it. This is not the case for us...

Fortunately, there's FPGA-based flash carts for the console (Enables you to run games for both the base model and the 32X; and there's even specialized "cores" that provide Sega CD support if you don't have the add-on!). One of those (series) of carts comes from Krikkz.

He basically did what flashcarts do but went the extra mile for pretty much everything, it's not just about running games!

There's a special bit of those carts that we're interested on:

Extended SSFv2 Mapper

Older cartridge-based systems often had this notion about "mappers".

Mappers are basically ICs that sat on the cartridge which enhanced the capabilities of what the console can handle in terms of memory.

They could also offer other features not related to memory, an example is Sega's Virtua Processor (SVP), which had a Samsung-engineered DSP to do heavy math computation (To enable Virtua Racer to display all it's 3D polygon glory on a base Genesis); or Nintendo's SuperFX chip; which provided similar capabilities to the SVP and was made by the creators of Days of Thunder or Birds of Prey: Argonaut Games.

Usually mappers were used to enable more ROM space for larger/more detailed games in a bank-switch fashion. You'd write to a certain memory address/do a specific set of accesses and the IC would internally remap parts of the "extended" ROM to the memory window the console can directly address. Kinda like MS-DOS's HIMEM.SYS in spirit if you think about it.

So Krikkz basically did us all the work here, turns out we can have ROM as RAM (ROM which can be made "writeable" by arranging certain configuration bits). This gets us 4 MBs of RAM.

I had to basically extend the SDK example to do the ROM->RAM mappings but had to leave a bit of space for the Linux + initramfs binary to live on (The actual ROM within the cart memory window, basically). I ended up finding a sweet spot of 540~KBs of ROM and rest of RAM. The Linux loader basically copies the binary on the memory location it was linked at and jumps to it.

Compiling and running Linux

So, I made a sh2eb-elf cross-compiler and got to compile Linux... but the compilation failed rather soon.

Out of all the issues I've had with toolchains, this one was probably the funniest.

For some reason GCC was having an ICE (Internal compiler error) when compiling kernel/nstree.c. I was able to find a workaround (Didn't have the time at that moment to file a bug report) fortunately.

After all this work (And writing a small earlycon driver to get some UART output carried from the 32X to the Genesis) I had the very first Linux messages.

There was a separate issue, at some point Linux seemed to stop the boot process and was hanging indefinitely.

I then discovered another set of issues, this time regarding: ashlsi3, ashrsi3 & lshrsi3. I'm not really sure how those are written (Do they get copied/inspired from their libgcc-counterpart or something similar?) but comparing what the cross-compiler did for those and what Linux did, it was obvious that we were clobbering a register we should not be clobbering. I found this because I started noticing that some pr_info(s) and pr_notice(s) were getting it's stack corrupted (And never returned to the caller) and was able to trace it here.

I then made an additional (Similar) fix for arch/sh/lib/checksum.S just to be on the safe side of things; where I basically gated shrd because it doesn't exist on the SH2s contained within the 32X (And if used... Illegal Opcode).

This basically got us through the entire boot process right before the clock calibration; for which I decided to make a simple driver for the SH2(s) FRT (Free running timer); this also got us further and we were at the verge of launching the init process.

Fortunately, this time around I was able to leverage ELF FDPIC support (Which made my life easier in comparison with Jaguar's FLAT binary support) to build my rootfs. I was also blessed by musl-cross-make; which landed me a sh2eb-linux-muslfdpic- toolchain I could use (I must say I battled hard with crosstool-ng, buildroot and a few others but didn't succeed).

So with this, everything was booting.

Case... closed...?

SMP

Yeah, I couldn't help myself.

It turns out that you need some hardware primitives to enable inter-CPU synchronization... which the SH2s on the 32X don't have¹.

¹: TAS.B opcode seems to be used on projects like D32XR even after the developer manuals asked not to do so due to the adapter bus not having the locked read-modify-write cycle mechanism.

We also don't have cache coherency so another problem to add to our bag.

But I thought, hey, both SH2s are equal features-wise; we sure can come up with something... right?

It turns out that sparc32 has precisely some code to aid in that.

We can basically leverage Peterson's algorithm to implement the required primitives entirely in software. Mind you that this isn't performant but it does work.

I basically had to do the bringup for all the primitives that needed care (cmpxchg, for instance) with the ideas that Peterson described.

I also needed to wire the 68000's code (Which for now was only handling UART stuff) to aid on delivering IPIs to both SH2s (I extended the commands the 68000 can receive so that it could correctly do it's thing) plus setting up the shared memory region to aid on the syncronization tasks. Think of the 68000 as being a bus arbiter/extra interrupt router at this point.

From the SH2s I just send an "IPI" command which the 68000 parses and decides which interrupt line to assert based on a bit of the data word (0 primary, 1 secondary).

Then there's the cache-coherency issue, which I solved by not using cached mappings (And well, cache is disabled, regardless) to place Peterson's flag and turn variables (You know, to avoid SH2s spinning on stale cache values for flag; for instance).

There was another issue I faced, how do I identify which CPU is which (So Linux can properly manage and issue hard_smp_processor_id()): Well, considering I was doing softfloat regardless; I just made the CPUID_REG point to the SH2-private hardware divider register (DVSR) which had no other use so it effectively became my what cpu am i register. That register is written by the Linux loader at boot for both the primary and secondary SH2s.

Performance is abysmal, bus contention is bonkers; but it does work, and that's fine by me.

smp: Bringing up secondary CPUs ...
MARS SMP: released secondary SH2 to 22082000
smp: Brought up 1 node, 2 CPUs

I must admit that it wasn't as easy as that, it really involved lots of tinkering and (Mostly) trial-and-error (I also spent 2 nights chasing why both CPUs would get woken but soon after they'd idle, turns out I needed a fix on default_idle() that was making the scheduler bug out and not correctly continue execution).

But it was well worth it.

I just then basically made a simple console display driver (So you know, you could see it in action in your actual screen)...

Linux on 32X

Voilà

Needless to say that all of this required lots of fine-tuning (I was also OOM-ing constantly and needed to put both Linux and Busybox through a good diet) but it was again, a very rewarding experience.

Misc. config. files if anyone wants to try

Linux tree (mars_defconfig)

Busybox config

Init script

Hacker Times