Since posting on X, I've gotten many DMs asking exactly how I want to approach the next phase of this project: making the drone fly with RL. Here's the plan I have so far.

Most importantly, the RL policy will directly command all 8 motors at 50 Hz over a serial link to the flight controller with no traditional PID loop in the path. This is the only architecture that gives the policy full authority to reallocate thrust when motors fail.
I'm focusing on six unique failure classes (ignoring rotational equivalence): single motor, adjacent pair (45°, mixed CW/CCW), 90° same-type, 135° mixed, 180° same-type, and full ESC loss (each ESC controls its own quad). The hardest case is the 90° same-type failure, because it's the only one that hits both problems simultaneously: a yaw torque imbalance (the two dead motors were the same spin direction) and a spatial asymmetry in the remaining thrust geometry.

The single- and dual-motor failures that I want to support, plus ESC loss

Losing two same-spin motors leaves 2 CW and 4 CCW running (or vice versa), yaw-torque imbalanced 2:1 at equal throttle. Balancing them forces the CW motors to run at 2× the per-motor thrust of the CCW motors. At 1393 gf max per motor, the yaw-balanced thrust ceiling works out to 5,572 gf -- enough to maintain a 2:1 thrust-to-weight ratio up to ~2.8 kg of drone weight (we're at 1 kg). The remaining 6 motors span a 270° arc, so roll and pitch authority still exists. The worst case is survivable -- the drone would be spinning, but it could still hover to a soft landing.

Simulation

I'm building the sim in MuJoCo, because it runs fast on a CPU and I have a Mac, which rules out Isaac Lab and basically everything else NVIDIA-shaped. For a single rigid body with 8 thrust points, MuJoCo is more than enough, and I can run ~128 environments in parallel on my laptop.

The model itself comes from measurements, not the CAD. I'll be gathering data on:

• Total mass
• Inertia tensor via the bifilar pendulum test
• Motor thrust curves
• Motor time constant
• Hover throttle point

I'm also adding two things to my sim environment that I keep reading are what actually kill sim-to-real transfer for motor-level control:

1. Motor lag: real motors take 20–50 ms to reach a commanded speed. In sim, thrust changes instantly unless you model it. A policy that learns with instant motors learns to twitch.

2. Loop latency: on the real drone, there's ~15–30 ms between the IMU reading and thrust actually changing (serial read, inference, serial write, ESC response). If I train with zero latency, the policy will oscillate the second it touches hardware. This one scares me the most, so it's getting randomized aggressively (the policy trains against a delay that changes every episode and jitters within episodes).

The high-level plan moving forward

puffer. just puffer. trust me. you will thank me in about a month from now.

— Chris von Csefalvay 🔜 CVPR26 (@epichrisis) June 7, 2026

Everything else physical gets randomized too: mass ±10%, per-motor thrust constants ±15% (cheap motors are not identical, I own eight data points proving this), center of mass, battery sag over a flight, sensor noise[4].

Training

PPO[1] via PufferLib. I looked at SAC since it's more sample-efficient, but sample efficiency solves a problem I don't have -- my sim steps are nearly free. PPO with a pile of parallel environments is what almost every sim-to-real flight paper I've read actually shipped, and it scales more simply across parallel envs and is more stable under the variance that heavy randomization adds". (Also: an X reply told me "puffer. just puffer. trust me.")

Two more decisions I stole from sim-to-real literature:

1. The critic gets to cheat. During training, the value network sees ground truth the real drone will never have, like which motors are dead, the exact thrust constants, and true velocity. The actor only sees what real sensors provide. The critic gets thrown away after training, so this costs nothing at deployment. (This is called asymmetric actor-critic[2], and I've read that it makes a huge difference when the physics are randomized this hard.)

2. No fault detector (for now). The policy sees its last 5 observation/action frames and has to figure out failures on its own, from the gap between what it commanded and what the drone did.

Under a same-type dual failure the drone physically cannot hold its heading -- the torques don't balance at any throttle combination. The right behavior is to give up on yaw, spin slowly about vertical, and stay level. If the reward punishes spinning, the policy sacrifices roll and pitch chasing a heading it can't have. Mueller & D'Andrea showed the same thing for quads losing a motor[3] -- their recovering quad spins the whole time. Mine will too, on purpose.

Deployment

If the policy shows promising survival rates in sim, it'll get exported to ONNX and run on the RPi 4 (I think. Any opinions on this vs other microcontroller options?) the network is ~45k parameters, which is under a millisecond of inference, so the Pi is not the bottleneck. The 50 Hz loop will read attitude and gyro over serial, run the policy, and write 8 motor commands.

Then, the actual experiment: fly, kill motors from the transmitter, and find out if millions of simulated crashes taught it anything!

1. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, "Proximal Policy Optimization Algorithms," arXiv:1707.06347, 2017.
2. L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, "Asymmetric Actor Critic for Image-Based Robot Learning," RSS, 2018.
3. M. W. Mueller and R. D'Andrea, "Stability and control of a quadrocopter despite the complete loss of one, two, or three propellers," IEEE ICRA, 2014.
4. J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, "Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World," IROS, 2017.