I Let an AI Tune My Firmware for 40 Iterations. It Made Things Worse.

If you build firmware today without an AI assistant, you are doing it the hard way.

The shift happened gradually and then all at once. A year ago I was using AI to autocomplete functions. Then to write entire modules. Then to debug problems I described in plain English. By now, the thing I would have spent two or three days writing, reading datasheets, sketching logic, iterating on edge cases, I get done in a focused afternoon with the right prompts. The output is clean, correct, and often better-structured than what I would have written myself.

That is not an exaggeration. On pure technical output, reading code, understanding protocols, spotting logic errors, writing idiomatic firmware, a good AI assistant today matches what I would expect from a solid senior developer. Maybe better, because it never gets tired and never skips the boring parts.

So naturally, I started wondering what would happen if I stopped guiding it and just let it run.

---

The idea

I was building an accelerometer-based device for velocity-based training, the kind of sensor that attaches to a barbell and tracks movement so a coach can read metrics like bar speed and rep quality in real time. The firmware’s job was to identify the phases of each repetition: when a lift begins, when the athlete is in the eccentric phase (lowering the bar), when they shift to concentric (driving it back up), and when the rep ends.

Phase detection like this is not trivial. It involves a chain of decisions: which axes to read, how aggressively to filter the signal, where to set the thresholds that separate noise from movement, how long a pause is a rest versus a slow reversal. Every one of those decisions is a parameter. Change the wrong one and the rep boundaries shift by half a second.

The idea was simple: instead of tuning those parameters manually, let the AI do it autonomously.

The setup would work like this. I had two devices collecting data simultaneously: my prototype and a reference sensor I trust. Both connected via Bluetooth to a computer. A Python script read both serial streams, timestamped everything, and wrote it to a .log file. Claude Code, running as an autonomous agent, would read those logs, compare my device’s phase labels to the reference, identify the discrepancies, and then modify the firmware parameters to close the gap. Then we would run another set of reps, collect new logs, and repeat.

Automatic. Iterative. Self-improving.

I gave it a clear objective: make the phase labels from my firmware match the reference sensor. I gave it access to the full codebase. I told it to keep going until the output converged.

Then I started lifting.

---

Forty iterations

The first few cycles were encouraging. The agent read the logs, found genuine mismatches, proposed parameter changes, and the numbers got slightly better. It felt like it was working.

Around iteration ten, things started getting strange. The agent kept finding discrepancies, but not always the same kind. Some reps it flagged the eccentric boundary as drifting early. Other reps the concentric looked too short. It would fix one, and something else would shift. The fixes were individually plausible but collectively they were steering the firmware somewhere odd.

By iteration twenty, I started to see what was happening. The problem was not the firmware. The problem was me, or rather, the data I was generating.

Forty sets of reps sounds like a lot of training data. But they were not forty identical sets. Some were fast, some were slow. Some deadlifts touched the floor; others I stopped just short. In a couple of sets I let the barbell rotate slightly on the way up. Normal variation, the kind that any experienced coach would read as “execution variability” and filter accordingly.

The AI could not filter it. To Claude Code, the log files were just numbers. A rep where the barbell touched the floor looked like a different signal than one where it did not, and the agent had no way to know whether that difference came from a firmware miscalibration or from the fact that I had done the rep differently. So it kept trying to make the firmware explain variations it was never meant to explain. With each iteration, the parameter changes got more aggressive. The filters got tighter. The thresholds shifted. The logic that had been reasonably correct at the start was being overwritten, piece by piece, to account for noise that had nothing to do with the firmware.

After forty iterations, the result was meaningfully worse than where I had started. The AI had not converged on a solution. It had overfit to a dataset that was too noisy to learn from.

I killed the loop and took back control.

---

The fix

I looked at the logs myself. Not all forty sets, just a handful of clean, consistent reps where I knew the execution had been tight. The problem was clear within a few minutes: the AI had replaced a simple, robust phase-transition logic with something much more complex that tried to account for every variation in the data. The new logic was technically sophisticated. It was also wrong in a way that mattered.

I rewrote it, not by reverting to the original, but by thinking through what the signal actually looks like during a clean rep and designing the logic around that. Simpler thresholds, a different approach to filtering, a cleaner state machine. It worked.

Twenty minutes, maybe thirty. After four hours of automated iterations.

---

What I actually learned

The experiment failed for a specific, recoverable reason: the input data was not controlled enough to act as a reliable ground truth. That is not a critique of the AI, it is a property of the problem. If I had collected data from a fixture-mounted bar with a metronome-paced protocol, the variation would have been small enough for an autonomous loop to converge. The fault was in how I designed the experiment, not in the agent itself.

But the failure surfaced something more interesting than a methodology mistake.

A senior developer, looking at those logs, would have stopped after five iterations and said: the reps are not consistent enough, we need to control the input before we can tune the parameters. That observation does not require deep technical knowledge. It requires the kind of everyday reasoning that tells you not to calibrate a scale while the wind is blowing.

The AI never made that observation. It just kept going, because I had told it to keep going until it converged, and it had no way to recognize that convergence was impossible under those conditions. It had every technical skill it needed and none of the judgment to know when to stop.

That is the line I keep coming back to: AI today has the technical capabilities of a senior developer, but not the logical instincts of a normal person.

The lesson is not that autonomous AI loops are useless. The lesson is that they work when the objective is precisely defined, the inputs are clean and controlled, and the feedback signal genuinely reflects the thing you are trying to optimize. Remove any of those three and the loop will optimize for something other than what you want, efficiently, tirelessly, and with complete confidence.

I will try this again!