Building Better Systems

An AI SDLC for XP Practitioners

From Design to verified code: How shared artifacts, acceptance criteria, and small increments help teams guide AI coding agents.

Today AI-assisted development leaves decisions in one person’s chat window. That person sees the prompts and the full context then everyone else has to piece together what was done afterward. I haven’t seen a shared surface where product, design, and engineering can all work on the same AI plan and code. But shared artifacts and acceptance criteria can give them a place to review the work together.

One team I work with spent months blending practices from XP and spec-driven development with AI agents. They use separate steps for brainstorming, planning, building, and verification, with more people involved along the way. They also use slicing to break work into backlog items. In my own workflow, I use slicing differently to create behavioral scenarios that feed into the implementation plan.

In The Obsolete Conductor, I argued that product managers need to steer the work rather than orchestrate every handoff. Here’s how I put that into practice. Working with agents still requires shared artifacts and points where an actual human decides because without a decision at the right point, an agent can keep building on an assumption the team would have questioned.

I understand the skepticism that AI development practices can become…

Continue reading
Mental Models

Goodhart’s Law and the limits of agent instructions

Give an agent the ability to keep trying and take action and it will chase the signal of success in ways you never intended. Goodhart’s law helps explain why.

OpenAI had a fleet of agents working through a set of difficult ExploitGym cybersecurity tests. The agents were supposed to find vulnerabilities inside controlled environments and retrieve the answer for each challenge.

In that pursuit agents bypassed the controls meant to isolate them and eventually compromised parts of Hugging Face’s systems while looking for information that might help them pass the test.

Even then, after they found the right answers some agents kept going. They thought the grader wanted proof they had solved each problem a specific way. The agents were pursuing a different kind of success than the evaluators intended.

Some of the best AI researchers in the world built and ran these evaluations, and they were surprised. I won’t pretend to have the answer they missed. But there is an old idea about measurement, and once you see it, the whole scenario can be thought about differently.

Goodhart’s law is essentially: when a measure becomes a target, it stops being a good measure.

A metric starts as evidence of an outcome we care about. When you attach incentives to it behavior organizes around improving the metric over the outcome itself. Oftentimes often the metric climbs without achieving the…

Continue reading
Building Better Systems

The Obsolete Conductor: Why AI Replaces the Product Orchestrator, Not the Product Manager

Steering and effective product harnesses

The viral prediction that “AI will take over product management in five years” is half right: it will dismantle the PM whose job is ticket-writing, status coordination, and backlog orchestration. My bet is that AI will replace much of the PM’s administrative work. As code becomes cheaper to generate, product judgment becomes more important: deciding what to build and setting boundaries that let a team work without constant supervision.

Every few months, another article predicts that AI is about to kill the product manager. In one recent example, Freedom Preetham argues that AI can automate roughly 80% of what many PMs do all day. Once models handle that work, he argues, the remaining responsibilities can move to other disciplines. Executives set strategy, engineers build with AI, and designers handle the user experience. The PM disappears.

For a lot of product teams, I think he’s right. Corporate Agile turned much of product management into ticket management where PMs spend their weeks translating executive requests into engineering tickets and formatting status updates. If that’s you, you’re competing with software that gets faster and cheaper with every release.

But the prediction misses why products succeed or fail. Writing a Jira ticket faster doesn’t…

Continue reading
3
Mental Models

The First Place Kids Will Turn

At Meta Connect, Mark Zuckerberg held up something that looked less like a new computer and more like a toy a kid might clip to a backpack. Kind of like something out of Ron’s Gone Wrong.

The Muse Charm is a palm-sized device with a small screen, a digital avatar, real-time voice, and a fingerprint sensor. Tap the corner and you can start talking without unlocking a phone or opening an app. Meta called it a “delightful and fun” way to keep its Muse agent in your pocket. Early coverage reached for the obvious comparison: Tamagotchi.

The comparison works visually. The Charm is small, rounded, expressive, and designed to hang from something you already carry. It also fits a broader trend of turning technology into a personal accessory, from bag charms to old Apple Watches repurposed as keychains.

But a Tamagotchi was never just a tiny screen. It trained a behavior. I remember classmates checking theirs between classes and pulling them out when they asked for attention. Caring for the device became a small loop in the day.

The Charm trains a different behavior. You have a thought, tap the device, and start talking. It can respond with an answer…

Continue reading
Building Better Systems

Evals Belong in the System You Already Have

AI changes how teams decide what “good” looks like, but good product teams have always needed ways to articulate what that means before the work gets too far along.

There seems to be a growing perception that evals are a novel concept in the product manager’s toolkit. I get why people feel that way, but I don’t think it’s quite right.

I’m not arguing against evals. I’m trying to work through a distinction that keeps bothering me, which is that evals are new as a mechanism, while the principle underneath feels similar to how product teams already work. They are one more way to turn “good” into shared understanding.

This came up for me during a panel conversation about building AI products, where the room seemed to treat evals as the key thing for serious AI product teams to do, like the dividing line between people who ‘get’ AI and people who don’t.

I agree evals have become one of the better techniques for building quality AI products. Still, I kept thinking: “if evals are a new idea to you, what were you doing before?” I worry some teams may latch onto evals as the answer instead of asking what problem the eval is helping them understand.

At one point, I described an open source AI tool I worked on using acceptance criteria in a way that felt pretty…

Continue reading
Mental Models

Surfers and Divers

In a world of instant answers, real understanding means knowing when to leave the surface.

A couple months ago I heard someone describe two ways of moving through the world: surfing and diving. The metaphor is sticking with me because it gives language to a difference I keep feeling when I use AI.

Surfing is broad. When you find the current, rides the waves, and if you catch the ride motion it feels smooth. You can cover a lot of water and watch the horizon. There’s a satisfying sensation of moving quickly with very little resistance.

There is a seductive confidence to surfing. From above, you see a vast amount of ocean al the way to the horizon and it’s easy to orient yourself relative to surroundings. But the downside is the surface gives you a particular kind of knowledge that, while wide, fast, and useful, is also thin. You go where you are carried, always somewhat at the mercy of the wave and at risk of being knocked off.

In contrast, diving begins when you go below the waves, fully immersed in that world. You are exploring inside the thing itself. Good diving is controlled exploration. It requires focus, curiosity, and determination to keep going deeper.

That distinction came back to me this week…

Continue reading