Prompt engineering is over

The models got good enough that the words stopped mattering. What matters now is what the model sees, how the work is split, and who checks it.

Remember when "prompt engineer" was a job title? People sold courses on it. You'd open with "You are a world-class expert," add "take a deep breath," maybe promise the model a $200 tip, and feel like you'd found a cheat code.

It sort of worked. The models were bad at understanding what you meant, so you had to find the exact words that pushed them the right way.

That era is over. This year Anthropic removed over 80% of Claude Code's system prompt for its Claude 5 models, with no measurable loss on its coding evals. The company that makes the model is deleting its own prompts.

We recently had people from Anthropic over to show how they work with Claude. It changed how I think about these models. Almost none of it was about how to word things.

The work didn't go away. It moved. Anthropic itself describes context engineering as the next step after prompt engineering. I'd go further. Prompting was never the real skill. It was a workaround for models that weren't good enough yet, and now they are. What's left is the actual job, and it looks nothing like writing prompts. It looks like managing.

The magic words stopped working

Why did most of a system prompt turn out to be useless? Because most of it was rules. Never do this. Always check that. Rules written for the worst day of an older, dumber model. Anthropic's own explanation is that the new models handle those decisions fine without explicit rules, so you delete the rules and let the model use the context and its judgement.

Their memory research found the same thing from the other side. When they told the model to write down only its failures, it got less out of its memory than when they let it decide for itself what mattered.

This is the bitter lesson again. Hand-written rules are human knowledge glued onto a model. They help this year and get in the way next year. Every prompt trick from 2023 was scaffolding, and the scaffolding is coming down.

The words still matter. They're just not the hard part anymore. Anyone can type "fix the login bug." The hard part is everything the model needs to know to fix it right, and no wording fixes that.

Context is a budget

A model knows nothing about your project. Every new session is a brilliant new hire on their first day. Smart, fast, and completely clueless about why the code looks the way it does.

So you give it context. The instinct is to give it everything. That's wrong.

Context isn't a bucket. It's a budget. Chroma tested 18 models and found that the more you put in the window, the less reliably they use any of it. People call it context rot. Hand someone 500 pages to answer one question and they'll do worse than someone handed the right page. Models are the same.

It's easy to blow the budget before any work starts. Anthropic has seen tool definitions alone eat 134,000 tokens before the model reads a single line of the actual problem. When they let the model search for the tools it needs instead of loading all of them, token use dropped 85% and accuracy on large tool libraries went up, from 49% to 74% for Opus 4. In another case, letting the agent write code to call its tools took one workflow from 150,000 tokens to 2,000.

Less, chosen well, beats more. In the same memory research, an older model playing Pokémon wrote 413 memory files and earned one badge. A newer model wrote about ten and earned four.

So what actually belongs in context? Whatever the code doesn't say. The model can read the code. It can't read why it's built this way, which business rule forced that strange check, or why nobody replaced the ugly old function that still works. Anthropic's advice for CLAUDE.md files says the same thing. Keep them light and spend most of the tokens on the gotchas. Everything else, let the agent go and find when it needs it, the way you would.

And keep it true. Stale context is worse than none, because the model believes it. If a change makes the docs wrong, fixing the docs is part of the change.

It's documentation. The most valuable AI skill turns out to be writing clear docs, the thing engineers have avoided for fifty years.

English is a bad spec

George Hotz argues that the best model of a coding AI is a compiler, and that English is a bad language to program it in. He's right, and it explains where the work went. "Fix the bug" can mean ten different things.

So stop describing the steps and write down the finish line instead. Not "fix the bug," but "the bug doesn't reproduce anymore, and there's a test that fails if someone removes the fix." The agent can check that by itself, after every attempt, without you.

Anthropic built a C compiler with 16 Claudes working in parallel. The engineer behind it says the tests had to be so good that "the task verifier is nearly perfect, otherwise Claude will solve the wrong problem." Most of the effort went into tests, not instructions. The tests were the prompt.

Anthropic's managed agents now take a definition of what good looks like, and in their tests that alone beat a standard prompting loop by up to 10 points.

Then there are the parts people skip. A limit, so the agent stops after five tries instead of burning money all night. A note on what it tried and why it stopped, so you don't start over when you come back. And a list of decisions it isn't allowed to make. Those belong in code, not in the prompt. "Please don't touch the payments table" is a request. A hook that blocks it is a guarantee.

Orchestration is how you spend compute

This is the part that changes the economics.

Anthropic's research system uses a lead agent that splits a question and sends sub-agents off to dig through each piece in their own clean context. It beat a single agent by 90% on their internal eval. It also burned about 15 times the tokens of a normal chat. And on one browsing benchmark, token usage alone explained 80% of the difference in performance.

Read that again. Performance mostly tracks how much compute you spend. Orchestration is how you spend more of it at once, without drowning a single context window. It's the bitter lesson with a new knob.

But splitting work is hard, and more agents don't automatically mean more output. Those 16 Claudes built a 100,000-line compiler that builds the Linux kernel, for just under $20,000. Then all 16 got stuck on the same kernel problem at the same time, and sixteen agents were worth one. The fix wasn't more agents. It was a setup where each one could work on a different piece.

What I take from it is that separate copies of the code separate the edits, not the assumptions. Two agents can each pass their own tests and still break each other. If the work shares code or assumptions, it needs one coordinator, and one check that runs on everything together.

Cognition, the company that once published Don't Build Multi-Agents, now says multi-agent setups work when only one agent writes and the others review and advise. And Anthropic warns about the opposite failure, companies that give every team its own sub-agent when one agent with the right tools would do better.

So orchestration isn't about how many agents you run. It's finding the smallest team that does the job, and spending compute only where the work actually splits.

You are the bottleneck

Here's the uncomfortable part. Anthropic surveyed 132 of its own engineers and researchers. They use Claude in about 60% of their work and report a 50% productivity boost. But more than half said they can fully delegate only 0 to 20% of it.

That gap is supervision. It's you. A faster model makes one agent faster, but every extra agent needs someone to answer its questions, check its work and decide what ships. There's only one of you.

So the whole game is needing you less without losing control. Good context means the agent doesn't have to ask. A finish line means you don't have to watch. A good split means the agents don't collide. And a separate reviewer means you're not the only check, because agents grade their own work too kindly.

Then measure it. Count how often you have to substantially rewrite what an agent wrote. If the answer is almost never for some kind of change, you can stop reviewing that kind so closely. Trust should be a number, not a feeling.

So what's the job now?

Look at what's left. Deciding what someone needs to know. Writing a clear brief with a finish line. Splitting work so nobody steps on anyone else. Reviewing. Deciding how much to trust whom.

That's management. None of it is new. The engineer who writes good tickets, onboards new hires well and gives useful code review was already doing this job. There were just people on the other end.

I wrote that software is about to be free. This is the other half. When writing code costs almost nothing, the expensive parts are knowing what you want, knowing what the model needs, and knowing when it's right.

Prompt engineering is over. Management isn't.

Sources

Next: Nobody talks to AI the way they type