Nobody in my seventh-grade class made a better mix CD than me.
Talk about a narrow market, but I dominated it.
In 1998, I could fill nearly all 80 minutes with the right songs in the right order. The opening track had to announce something. The fourth or fifth could get weird. The last song should feel ineluctable, even if nobody knew why. My classmates couldn’t get enough of the mixes.
Today, anyone can stream almost any song ever recorded. My CD-burning skills are useless. My teenage self would be awe-struck.
A disc full of good songs could still be a bad mix.
Filling the disc was easy. The obsession was hearing the shape of the whole thing: where the energy sagged, which two songs became better beside each other, when one more track would ruin it.
I just happened to learn that through a piece of software and a stack of blank CDs with a very clear expiration date.
I’ve watched the same thing happen in technical work.
Early at Apple, I built C++ Wi-Fi factory test stations, including LitePoint IQ integrations for our first handheld MIMO work. I got very good at knowing which command, timing quirk, or instrument setting made a calibration pass. Then the chipset changed. The station architecture changed. The product changed.
In 2016, I knew an Objective-C plotting framework called Core Plot well enough to patch imageOfLayer so exported charts respected Retina scale. That commit is still sitting on GitHub, patiently remembering the method name for me.
The exact mechanics faded with time, but the instincts kept earning their keep: establish a baseline, change one variable, inspect the failure, and know what a passing test actually proves.
The expiring skill is usually easier to name.
It fits in a course title or the skills section of a résumé. C++ test stations. Core Plot. Prompt engineering.
The skill that lasts tends to sound annoyingly vague: taste, judgment, problem framing. Knowing what good looks like before any tool produces something plausible.
ChatGPT arrived on November 30, 2022, two days after my son was born. Soon there were online classes, prompt libraries, tricks, hacks, and a brand new job title. People learned to phrase the request just right. Give the model a role. Add delimiters. Include examples. Tell it to think step by step. Add “YOU ARE AN EXPERT” in all caps when politeness fails.
For a while, getting good at prompting felt like learning a new programming language.
Now prompt engineering as a discrete job is starting to recede. Models reason longer. Agents read files, call tools, run tests, inspect their own work, and try again. The harness carries context that used to be crammed into one giant block of text. Goodbye copy/paste.
Prompts still matter, often a lot. Their shelf life is getting shorter.
An August NBER working paper put numbers around this on a narrow set of econometric coding tasks. Adding a worked example improved a basic chatbot’s score by 14.3 percentage points. It improved a constrained agent by 1.6 percentage points. Moving from the chatbot to the agent raised scores from 74.4% to 95.7%.
The study covers well-specified coding tasks, and the result did not hold equally across every model. Treat it as a signal more than an obituary. As the system gains more ability to plan and execute, extra prompt craftsmanship buys less.
A few weeks ago, I spent nine hours tuning Attrove’s retrieval system. That night included a lot of prompt editing. I versioned the prompts, adjusted how searches were planned and merged, and reran the benchmark after every change.
The eval eventually showed that my “best” one scored zero. Oops.
The crashes were never scored. They made the token and latency averages look awesome. They were not.
I had spent the night getting better at steering a system before checking whether it could still reach its own data. The prompt work was competent. The evaluator did its job. My interpretation of the scoreboard was simply wrong.
I wrote about that night because it reminded me how easily mechanical fluency can impersonate progress. You can become excellent at adjusting the knobs before noticing that the machine is unplugged.
Anthropic found an adjacent pattern in a privacy-preserving analysis of roughly 400,000 Claude Code sessions. People made about 70% of the planning decisions. Claude made about 80% of the execution decisions. Their definition of planning included deciding what counts as done.
Task-specific expertise tracked success far more closely than what someone did for a living. Every one of the ten largest occupations in the data landed within seven points of software engineers. But sessions rated novice succeeded about 15% of the time, while intermediate and expert sessions landed between 28 and 33%.
The accountant who understands the reconciliation rules can direct an agent extremely well. The senior engineer asking a first Rust question is still a beginner at that task. Tough to know what good looks like.
The study is observational, so hold it loosely. The direction still matches what I see. Humans increasingly own the outcome, the constraints, and the standard. Agents increasingly own the path through the work.
That changes where the craft lives. A useful definition of done now matters more than a clever synonym for “thorough.” Before the agent starts, decide what must exist when it stops. Which sources count? What would make you reject the result? Which failure modes deserve an explicit check? What evidence can the agent produce on its own?
No degree of prompt engineering can rescue a goal you have not understood.
This used to sound like project management. It is becoming how one person delegates to software.
Every new tool still deserves a period of unhealthy fascination.
You have to play with it, discover its shape, develop a feel for what good looks like, and learn the failures it produces with a straight face. Short-lived mechanics are often how durable judgment gets trained.
Prompt engineering taught me how much ambiguity I had been leaving inside ordinary instructions. The lesson outlived the technique.
With music, I called that taste. With AI systems, I call it an eval.
An eval is a way to write down what your eye, ear, or experience notices before you can fully explain it. It gives you a baseline, a set of examples, and somewhere specific to argue when two people disagree. Taste stays subjective. The file makes the disagreement inspectable.
That is what I want to carry forward as models and interfaces keep changing. Get close enough to understand the current tool. Learn its weird edges. Become fluent. Then ask which ability you are actually training.
You have to get close enough to a tool to be fooled by it. Then remember when it fools you.
I do not regret learning skills with a short shelf life. Mastery gets you close enough to see where the rough edges are. The mistake is expecting the mechanics to carry forever.
The CD burner disappeared. The urge to sequence things did not.
The test station changed. The measurement reflex stayed.
The overly detailed prompt is giving way to a goal, a harness, an agent, and an eval. I’m trying to spend less time copy/pasting the prompt directions and more time deciding what a good outcome would prove.
I can still make a good playlist. Nobody needs me to burn it onto a CD.
What were you good at a decade ago that nobody needs anymore? And what did it turn out you had actually learned?
The best voice models, now with full orchestration. Build real-time voice and chat agents on one low-latency stack: any LLM, your tools and knowledge, testing, Guardrails, and omnichannel deployment.
