Prompt Iteration

This is the skill that turns prompt writing from guesswork into engineering. When you finish this page you will be able to take any prompt — a product description, a customer-support instruction, a code-generation request — and improve it systematically, the way a developer improves software: change one thing, test, measure, repeat. You will be able to say 'this version is better because...' with evidence, not just a feeling. This is the foundation for everything that comes next in your library: Prompt Optimization (letting a program find the best prompt for you), Prompt Registry and Versioning (storing prompts so you can roll back if a change breaks something), and Context Engineering (because a prompt is only as good as the context it sits in). Without this, those later topics are just jargon. With it, you can build reliable systems on top of language models.

Think of the last time you asked someone for a favour and they did the wrong thing. You probably didn't just repeat the request louder. You reworded it: 'No, I meant the blue one, not the red one.' You changed one detail, checked their response, and adjusted again. That is iteration. Prompt iteration is the same process applied to an AI model, but with discipline. You write a prompt, you test it on a set of inputs you know the right answers to, you measure how many it gets right, and then you change one thing — one word, one example, one formatting detail — and test again. The key insight is that you only change one thing at a time. If you change three things and the performance goes up, you don't know which change helped. If it goes down, you don't know which change hurt. By changing one variable, you can attribute the result to that change. This is the scientific method applied to prompting, and it is the difference between 'prompt tinkering' and 'prompt engineering'. Without measurement, you are just guessing. With measurement, you can build something reliable.

Imagine you are a chef developing a new recipe. You have a dish, and you want it to be perfect. You don't just throw random ingredients in and hope. You start with a baseline — maybe a standard recipe you found online. You cook it, you taste it, you ask your family to taste it, and you score it out of ten. Maybe it's a 6. Then you change one thing: you add a pinch of salt. You cook it again, taste it, score it. Now it's a 7. You change another thing: you reduce the cooking time by two minutes. Now it's a 8. You change one thing at a time, and each change is measured. If you had added salt and changed the cooking time at the same time, you wouldn't know which one made it better — maybe one helped and one hurt, and they cancelled out. The same logic applies to prompts. You have an evaluation set — a collection of inputs with known good outputs. You run your prompt on that set, score it, change one word, run it again, score it again. That is prompt iteration. The analogy breaks down when you consider that a recipe is a fixed set of steps, but a prompt is a set of instructions that can be interpreted in many ways. The model might respond differently to the same prompt on different days or different versions. That is why you need a fixed evaluation set — to measure against a constant baseline, not a moving target.

Prompt iteration is the process of refining a prompt through repeated cycles of change and measurement, where each cycle alters exactly one variable and evaluates the result against a fixed evaluation set.

You have not learned anything yet, so this page stands on its own. But you should know that this concept is the foundation for the rest of your prompt-engineering library. Your notes point to Prompt Optimization (which automates this cycle with a program) and Prompt Registry and Versioning (which stores the results of each cycle so you can track changes). This page gives you the manual version — the skill you need before you can trust the automated one. The library also mentions that 'without an eval set, iteration is just churn' — that idea is the heart of this page. You will also see connections to RAG (Retrieval-Augmented Generation) later, because the context you retrieve changes what your prompt means — but that is a later topic. For now, focus on the loop: change, measure, repeat.