Constraint Generation

Constraint generation is the engine that makes data quality checks practical. Once you master it, you can automatically discover the rules that your data must obey — without writing hundreds of rules by hand — and then enforce those rules on every new batch of data. This unlocks three neighbouring topics in your library: Test-Driven Development for data (where constraints become your tests), Data Governance (where the discovered constraints are your documented standards), and Model Monitoring (where the same constraints catch drift in production). You'll also be able to build a Data Dictionary that writes itself, because the constraints you generate describe your fields' valid values and ranges. Every AI system, from a small phone app to a large language model, depends on data that obeys constraints — this is how you ensure that.

Imagine you are a new quality inspector at a factory that makes metal bolts. On your first day, you are handed a box of a thousand bolts and told: 'Figure out what a good bolt looks like.' You don't have a spec sheet. So you measure every bolt — its length, its diameter, its weight — and you record the numbers. After measuring all thousand, you notice that the lengths are all between 10.0 mm and 10.2 mm, the diameters are all between 5.0 mm and 5.1 mm, and the weights never go below 3.5 g or above 3.8 g. From those observations, you write a rule: 'A bolt is good if its length is in [10.0, 10.2], its diameter is in [5.0, 5.1], and its weight is in [3.5, 3.8].' From now on, any new bolt that falls outside those ranges is rejected. You didn't invent the rules out of thin air — you discovered them from the data. That is constraint generation: looking at existing data, noticing the patterns and limits, and turning them into rules that future data must satisfy. The key insight is that you don't need a human to write the rules; the data itself tells you what is 'normal'. But — and this is the crucial part — you need to review the rules before you enforce them, because the data you observed might be a narrow sample that doesn't show the full range of what is acceptable.

Think of a teacher grading a new batch of essays. The teacher has never seen this class before, so they read the first ten essays to get a feel for the level — what kind of vocabulary, sentence length, and structure is typical. From those ten essays, the teacher extracts a mental rubric: 'Essays are between 500 and 800 words, use formal language, and have at least three paragraphs.' Now the teacher can grade the remaining essays quickly by checking them against this rubric. If an essay comes in at 300 words, it gets flagged. But is it a bad essay? Maybe the assignment explicitly asked for a short essay. This is where the analogy breaks down: the teacher's rubric is based on a small sample, and the sample might not represent the full range of acceptable work. In constraint generation, the same danger exists — you might discover constraints that are too tight because your data only covered a slice of the real world. So the process is never fully automatic: you must review the discovered constraints with a human in the loop before you enforce them. The teacher can adjust the rubric after seeing the rest of the essays. Constraint generation works the same way: you discover, you review, you adjust, and then you enforce.

Constraint generation is the automatic discovery of rules (constraints) that existing data obeys, followed by human review and then enforcement of those rules on future data.

You have no prior topics in your library yet, so I'm building from zero. But this concept belongs to the Data Quality family. It is the mechanism that makes Data Validation Constraints practical — you don't write them by hand; you generate them. It also connects to Data Dictionaries because the constraints you generate effectively document the field definitions. In your library, you have a note: 'Constraints can be automatically discovered from data and then enforced as tests — 'constraint generation' is the mechanism that makes data TDD practical.' That means constraint generation is what turns data testing from a tedious manual chore into an automated, continuous process.