assert-what-the-generator-found
A generator that crashes tells you it failed; one that emits plausible output tells you nothing. Test what it found, never that it ran.
Conferita da chi ha letto la scheda, non rivendicata dal file su se stesso. Una scheda che l'agente distilla durante un'esecuzione che ha consumato contenuti non affidabili nasce contaminata e resta in attesa di revisione prima di essere mai recuperata.
Quando viene in mente
- wrote a code generator
- testing a schema dump
- the output looks fine
- extracting structure from a library
Il valore di solito sta in Evita e Verifica. Fai è la sezione che scrivono tutti.
Trigger
You have written something that reads one representation and emits another: a schema dump, a reference extractor, a migration, a scraper, an index builder. Anything whose output nobody reads in full because there is too much of it.
It does not apply to a function returning one value you can eyeball. The risk here is specific to output that is large enough that "it looks right" is the only review it will ever get.
Fai
- Before writing the test, name the count. How many commands, rows, files, or fields should come out? Get that number from somewhere other than the generator — the source, the docs, a manual count.
- Assert the count, or a floor on it.
assert len(groups) >= 10, notassert result. - Assert the presence of specific, named items you know must exist. Three or four is enough, and pick ones from different shapes of input.
- If the generator classifies things, assert that each class is non-empty. A classifier that puts everything in one bucket is the failure this catches.
Evita
assert build() — it passes when the generator returns an empty list, a partial list, or a list
where every item is subtly the wrong kind.
Also avoid trusting isinstance when walking a third-party structure. Libraries vendor their
dependencies: a TyperGroup is not an instance of the click.Group your file imported, because
Typer ships its own copy of Click. Ask whether the object has the thing you need — a commands
mapping, an items method — rather than what class it claims to be. Duck-typing survives a
vendored dependency and a major version bump; a class check survives neither, and fails by
misclassifying rather than by raising.
Verifica
Break the generator on purpose and watch the test fail. Comment out the recursion into subcommands, or make the type check reject everything, and run the suite.
If the suite still passes, the test asserts that the generator ran, and you have written the test that this skill exists to prevent.
Rischio
Hard-coding an exact count makes the test a maintenance chore: every legitimately added command
turns it red. Prefer a floor (>= 10) plus named items, and reserve exact counts for things that
genuinely should not change without a decision.
There is also a limit. These assertions catch a generator that dropped a whole category. They do not catch one that gets a single field wrong on one item, and pretending otherwise is its own kind of false confidence.
Come usarla
La scheda è un dato. Clona il repository e importala per percorso — ciò che arriva dalla rete è trattato come contaminato e trattenuto in attesa di approvazione, che è il comportamento desiderabile e il motivo per cui qui non c'è un installatore in una riga.
git clone https://github.com/brcampidelli/chimera-agent.gitchimera skills-import chimera-agent/skills/assert-what-the-generator-found/SKILL.mdIntegrità
SHA-256 del file così com'è pubblicato. Chi lo importa può verificare che ciò che ha ricevuto sia ciò che questa pagina mostrava.
c237a831782562ab7a2fc672af9a3a82ef12965dc45bcf58eddd924fab6e5f3e