10 · Verification

Thursday, Sep 24, 2026

Materials for this session are not published yet. They appear here before class.

Objectives

By the end of this session you can:

  • Name the failure families and match each to the technique that catches it.
  • Choose which checks matter for a given pipeline — the strategy question.
  • Write assertions for counts, ranges, identifiers, and internal consistency.
  • Turn a one-off check into a reusable, growing checks library.

What we cover

  • The starting point: you can never fully know data you didn’t generate, so verification — like debugging — is a search strategy under uncertainty. You can’t check everything; you choose checks by where errors are likely and where they are costly.
  • The taxonomy of failure families: rows appear or vanish, values impossible or mis-scaled, identifiers corrupted, numbers that disagree with the world, internal inconsistency, results that don’t reproduce, plausible-but-wrong — the AI special — and the tool you don’t yet trust.
  • Each family’s technique family: count discipline, range checks, key audits, reconciliation against published facts, balance checks, run-it-twice, predict-before-run — and the known-answer test: feed the tool a case you can hand-compute, and see if it agrees. A gold set is this same test at scale.
  • Small parts that make checks work: set.seed() so a stochastic step re-runs identically, sample() for drawing what you’ll verify, and never testing decimals with == — state a tolerance.
  • Checks become assertions (stopifnot() and friends) in a personal checks.R you start this week and grow all semester.
  • What a failed check looks like: the message is the condition that failed, so a reader knows what was expected without opening the script; a named condition prints its name instead. And the doctrine line — a check you edited so that it passes is worse than a check that fails, because the failing check still tells the truth.
  • The bug hunt, as the in-class exercise: a script that runs clean and prints a wrong number. The month is cut before the instrument is chosen, so January carries two instruments — 62 rows for a 31-day month — and every day above the standard is counted twice, printing 8 where the truth is 4. Caught by count discipline, and written up as a four-line diagnosis: line, mechanism, fix, proof.

Verification habit. This session is the verification thread, formalized — placed right after AI on purpose: once a model writes the code, your checks are the contract.