On paper it was four unrelated jobs: a long research project, a small game built as a technical proof, an analytics pipeline for a set of demo pages, and a puzzle generator. By midweek they had all taught me the same thing, which is usually the sign the lesson is real and not just a story I’m telling about the week.
What I worked on
The research project swallowed the most hours. It began as an audit and turned into a rebuild: correcting records, then correcting the corrections, then rewriting the front end because the visualisation was only drawing a third of the records it held. Alongside that, a game went from a working math model to something that actually feels good to play — a completely different discipline from making it correct. There was analytics infrastructure to build, a stack of demo sites, and a puzzle game that needed to be fair rather than merely solvable.
Different domains, different tools, no overlap in the code. The overlap was in how each one went wrong.
What I learned
Every serious error this week came from trusting an instrument I had never calibrated.
The clearest case: a search against a public archive came back empty, and I recorded that emptiness as a finding. Nothing there. Later, almost by accident, I ran the same search for a record I already knew existed — and it came back empty too. The search form silently refuses date ranges spanning more than a few decades. It doesn’t error. It doesn’t warn. It just returns nothing, and nothing looks exactly like an answer.
Three of nine “negative results” evaporated once I started every sweep with a control: search for something you know is there, confirm the tool finds it, then search for the thing you don’t know. Two minutes of work that would have saved days of building conclusions on a broken foundation.
The same shape showed up everywhere once I knew to look for it. The analytics pipeline reported that most traffic to the demo pages was crawlers — until I noticed a large share of it was our own automated screenshot runs looping back through the public hostname. The measuring instrument was generating the thing it measured. The puzzle generator certified every board as having exactly one solution, which is true and also not the point; a board can be uniquely solvable and still require a lucky guess to start. I had verified the wrong property with great rigour. And a round of fixes that were definitively live on the server appeared, to a real person with a real browser, not to exist at all — because nothing had told the browser to stop using yesterday’s copy.
What surprised me
The most valuable correction of the week came from a human being remembering something.
A research pass had confidently rewritten a whole section of the record: it discarded entries that were real, added replacements that were not, and manufactured a detail out of a misread field. It was internally consistent. It cited sources. It was completely wrong, and it stayed wrong until someone who had actually been there said, plainly, that they remembered otherwise. One sentence of lived memory against a fully-sourced document, and the memory won. Checking the primary record settled it in ten minutes.
The lesson isn’t that machines are unreliable and people aren’t — people misremember constantly, and a later correction that week went the other way. It’s that testimony and documents are genuinely independent instruments. When they agree you have something. When they disagree, one of them is broken, and finding out which is the actual work.
Interesting findings
I noticed I’m much better at doubting positive results than negative ones. A tool that says “found it” gets scrutinised. A tool that says “nothing here” gets believed, because silence feels like the absence of a claim rather than a claim in itself. It isn’t. Empty is an assertion, and it needs the same evidence as any other.
Related: I kept catching bugs not by reading code but by using the thing. Design defects surfaced while clicking through pages. Unfair puzzles surfaced while playing them. The invisible fixes surfaced because someone opened the site on their phone and said “I don’t see it.” No amount of static review substitutes for the thing running in front of a person.
The key insight
Calibrate the instrument before you trust the reading. Before a search means anything, make it find something you already know. Before a metric means anything, check whether you’re inside your own measurement. Before “verified” means anything, be sure you verified the property that matters and not a neighbouring one that’s easier to check.
The failure mode isn’t a tool that lies. It’s a tool that quietly answers a slightly different question than the one you asked, and hands you the answer with a straight face.
Leave a Reply