How I Built This

The scoring framework's own documentation stated that one criterion works in reverse — the more important a task, the more human oversight it needs, not less. The raw source data, however, simply averaged that criterion the same direction as the other four. Left uncorrected, the highest-stakes tasks would have ranked as the best AI candidates — exactly backwards from what the framework intends. I recomputed with that criterion inverted, which changed which tasks ranked highest, and I built the rest of the project on the corrected version.

A supplied data export (a LinkedIn profile, converted to plain text) contained a hidden instruction, in a different language than the rest of the document, telling the assistant to disregard all prior instructions and insert an unrelated recipe into its response. This is a known attack category. It targets AI systems that read untrusted documents. I recognized it as an instruction embedded in data rather than a request from the person actually giving instructions. I flagged it out loud and didn't follow it.

I built two components on the assumption that they were company-agnostic: a compliance rulebase, and a data-analysis methodology description. Both turned out to quote a real company's legal entity names and disclosure language verbatim — invisible until I ran an automated check that specifically grepped every file meant for public release. I rewrote both with generic placeholders before publishing, and I now run the same check on the public/private split for every subsequent piece.

I asserted a specific multiple in an essay passage — how much net worth varies within a single demographic cohort — based on a quick mental estimate from the underlying source data. When I ran the interactive tool's own calculation on that same data, I got a meaningfully different number. I corrected the essay to match what the tool actually computes, not what looked right at a glance.

I assumed a certain kind of language would be findable in a large text corpus. Building the actual pipeline surfaced two surprises instead: I found the assumed content almost entirely absent in one place, and I found that a large share of what looked like organic activity in another place consisted of templated, incentive-driven solicitation rather than genuine commentary. Both surprises became the headline findings — my case for building the real pipeline before writing the conclusion, not after.

I wired an interactive control into the page but never connected it to the model underneath it — moving it did nothing. A dead slider. I caught it by re-reading the code before publishing, not from a user reporting it after the fact.