Engineering
How I engineer
The decisions I made, the bugs I traced to their root cause, how I test, and how I keep AI coding agents honest. Every claim links to the code, so you can check it instead of taking my word for it.
Test strategy
I start from what would hurt if it broke, and put the cheapest reliable check in front of it. A rule that can be code is code; a judgment that needs a model gets an eval gate with an answer key. Tests run offline and do not depend on today's date, so a red build means a real problem.
| Project | Tests | What they cover | Gate in CI |
|---|---|---|---|
| Market Terminal | 430+ | Unit, Flask route tests, an offline end-to-end run, replay from stored responses. Any network call in a test raises NetworkBlocked. | The whole suite, then again with the clock 400 days ahead. A daily run catches dependency drift. |
| ReqTrace | 36 | Parsing, deterministic checks, the citation guardrail, and tests proving the gate fails on a missed defect and on a false positive. | Eval gate: fails below 5 of 5 planted defects or above 0 false positives. Releases re-run it. |
| Take the Wheel | 16 (67 guard cases) | The security guard, the Bedrock backend and telemetry. A live end-to-end run checks the guard against Haiku, Sonnet and Opus. | Guard tests on Node 20, 22 and 24. |
| vlad.yt | 26 | 13 Playwright tests on desktop and phone: no console errors, axe WCAG A/AA, a no-WebGL fallback, reduced motion, structured data. | Deploys only if every test passes and the committed bundle equals a fresh build, then smoke-tests the live site. |
Design decisions
Each one names what I rejected and why. ReqTrace keeps a full decision log.
Take the Wheel: a browser agent
- The security boundary is a deterministic hook, not the tool allowlist. Claude Code auto-approves read-only commands before it checks
--allowedTools, and a wildcard rule let through the user's GitHub CLI and file uploads.guard.mjsruns before every command and fails closed. - Safety lives in code because the model is not consistent. In testing, Haiku tried the forbidden commands, and Opus sometimes refused even an authorized test. A check that depends on the model behaving is not a check.
- Submitting a form is locked in code. The final submit is refused unless the user's own task asks for it.
- Each run starts in an empty sandbox with no connectors or skills, so no stray instructions leak in and every turn costs less.
ReqTrace: AI for regulated engineering
- Anything that can be a rule is a rule. The model gets a closed four-type taxonomy enforced by the schema, and deterministic checks handle traceability.
- Reused test steps are copied verbatim, never paraphrased by the model. In a controlled document, a paraphrase is an unapproved change.
- The citation guardrail drops and counts, never repairs. A finding that cites an unknown requirement is removed and reported, not quietly fixed.
- A cheaper model earns a step only by passing the same gate. Three of four local-model routings failed and were rejected or redesigned. The one that passed cut model cost 55%.
- Reproducibility comes from recorded fixtures, because temperature 0 is not available. The fixture key includes the prompt, so a prompt change cannot replay a stale answer.
Market Terminal: a data pipeline
- Missing is not zero. A failed source is shown as missing, with the reason. No fallback constants, no carried-forward prices.
- One immutable snapshot per run; renderers never fetch. Database triggers block updates and deletes of snapshots.
- Every network call goes through one gate that bounds, deduplicates, logs and stores it, and makes no calls in offline runs.
- Known trade-offs are written down in known issues, not hidden.
Bugs I root-caused
The fix is half the job. The other half is the test or guard that stops it from coming back.
| Symptom | Root cause | What stops it returning |
|---|---|---|
An agent restricted to one CLI still ran pwd | Read-only commands are auto-approved before the allowlist is checked, and the wildcard also admitted the user's GitHub login, file upload and blocked sites | A fail-closed hook with 67 test cases in CI, plus before-and-after recordings |
| The agent reported 25 of 25 done; 23 were | The way it read the page let it lose track while scrolling | A page digest and a chained check; it now verifies 25 of 25 in 4 turns instead of 29 |
| A price premium chart ranged from −2% to +39% | 69 of 325 rows used a frozen futures price | Each price paired with the same run's benchmark, with dedicated tests |
| CI failed with no code change | A test depended on today's date | Injected clock, and CI re-runs every test 400 days ahead, which found three more |
| The test suite was calling live APIs | A test replaced one fetcher but not another | Network access in tests raises an error that wrappers cannot swallow |
| Every Yahoo symbol failed on the live server | A test run had cached an empty cookie in the shared cache | Tests use their own cache |
| A prompt edit could replay a stale model answer | The fixture key hashed the input but not the prompt | The key includes the prompt; mock mode fails on a stale fixture |
| Intermittent false positives in the eval | The reviewer was right: my synthetic spec had two real gaps | Fixed the input, left the answer key alone, and disclosed it; 5 of 5 with 0 false positives on every run since |
Working with coding agents
I use Claude Code and other agents to build faster, and I treat their output the way I treat any system under test: it has to prove itself. The root causes above are what that looks like in practice.
- One rulebook, enforced by CI. Market Terminal's
AGENTS.mdsets ten hard rules for every agent: never send or publish anything, missing is not zero, tests stay offline. The rules a machine can check are checked in CI, so an agent that breaks one fails the build. - Verify, don't accept. I re-run the tests and the linter myself after delegated work, and the decision log records it.
- Humans approve. Every ReqTrace output is stamped Draft, requires engineer review. Not for execution.
- Cost is measured, not guessed. Every eval run lands in a ledger with its model, cost and result.
Leading quality
- U.S. Bank: owned scope, timelines, defect management and go/no-go input for several mortgage initiatives, and mentored and onboarded new QA testers.
- Bausch + Lomb: audited legacy firmware protocols, retired obsolete requirements and removed dozens of invalid test cases before they moved into IBM Rational RequisitePro.
- Centene: built UAT and end-to-end automation from scratch in Ranorex and Cypress, and helped establish a CI/CD pipeline.
- Answers.com: wrote the test plan and built a Cypress suite from scratch, cutting manual regression effort by about half.
- Emerson: ran exploratory hardware and software testing on the Sensi thermostat before its public launch.
Code from before AI tools
I was writing software long before coding assistants existed. These are early projects I built while teaching myself to program.
| Project | What it is | Written |
|---|---|---|
| search-engine-php | A complete search engine in PHP | 2013–2015 |
| search-crawler-php-python | Its crawler, which ran on a jailbroken iPod and indexed about 4,000 sites a night | 2013–2015 |
| snapchat-client-php | A Snapchat web client built on the old private API | 2014–2015 |
| pokes-social-network-php | A small social network | 2016 |
| android-notes | A Pastebin-style Android app in Kotlin | 2017 |
Check any of it
Pick any row on this page and I'll walk you through the code, the decision and what I'd do differently.