← Back to Daily Builds

Daily build note · May 8, 2026

QA Watchtower

QA Watchtower is a local browser QA supervisor for Codex-built web apps.

Browser Qa Devtool Build note published Public demo coming soon

What shipped

QA Watchtower is a focused Node.js CLI that runs Playwright acceptance flows against a local web app and writes repair-ready artifacts. The implementation includes:

  • qa-watchtower init to create a starter watchtower.config.json.
  • qa-watchtower run to execute configured flows across desktop and mobile viewports.
  • Deterministic step support for goto, click, fill, expectText, expectVisible, and screenshot.
  • Console warning/error capture and uncaught page-error capture.
  • Timestamped Markdown and JSON artifacts under qa-runs/<timestamp>/.
  • A local demo app with one passing flow and one intentional form acknowledgement bug.
  • Smoke scripts for both the expected failing demo and the fixed-path demo.

Architecture

  • Node.js ESM keeps the CLI easy to inspect and run in web app repos.
  • Playwright is the only runtime dependency because browser isolation, screenshots, and viewport control are the core product value.
  • The CLI parser is hand-rolled to avoid pulling in a command framework for two commands.
  • Reports are generated from one normalized result object so Markdown and JSON stay aligned.
  • Each flow runs in a fresh page inside a viewport-specific browser context, which keeps state leakage low while still grouping results by viewport.

Trimmed scope

  • No hosted dashboard, accounts, billing, cloud storage, or external APIs.
  • No AI-generated test authoring.
  • No visual diffing or screenshot baselines.
  • No full Playwright DSL passthrough. The MVP supports only the step types needed for critical flow checks.
  • No HTML report yet; Markdown plus JSON are enough for local handoff.

Limitations

  • CSS selectors are the primary selector contract for click, fill, and expectVisible.
  • Console errors are reported but do not currently fail a run by themselves.
  • The runner stops a flow at the first failing step for that viewport.
  • Browser installation is still the user's responsibility through npx playwright install chromium.
  • Parallel execution is not implemented; flows run sequentially for clear reports and lower flake risk.

Suggested next steps

  • Add failOnConsoleError and failOnPageError config options.
  • Add an HTML report generated from result.json.
  • Add optional retries per flow with retry evidence in the report.
  • Support richer selectors such as role/name and label-based actions.
  • Add a doctor command that checks Node, Playwright, browser install, config validity, and URL reachability.
  • Let init optionally inspect a running app and propose starter flows without requiring a model provider.