How I Built TestForge - Bootstrapping an E2E Test Suite from a Prompt

Every time I need to write end-to-end tests for a web app I didn’t build, the first day disappears the same way.

I open the site and click around trying to figure out what the workflow actually is. I inspect elements to find selectors that look stable. I ask around to discover which role can do what. I decide on a folder structure. I write a runner. I bolt on a screenshot pipeline. I glue in a report generator so QA has something to open. The tests themselves — the part that has any business value — come on day two.

The frustrating part is that the day-one work is nearly identical across projects. Different portal, same choreography.

TestForge is my attempt to collapse that day into a form.

The problem in concrete numbers

The trigger for building TestForge was a specific week: I had just finished an E2E test harness for one Joomla portal (an environmental-assessment workflow with 12 phases, 5 user roles, a Word report generator). It took roughly eight hours end-to-end.

Then the same client asked for the same treatment on a second portal — the marine-concession one, same CMS, same auth mechanism, largely the same reporting requirements. Different forms, different phase names, different business rules, but architecturally 90% identical.

I started copy-pasting.

Halfway through I stopped and did the math. If the next portal was going to take another eight hours mostly by copying the first, I was doing something wrong. The information the second run needed was already in three places: the source code of the first harness, my notes from exploring the second portal, and a specification document the client had emailed me.

The problem wasn’t that E2E test bootstrap is hard. It’s that no tool captures the structure of the work — recon, scaffold, phase-by-phase implementation, validation — and lets you feed a new project into it.

Claude Code could actually do all of this. If I wrote it a good enough prompt.

Why a prompt and not a scaffold

The tempting answer is: build a CLI that generates the harness. testforge init --stack laravel --port 3131 --target https://example.com/app and it emits a directory full of files.

I tried this shape mentally for a few hours and dropped it.

The problem is that the interesting part of the work is the recon — walking the target application, discovering which selectors are stable, mapping roles to landing pages, noting where the framework’s debug bar overlaps your content. A CLI can generate boilerplate. It can’t do recon. And boilerplate without recon produces tests that compile but never actually work — the classic page.click('#save-button') failing because the button is actually a submit input with a text label.

What I needed wasn’t a scaffolder. It was a well-structured brief for a coding agent that could do recon. Claude Code has bash, file editing, and MCP browser access. Given a good enough prompt with domain knowledge baked in, it can absolutely walk a site with Playwright, find selectors, propose a phase list, get user confirmation, and only then start writing code.

So TestForge doesn’t generate a test harness. It generates a prompt that generates a test harness.

That distinction turns out to matter a lot. It means TestForge itself is trivial to build (a form and a template renderer). All the complexity lives in the prompt template, which I can iterate on independently of any code.

The five inputs

I spent longer on the form than on anything else. The friction of filling it in is what determines whether the tool gets used or forgotten.

1. Target project. Name, base URL, tech stack (twenty options plus “auto-detect”), UI language (matters for text-based selectors — Save vs Salva), optional path to local source code.

2. Roles and users. Three tabs:

• Use existing users — one entry per account, with username, password, and a free-form notes field. The notes are what actually carry information: “admin — approves records and can see all provinces” is more useful than “admin” alone.
• Let AI create users — for cases where you have DB access but no existing accounts. Either the AI reads credentials from the source code path, or you provide DB connection details in the form.
• Other / paste raw — for messy inputs. A screenshot table copy-pasted from a spreadsheet, a chat message, whatever. The prompt instructs the AI to parse it, map users to roles, and use them.

The three tabs exist because the first version had only one, and I kept trying to shoehorn a pasted user table into a structured field. If your users mostly send you raw notes, giving them a “raw” bucket is respect for how they actually work.

3. Workflow to test. A textarea. Free-form. Describe end-to-end what happens: who submits what, who approves what, where the handoffs are. Plus a drop zone for reference documents (specs, PDFs, changelogs). Text files are read and embedded in the prompt; binary files are referenced by absolute path (because the browser sandbox doesn’t expose local paths for dropped files — you paste the path in an inline input next to each file).

4. Output. One field: where the AI should scaffold the harness. Everything else — port, report format, output directory, test mode — is deferred to the runner UI that gets generated, because those are runtime decisions, not scaffold-time decisions.

5. Advanced. Optional target agent (tunes tool hints) and free-form extra constraints.

That’s it. Five sections, most fields optional. The whole form fits on a laptop screen.

What goes in the generated prompt

The prompt template is where the actual thinking is. It has six sections:

Context — restates what the user gave, so the AI can’t misremember. Names the target, the directory, the language.

Deliverable — an explicit file tree of what the harness should contain: server.js, public/index.html, e2e/phases/*.js, helpers/, reporters/, docs/recon.md. Each file has a one-line description of its responsibility.

Architecture reference — the important part. This is where I encode the DNA of the working harness I built for the first portal. Not as code — as a description of patterns. What the Node server does (SSE broadcast, POST /api/run, POST /api/stop, single-active-job state). How the runner UI is laid out (two columns, terminal card, semantic coloring for stdout lines). How phase files are structured (async function receiving {page, users, report, screenshot, log}, must produce three screenshots, must never throw). A few small code snippets as scaffolding hints — not enough to copy, enough to anchor.

The AI reads this and knows what to build without me having to include the entire source of the reference harness. The description is ~2000 words. The alternative was including 3000 lines of JavaScript inline, which would have made the prompt unusable.

Target-specific specification — the stuff that changes per project. Stack-specific selector hints (twenty variants), roles and users (rendered differently depending on which tab was active), workflow description, reference documents (embedded content or absolute paths), extra constraints.

Pre-flight check — the section that saved the tool from itself. More on this in a moment.

Working sequence — eight numbered steps from recon to validation. Explicit about what each step must produce before moving to the next.

Final report — a schematic block the AI must print when done. Five bullet categories, no prose: What Was Built, Last Full Run, Unclear/Assumed, To Improve, Known Limitations.

The pre-flight check is the most important part

The first version of the prompt was strictly sequential: step 1 recon, step 2 scaffold, step 3 helpers, and so on. The AI would start step 1, discover it was missing something (the path to a PDF, credentials for a role, whether a specific workflow step existed), and then had to decide: ask now, or push through and hope.

What happened in practice was ugly. It would push through, get halfway into scaffolding, then get stuck, then ask three separate questions across three separate turns, then have to unwind assumptions it had already committed to files.

I added a pre-flight check block before step 1. It’s a six-item checklist: users mapped to roles, passwords available, reference document paths, workflow specificity, DB access, URL reachability. The rule is: if any item is unclear, stop and ask in one consolidated message. Wait for answers. Then start.

The block is not an extra step in the sequence. It’s a gate before the sequence starts. That distinction matters — I don’t want an AI that adds a “step 0 — think about the problem” ceremony to every task. I want an AI that surfaces every question up front, once, so the human replies once, and then the actual work starts.

In testing, this cut the first-hour interaction from “AI does stuff, then asks, then does more stuff, then asks” down to “AI asks four questions in one message, human answers, AI works uninterrupted for the next 45 minutes.” The wall-clock time dropped noticeably.

Why the final report is schematic

The other change that came from testing was reformatting the wrap-up.

Early runs would end with the AI saying “I’ve finished building the test harness. All eight phases are implemented and passing. The runner UI is at localhost:3131. Let me know if you’d like me to make any adjustments.” Which sounds fine, but is useless. What phases? Passing against what? Any selectors flaky? Anything ambiguous in the source docs I gave it?

The current prompt requires the completion message to follow a fixed schema:

▸ WHAT WAS BUILT       (dir, phases with names, roles, reporters, runner URL)
▸ LAST FULL RUN        (result, pass/fail counts, per-phase, artifacts)
▸ UNCLEAR / ASSUMED    (what it had to guess or was ambiguous)
▸ TO IMPROVE           (concrete asks — what to give it next)
▸ KNOWN LIMITATIONS    (flaky selectors, external systems unreachable)

The schema forces the AI to enumerate exactly what it built, exactly what worked in the last run, and — the two most valuable categories — what it had to guess and what it would need to make the tests more robust. Those two lists tell you where to iterate next. Without them, you’re re-reading the whole conversation trying to reconstruct what still needs attention.

I’ll admit I was skeptical that forcing an output format in a prompt would actually work reliably. It does. Claude Code and Codex both respect the schema when it’s stated this explicitly.

What TestForge is not

It’s not a test generator. It doesn’t produce Playwright code. It doesn’t have opinions about testing strategy beyond the ones baked into the prompt.

It’s not a replacement for writing tests thoughtfully. A generated harness can be structurally correct and still test the wrong things — you need someone who understands the target’s actual business rules to review the phase list before signing off.

It’s not portable across all app shapes. The prompt is opinionated toward server-rendered web apps with role-based workflows: government portals, admin dashboards, internal tools. For a public marketing site with three routes it’s overkill. For a mobile-first React Native app it’s the wrong tool entirely.

Being clear about what it doesn’t do is important. The point of TestForge is to compress the boilerplate part of E2E setup. It doesn’t compress the domain expertise part, and pretending otherwise would produce worse tests, not better ones.

Technical architecture

TestForge itself is one HTML file, around 1600 lines. No build step, no npm install, no server. Open it in a browser and it works.

State: a single state object holds the users array, active tab, and uploaded file list. Everything is JSON-serializable.

File upload: FileReader.readAsText for text files under 100KB (embedded in the prompt); binary files are added to a list with an inline input for the absolute local path (because browsers don’t expose paths for security reasons, and the AI needs the path to open the file).

Prompt building: one buildPrompt() function, roughly 400 lines. Reads form fields, picks conditional branches based on which tab is active and which fields are filled, concatenates a series of template strings. No template engine.

Output: the generated prompt is dumped into a monospace textarea. Three buttons: copy to clipboard, download as .md, reset.

Design system: the same palette as machina.chat — dark background #07080e, teal accent #00c8e0, purple accent #7c3aed, Inter for UI, JetBrains Mono for the prompt output.

What I’d change

A few things I noticed after using it a few times:

• Soft language detection. If the URL contains .it or lang=it, the language dropdown should hint at Italian without forcing it. Currently it defaults to Italian, which is fine for me but wrong for anyone else.
• Stack fingerprint. URLs ending in index.php are almost certainly Joomla or WordPress. A soft hint under the stack dropdown would save the recon step some time.
• Discrepancy warnings at generate time. If the “raw notes” tab is active but empty while the auth notes field has a tabular block, the tool should offer to move it. Small nudge, not a block.

None of these are blocking. They’re the kind of thing that improves the second version.

Composability

TestForge fits into the rest of Machina the way I hoped it would when I started thinking about the tool suite as a suite:

• BugCapture records a bug in the target app with screenshots and voice narration. The resulting .md becomes the workflow description you paste into TestForge.
• ContextForge pulls the git diff and recent server logs from the target app’s source directory. The output becomes reference material.
• PromptBoard lets you sketch the workflow visually before describing it in the TestForge form.

A realistic pipeline: user shows me a bug in an unfamiliar portal → BugCapture records it → PromptBoard structures what I learned → TestForge generates the harness prompt → Claude Code scaffolds the harness and adds a regression test for the specific bug.

That’s the shape of the tool suite. Small individual tools, but the value compounds when they compose.

What’s next

The obvious extension is a companion tool that reads a generated harness’s docs/recon.md and produces the next-iteration TestForge inputs automatically — closing the loop so the second run on the same portal is nearly free.

The less obvious extension is publishing the AIUnitTest reference harness as a standalone repo. Right now its DNA is described in the prompt in prose. It works, but a template repo would give the generated code a more concrete anchor. I want to test more portals with the current setup first, to see if the prose-only anchor holds up.

For now, TestForge does one thing: takes a description of an app you want to test end-to-end and produces a prompt that produces a test harness. If you’ve been staring at a portal wondering where to start, try it — the whole process from empty form to running tests is comfortably under an hour.

TestForge is part of Machina — a free, open-source suite of AI developer tools.
→ Try TestForge · View on GitHub