Flagship project · Phases 0–13 shipped · Project complete
AI-assisted software testing from exploration to automation.
A practical workflow for testers: explore software, identify user flows, design scenarios, generate BDD specs, produce deterministic Playwright automation, and ship quality reports — all from the terminal. Not a replacement for testers. An assistant for them.
Autonomous where safe. Human-governed where it matters.
$ ./bootstrap.sh && make install
$ /tc:init
$ /tc:review-requirements
$ /tc:learn-from-docs
$ /tc:create-charter --target "Sign-in flow"
$ /tc:explore --charter CH-001
$ /tc:test-ideas --session SESS-20260528-600
$ /tc:generate-bdd
$ /tc:automation-plan
$ /tc:automate
$ /tc:traceability-map// These commands take a tester from a cold local app through requirements, knowledge, exploration, BDD, and a generated Playwright suite. The agent does the heavy lifting; the tester stays in charge of every decision in between.
What it is
Test Commander is a workflow and framework concept for modern quality engineering. It treats AI as a structured assistant, not a magic test generator. The tester stays in control; the agent does the heavy lifting.
Why it exists
Manual testing knowledge stays trapped in people’s heads. Exploration produces insight but inconsistent documentation. Automation lags behind delivery. Reports are manual, inconsistent, or missing. Teams want to use AI but worry about reliability. Test Commander connects exploration, design, automation, and reporting so the work compounds instead of dispersing.
For manual testers
Think of Test Commander as an assistant that turns your testing instincts into organized artifacts. You decide what matters. The agent helps document, structure, and automate. Here is what that conversion looks like on a real workflow.
Exploring a shopping-cart flow, you notice friction the team has not catalogued.
Those observations become reusable artifacts in minutes, ready for review.
The workflow
Seven steps, designed to feed each other. Insight gathered in one cycle seeds the exploration in the next.
Phase 4 (shipped 2026-05-28). /tc:create-charter scopes a session against the project knowledge; /tc:explore classifies every recorded Playwright event into six universal observation types and six universal anomaly categories with a Charter-Coverage matrix; an internal exploration-review sub-mode routes gap signals to requirements/open-questions.md.
Phase 3 (shipped 2026-05-27). Five /tc:learn-from-* commands extract entities, terms, journeys, endpoints, modules, recorded responses, and test coverage into ten structured product-knowledge artifacts under .test-commander/product-knowledge/ with full path:line provenance. A shared synthesizer rebuilds system-model.md byte-deterministically at the end of every run.
Phase 2 (shipped 2026-05-27) ships the requirements layer; Phase 5 (shipped 2026-05-29) turns enriched test ideas into Gherkin. /tc:generate-bdd renders one scenario per enrichment candidate with machine-readable @req:/@cs:provenance, /tc:review-bdd runs a six-category universal rubric, and /tc:traceability-map rebuilds the requirement and scenario-level maps tying each requirement forward to the scenarios that exercise it.
Phase 6 (shipped 2026-05-29) is the project’s first executable artifacts. /tc:build-framework lazily scaffolds a Playwright + TypeScript framework; /tc:automation-plan scores every scenario against a seven-factor suitability rubric; /tc:automate generates page objects, fixtures, and specs with @req:/@cs: provenance; /tc:review-automation enforces quality; and /tc:generate-test-data keeps data in .test-commander/test-data/ rather than inline in code.
Phase 7 (shipped). /tc:run orchestrates suite execution and /tc:analyze-results triages failures; per-run records land in .test-commander/runs/; screenshots, traces, and logs route to .test-commander/evidence/ with the policy defined in config.yaml. The same workflow runs locally and in CI.
Phase 7 (shipped). /tc:report writes .test-commander/quality-report/current-quality-report.md and snapshots a copy to history/YYYY-MM-DD-HHmm.md. /tc:quality-gate evaluates release-readiness against project-defined thresholds. Facts, interpretation, and human-review items stay clearly separated.
Phase 8 (shipped). /tc:learn, /tc:learn-from-failures, /tc:learn-from-exploration, /tc:learn-from-feedback, /tc:review-lessons, and /tc:promote-lessons turn the workspace into a learning loop. Every promotion is visible in git diff — Test Commander never silently rewrites methodology.
The full roadmap
Test Commander was built phase by phase, each landing under strict test-driven discipline with its own annotated git tag (phase-0 … phase-13). The foundation (Phases 0–6) established the workspace and the exploration-to- automation pipeline; the later phases (7–13) added execution, reporting, learning, visuals, a governed web console, an API and MCP server, sandboxes, and a continuous quality agent. The whole roadmap is now complete.
What ships
Each tc-* skill is owned in-repo (Decision D1 — no community-skill dependencies). The commands route to bundled Python helpers; the workspace lives at .test-commander/ inside the consuming project and is committed to git like any other source artifact. Three skills (tc-evidence, tc-governance, tc-mcp) ship runtime rather than /tc:* commands. The full per-command reference lives on the documentation page.
Workspace orchestration. Initialize, inspect, journal, recommend.
Requirements quality. 16-dimension rubric, INVEST review, AC review, coverage, seed test-ideas.
Project knowledge ingestion. Five helpers extract structured artifacts from documents, specs, code, recorded API traffic, and existing tests.
Charter-based exploratory testing. Scope a session, replay a recorded Playwright run, synthesize the summary, enrich the Phase-2 test-idea seeds.
BDD generation and review. Render Gherkin from enriched test ideas with @req:/@cs: provenance; run a six-category universal rubric.
The cross-cutting map. Rebuild the requirement and scenario-level traceability chains; downstream links resolve as phases populate them.
The lazy framework. Scaffold the project-root tests/ tree, playwright.config.ts, and package.json only when automation first needs them (D8).
The strategic gate. Score every scenario against a seven-factor suitability rubric and rank each automate / consider / manual.
Generation and review. Render page objects, fixtures, and specs with provenance and fixture-mediated data; mechanically review the result.
The data discipline. Populate test-data/ seed JSON and a per-area spec so nothing is inlined in test code (D6).
Execution and triage. Orchestrate the suite, capture per-run records, and classify failures without weakening assertions.
The quality report. Write the current report with committed history and evaluate release-readiness against project thresholds.
The evidence indexer. Route screenshots, traces, and logs into .test-commander/evidence/ per the config policy. Runtime; no /tc:* commands.
The governed learning loop. Capture lessons from failures, exploration, and feedback; review and promote them in visible git diffs.
Visual documentation. Eight diagram types, infographics, and a deterministic renderer turn the workspace into Mermaid sources and rendered assets.
The read-only web console. A team-facing viewer over the committed workspace — dashboard, journal, BDD, runs, evidence — that never invents data.
The controlled-execution pipeline. Intent → plan → policy → approval → bounded execution → validation → audit. Default deny; the single path every action takes. Runtime; no /tc:* commands.
Runtime API + MCP server. Alternative front-ends that drive Test Commander over HTTP and the Model Context Protocol — through the same pipeline. Runtime; no /tc:* commands.
Sandboxed environments. Launch an on-demand, team-accessible Test Commander environment from GitHub Actions, governed and safe-by-default.
The continuous quality agent. Watch changes, map impact, find coverage gaps, propose tests, and open labeled PRs — gated by five autonomy modes.
User-guide walkthroughs
Every shipped phase ships its own end-to-end walkthrough under docs/user-guide/ in the test-commander repo. Each one drives the seeded fixture end to end with verbatim sample output so a reader can reproduce the result in a tmp workspace.
Phase 1
From clone to /tc:next: init the workspace, edit project metadata, append a journal entry, ask what to do next.
Phase 2
Upload requirements.md, run the rubric pass, surface mutually-exclusive open questions, seed tc-test-idea/v1 files for every REQ.
Phase 3
Drive five /tc:learn-from-* helpers against the seeded sample-project fixture; produce ten product-knowledge artifacts with file:line provenance.
Phase 4
Charter -> explore -> session-summary -> test-ideas: scope an exploration, classify every recorded event into universal observation and anomaly cores, enrich the Phase-2 seeds.
Phase 5
generate-bdd -> review-bdd -> traceability-map: render Gherkin from enriched test ideas with @req:/@cs: linkage, run the six-category rubric, and rebuild the requirement and scenario-level maps.
Phase 6
build-framework -> automation-plan -> automate -> review-automation -> generate-test-data: score scenarios, generate a traceable Playwright/TypeScript suite, and keep test data out of the code.
Phase 7
run -> analyze-results -> report -> quality-gate: execute the suite, triage failures, write the quality report with committed history, and score release-readiness.
Phase 8
Capture lessons from failures, exploration, and feedback, then review and promote them — every promotion visible in git diff, nothing rewritten silently.
Phase 9
visualize -> the eight diagram-* commands -> generate-infographic -> render-visuals: turn the workspace into Mermaid sources and deterministically rendered assets.
Phase 10
web-init -> web-start -> web-sync: bring up a read-only, team-facing viewer over the committed workspace. Renders the artifacts; never invents data or runs a command.
Phase 10.5
The controlled-execution pipeline: how a user request becomes a planned, permission-checked, approved, validated, and audited action — with default deny and no backdoor.
Phase 11
Drive Test Commander from another tool or agent over the Runtime API or the schema-first MCP server — both front-ends to the same governed pipeline.
Phase 12
sandbox-init -> sandbox-launch -> sandbox-status -> sandbox-stop: launch an on-demand environment from GitHub Actions, allow-listed and private-range-blocked by default.
Phase 13
watch-changes -> impact-analysis -> coverage-gap-analysis -> propose-tests -> create-test-pr: the watch -> analyze -> propose -> PR loop, gated by the configured autonomy mode.
For a single hands-on tour that walks all four shipped phases against one tmp project, read the blog post Test Commander after Phase 4: a hands-on tour.
What it produces
The output is not just “a test ran.” It is structured artifacts your team can read, edit, version-control, and learn from.
Charter · YAML frontmatter (Phase 4 · shipped)
---
id: CH-001
mission: Discover whether the Sign-in flow plus workspace-detail asset upload
behaves correctly under the documented risk conditions.
target: Sign-in flow plus workspace-detail asset upload (POST /workspaces/{id}/assets).
time-box: 60min
risk-areas:
- Authentication / authorization boundaries
- Session lifecycle and token leakage
- Performance under documented load thresholds
acceptance-criteria:
- Every flow under '...' completes the happy path with documented status codes.
- Authentication is correctly enforced for every endpoint that should require it.
- At least one anomaly per universal category is documented or explained away.
created_at: 2026-05-28T18:47:33Z
phase_3_sources:
- product-knowledge/entities.md
- product-knowledge/user-journeys.md
- requirements/open-questions.md
---Exploration note · table excerpt (Phase 4 · shipped)
# SESS-20260528-600 - exploration note for CH-001
## Observations
| # | event_type | Page | Result |
| - | --------------- | ---------------- | ------ |
| 0 | page_load | /sign-in | ok |
| 4 | click | /sign-in | |
| 5 | network_request | /sign-in | 201 |
| 8 | network_request | /dashboard | 200 |
## Anomalies
| Category | Severity | Page | Evidence |
| ---------------- | -------- | ---------------- | -------- |
| auth-mismatch | high | /workspaces/ws-1 | S-005 |
| broken-link | medium | /account/profile | S-004 |
| slow-response | high | /dashboard | S-002 |Enriched test-idea · Phase-2 seed + Phase-4 enrichment
---
schema: tc-test-idea/v1
requirement_id: REQ-005
requirement_title: All API access requires an authenticated user account
status: enriched # was: status: seed (Phase 2)
phase_4_sessions: [SESS-20260528-600]
phase_2_findings: [completeness, consistency, testability]
candidates: # Phase 2 seeded; preserved byte-for-byte
- id: REQ-005-happy-01
title: Happy path
type: positive
generated_by: /tc:requirements-to-tests
---
## Phase 4 enrichment
### SESS-20260528-600
- **CS-600-001** (negative) - Reproduce auth-mismatch on /workspaces/ws-1
- **CS-600-010** (happy) - Happy path: POST /sessions returns 201BDD feature · @req:/@cs: linkage (Phase 5 · shipped)
@area:sign-in
Feature: Sign-in flow
@req:REQ-005 @cs:CS-600-010 @smoke
Scenario: Happy path - authenticated session is created
Given a registered user on the sign-in page
When they submit valid credentials
Then the session is created and the dashboard loads
@req:REQ-005 @cs:CS-600-001 @regression @anomaly:auth-mismatch
Scenario: Authorization boundary is enforced
Given an authenticated user without workspace access
When they request a protected workspace asset
Then the request is rejectedGenerated spec · provenance + fixture data (Phase 6 · shipped)
import { test, expect } from "../fixtures/sign-in";
// @req:REQ-005 @cs:CS-600-010
test("Happy path - authenticated session is created", async ({
signInPage,
data,
}) => {
await signInPage.goto();
await signInPage.signIn(data.validUser);
await expect(signInPage.dashboard).toBeVisible();
});
// Generated by /tc:automate · refine steps inside the preserved region.
// Data flows from .test-commander/test-data/seed/sign-in.json (D6).Terminal workflow
Test Commander is terminal-first by design. A tester who is comfortable with the command line never has to open an IDE to drive the workflow.
| Command | Purpose |
|---|---|
./bootstrap.sh | Verify prereqs (Python 3.12, PDM, Docker, git, make); auto-install the safe ones. |
make install | Validate plugin manifests, register the local Claude Code marketplace, install the test-commander plugin, verify the twenty shipped skills. |
/tc:init | Inside a consuming project, copy the 63-file workspace template into .test-commander/. Idempotent — existing files are preserved. |
/tc:status | Print a snapshot: per-bucket file counts, populated counts (bytes differ from template), per-phase status. Read-only. |
/tc:journal append | Append a timestamped narrative entry to today's journal/YYYY-MM-DD.md. Append-only; never edited in place. |
/tc:next | Read the workspace state and recommend the next /tc:* command for this project. |
/tc:review-requirements | Run the 16-dimension rubric on uploaded requirements.md; emit requirements-review.md plus [<kind>] open-questions. |
Who it’s for
The same artifacts serve manual testers, automation engineers, QA managers, recruiters, and clients — each gets value at a different point in the lifecycle.
Turn exploratory knowledge into reusable evidence.
Start from a model, not a pile of vague tickets.
A repeatable quality process with explicit guardrails.
Evidence of modern QA judgement, not just tooling.
A practical path to AI-enhanced quality engineering.
Design principles
Principle 01
The agent helps; the tester owns the quality decision. AI output is never treated as automatically correct.
Principle 02
Per Decision D19, every shipped detector uses universal English and software-engineering vocabulary only. Domain awareness enters additively through <workspace>/config.yaml — extensions union with the universal core; you cannot remove a default. The same rubric runs against a banking app, a hospital system, or an internal dashboard.
Principle 03
Every entity, business rule, endpoint, anomaly, candidate scenario, or open question Test Commander surfaces is paired with the path:line where it came from. The structured artifacts are indexes, not summaries. You can always answer 'where did this come from' without leaving the workspace.
Principle 04
Every shipped helper is idempotent. Re-running against unchanged input produces byte-identical bytes. The workspace is safe to commit to git like any other source artifact — reviews show up as real diffs; nothing flickers on re-run.
Principle 05
Automation starts from understanding. Identify what matters first, then encode it.
Principle 06
Readable specs connect manual testers, automation engineers, and product stakeholders. The tc-test-idea/v1 schema Phase 2 authors and Phase 4 enriches is the input contract Phase 5 reads — every generated scenario carries @req:/@cs: tags that are the mechanical join key the traceability map parses.
Principle 07
AI may help generate tests, but CI/CD needs reliable checks. Phase 6 generates and structurally validates a Playwright/TypeScript suite, but never invokes the runner — execution is Phase 7's job. Playwright stays the source of truth.
Principle 08
Reports distinguish observed, tested, passed, failed, inferred, and items needing human review. The Phase-7 quality report (shipped) enforces this separation.
Principle 09
From Phase 10.5 on, every action above read-only flows through a single controlled-execution pipeline — intent, plan, permission policy, approval gate, bounded execution, validation, audit. The web console, the Runtime API, the MCP server, sandboxes, and the continuous agent are all front-ends to it. Default deny; nothing bypasses the gates.
Capability roadmap
What the system can do, phase by phase — every one now complete, each landed under test-driven discipline with its own annotated git tag. Pair this with the team-adoption maturity model below — the two roadmaps answer different questions.
Phase 0
ShippedBootstrap script, plugin manifest, marketplace registration, skill verifier, link checker, CI scaffold.
Phase 1
Shippedtc-core: /tc:init, /tc:status, /tc:journal, /tc:next. 63-file workspace template; per-phase recommendation engine.
Phase 2
Shippedtc-requirements: 16-dimension rubric, INVEST review, AC review, coverage map, tc-test-idea/v1 seeds.
Phase 3
Shippedtc-knowledge: five /tc:learn-from-* helpers (docs, specs, code, api, tests) + shared synthesizer; ten product-knowledge artifacts with file:line provenance.
Phase 4
Shippedtc-explore: /tc:create-charter, /tc:explore + internal review sub-mode, /tc:session-summary, /tc:test-ideas enriching Phase-2 seeds.
Phase 5
Shippedtc-bdd + tc-traceability: /tc:generate-bdd, /tc:review-bdd, /tc:traceability-map. Reads enriched test-ideas; emits Gherkin features tied to REQ-IDs with @req:/@cs: linkage.
Phase 6
Shippedtc-build-framework, tc-automation-plan, tc-automate, tc-test-data: lazy Playwright + TypeScript scaffolding, seven-factor automation scoring, generated suite + review, test data outside test code. The first executable artifacts.
Phase 7
Shippedtc-run + tc-quality-report + tc-evidence: /tc:run, /tc:analyze-results, /tc:report, /tc:quality-gate. Per-run records; committed quality-report history.
Phase 8
Shippedtc-learning: governed lessons inbox; /tc:learn-from-failures, /tc:learn-from-exploration, /tc:learn-from-feedback, /tc:review-lessons, /tc:promote-lessons. Nothing promoted silently.
Phase 9
Shippedtc-visualize: eight /tc:diagram-* types, /tc:generate-infographic, and /tc:render-visuals — Mermaid sources plus deterministically rendered SVG/PNG.
Phase 10
Shippedtc-web: a read-only, team-facing viewer over the committed workspace — dashboard, journal, BDD viewer, run history, evidence. Renders the artifacts; never invents data.
Phase 10.5
Shippedtc-governance: the single governed-execution pipeline — intent, plan, permission policy, approval gate, bounded execution, output validation, audit. Default deny; no backdoor.
Phase 11
Shippedtc-mcp: an expanded Runtime API and a schema-first MCP server — alternative front-ends that drive Test Commander through the same governed pipeline. Seven permission levels enforced server-side.
Phase 12
Shippedtc-sandbox: on-demand, team-accessible environments launched from GitHub Actions via a provider abstraction (docker-compose-local MVP). Governed and safe-by-default targeting.
Phase 13· Current
Shippedtc-continuous-quality: watch changes, map impact, find coverage gaps, propose tests, and open labeled PRs — gated by five autonomy modes (0 advisor → 4 governed-autonomy). The final phase.
Implementation roadmap
Adoption does not have to be all-or-nothing. Start with visibility, layer in requirements review, exploration, BDD, and automation, then graduate to a team console and finally to continuous, governed autonomy.
Stage 1
A shared quality baseline.
Existing requirements, tests, defects, risks, and reports become a single living dashboard. No major process change — just the picture, made visible.
Stage 2
Stories become clearer and more testable.
Test Commander reviews stories before implementation for clarity, missing acceptance criteria, edge cases, data rules, and automation suitability. Quality shifts left through better questions, not more meetings.
Stage 3
Exploratory testing becomes durable.
A tester points Test Commander at a target environment and explores. Observations, screenshots, risks, bugs, locator candidates, and test data needs join the quality knowledge base instead of someone's notebook.
Stage 4
Test design becomes traceable to business intent.
Approved test ideas become BDD scenarios tied to requirements and risks. The team can see which stories have coverage and which edge cases are still missing.
Stage 5
Playwright tests with rationale, not guesswork.
Automation candidates are scored on business criticality, repeatability, determinism, UI stability, and maintenance cost. Only the candidates worth automating become Playwright tests, page objects, fixtures, and test data.
Stage 6
A quality command center the whole team sees.
Live dashboard, journal, BDD viewer, run history, evidence gallery, risk register. Testers explore, developers inspect traces, product owners answer questions, managers read the summary — one shared quality story.
Stage 7
No-code testing environments on demand.
A pull request spins up a temporary Test Commander workspace with UI, runtime, uploaded docs, target URL, and artifact storage. Open a link, start testing. No local setup. No Playwright install on a tester's laptop.
Stage 8
The system gets better at helping the team test.
Lessons accumulate from requirements, code, tests, failures, and production defects. Candidate lessons are reviewed, accepted, rejected, or flagged for human review — then promoted into project guidance. The loop is governed, not silent.
Stage 9
Continuous monitoring with human-approved change.
Test Commander watches code, requirements, and pipelines. It analyzes impact, proposes coverage, runs approved suites, captures evidence, opens pull requests, and explains itself. Humans still approve the changes that matter.
Autonomy modes
Phase 13 ships this as a concrete control: the configured autonomy mode is a ceiling on which permission levels the continuous agent may auto-approve in the governed pipeline. Five modes, cumulative — and destructive / admin never auto-approve at any mode.
Mode 0
Reads artifacts, maps impact, finds coverage gaps, and proposes tests. Auto-approves nothing — a pure advisor. The right place to start.
Mode 1
Auto-approves safe-write work — analysis and proposed artifacts. Anything that writes code, runs tests, or opens a PR still waits for a human.
Mode 2
Adds execute-tests to what auto-approves: the agent may run designated suites in safe environments. It still cannot open a pull request.
Mode 3
Adds code-write and may open clearly-labeled pull requests — new BDD scenarios, generated tests, refreshed traceability. Humans review and merge.
Mode 4
Adds external-network targets. The broadest auto-approval — but destructive and admin actions are never auto-approved at any mode, and nothing auto-merges.
A mature workflow tends to settle at Mode 3. Test Commander runs continuously, but every change to test assets arrives as a clearly-labeled pull request a human can read, accept, or reject — and the agent never auto-merges.
Continuous quality agent mode
Phase 13 ships the continuous quality agent: Test Commander watches the application and the delivery pipeline, responds to change, and produces evidence — continuously, transparently, and under the autonomy-mode approval rules above.
Continuously improving, human-governed quality automation.
When requirements change, code ships, or tests fail, the agent reacts. It analyzes impact, reviews updated stories, identifies coverage gaps, generates candidate scenarios, runs impacted suites, captures evidence, updates reports, and records lessons learned. Automatic observation, automatic analysis, automatic reporting. Human-approved implementation.
The same loop runs on pull requests, pushes, nightly schedules, release candidates, and manual dispatches. Read-only analysis happens automatically. Report updates happen automatically. Test execution happens automatically in safe environments. Generated changes arrive as pull requests. Core methodology improvements are proposed, reviewed, and promoted deliberately.
Sample PR comment · continuous-agent output
Test Commander Analysis
Changed areas:
- Checkout
- Saved addresses
- Payment error handling
Detected risks:
- Saved address validation behavior changed
- Payment failure scenario lacks automated coverage
Existing tests:
- 12 checkout tests passing
- 2 impacted tests failed
- 1 flaky test detected
Recommended actions:
- Clarify expected behavior for expired saved addresses
- Add BDD scenario for payment timeout
- Approve generated Playwright test candidate
Artifacts:
quality report · screenshots · trace · coverage mapThe agent says what it changed, what it analyzed, what it found, and what a human should look at next. No surprises, no silent edits.
What’s next
Test Commander is part of an ongoing body of work in AI-augmented software quality, Playwright automation, and human-guided agentic testing. If you are hiring — or your team is exploring practical AI-assisted testing — I would like to talk.