This runbook verifies Skill Workshop RAG V1 as a Claude-first local release candidate. It is designed for a fresh local user and avoids private machine paths, account names, API keys, OAuth paths, and Claude profile details.
V1 is ready when a local user can:
V1 does not require hosted Claude CLI auth, automatic login, automatic generation tests on page load, broad device scans, V2 provider adapters, or private account discovery.
The repo includes a tracked demo workspace at:
examples/demo-workspace
Use it for clean local smoke tests by setting:
WORKSPACE_ROOT=examples/demo-workspace
SKILLS_DIR=.claude/skills
NEXT_PUBLIC_APP_TITLE=Skill Workshop RAG
Then configure one provider path:
# Option A: API mode
LLM_PROVIDER=anthropic_api
ANTHROPIC_API_KEY=<set locally>
ENABLE_LOCAL_CLAUDE_CLI=false
# Option B: local Claude CLI mode
LLM_PROVIDER=claude_code_cli
ENABLE_LOCAL_CLAUDE_CLI=true
CLAUDE_CLI_PATH=auto
CLAUDE_LOGIN_COMMAND=auto
CLAUDE_CONFIG_DIR=
Keep .env.local local-only. Do not commit real provider keys or private profile paths.
From the repo root:
npm install
npm run dev
Open http://localhost:3000/settings.
In Settings:
Rebuild Index.Test CLI.Open http://localhost:3000/chat and ask:
Use the release readiness smoke skill. What exact phrase proves this workspace is indexed?
Expected phrase:
Skill Workshop V1 release candidate is ready.
Open http://localhost:3000/export, enable diagnostics, generate a zip, and inspect that diagnostics include only sanitized readiness, index, skill quality, Claude project, and settings summaries.
Run the repo-native automated release gate:
npm run verify:release
It runs project cleanup dry-run preflight, the full test sweep, lint, production build, production server smoke with desktop/mobile visual route checks, dependency audit, local browser/API smoke with keyboard action coverage, safe button smoke, manual QA helper auto smoke, project cleanup dry-run postflight, local artifact cleanup dry-run postflight, asset usage audit, documentation link audit, dead-code audit, unused-export audit, diff whitespace check, untracked text hygiene scan, and privacy scan.
Print sanitized release evidence after a verified gate:
npm run release:evidence -- --gate-result passed
The evidence output includes branch, commit, dirty/clean status, changed-file count, release-gate result, test-file count, command coverage, privacy-scan status, and manual gates still pending. It does not include local paths, provider keys, account identifiers, OAuth paths, or raw Claude profile folders.
To run a finite final package pass from the current checkout:
npm run release:prepare
release:prepare runs cleanup dry-runs, runs npm run verify:release exactly once, then prints the sanitized release evidence summary. It does not mark native folder picker, visible Open Login, provider auth, or real account-backed chat as passed.
After manual QA passes and the manual QA tracker issue is closed, create the GitHub draft release and tag with:
npm run release:draft -- --tag v1.0.0
Before creating anything, the draft command verifies that the checkout is clean, local main matches origin/main, the release notes file exists, the tag does not already exist locally or remotely, and GitHub issue #3 is closed. Use --dry-run to validate those guards without creating a tag or draft:
npm run release:draft -- --tag v1.0.0 --dry-run
If issue #3 is still open, the command fails intentionally. Do not use it to bypass native folder picker, visible Open Login, provider auth, real account-backed chat, or manual sanitization review.
If the manual QA session is intentionally skipped, do not create a final V1 tag. You may create an automated-only draft prerelease with an explicit prerelease tag and skip flag:
npm run release:draft -- --tag v1.0.0-rc.1 --manual-qa-skipped
That path keeps issue #3 open, creates a GitHub draft prerelease, and must be described as automated-only. It is not a fully manually verified V1 release.
Before publishing V1 evidence or updating the manual QA tracker, confirm the local release docs match the current checkout:
git status --short --branch
npm run audit:docs
npm run verify:release
npm run release:evidence -- --gate-result passed
Keep current automated status in latest-local-qa-evidence.md, keep detailed historical QA notes in qa-history.md, keep public-facing release changes in release-notes.md, and keep this runbook as the source of truth for command gates and manual external QA.
If you need to debug an individual gate, run the underlying commands:
npm test
npm run lint
cmd.exe /c npm run build
npm audit
npm run smoke:local
npm run smoke:buttons
npm run smoke:production
npm run audit:assets
npm run audit:docs
npm run audit:dead-code
npm run audit:exports
git diff --check
npm run smoke:local drives local interactive Settings, Skills, Chat, Export, Editor, and Guided Builder flows in Chromium and verifies keyboard paths, semantic route structure, ARIA references, accessible control names, and 44px action targets across those states.
npm run smoke:buttons clicks low-risk visible buttons across the main local routes and fails on console errors or real failed requests. It intentionally skips auth launchers, native folder pickers, save/delete/export/send actions, secret reveal buttons, and provider calls.
npm run smoke:production starts the built app with next start against the demo workspace and verifies production API guards including POST /api/chat, desktop/mobile visual rendering, landmarks, heading order, ARIA references, accessible control names, 44px action targets, local hash links, built-client Settings/editor/guided/chat/export interaction states, and browser console/page errors.
npm run audit:assets fails when tracked or visible untracked image/font/icon assets are not referenced by source or docs. Next’s conventional src/app/favicon.ico is allowed without an explicit import.
npm run audit:docs fails when README or docs markdown files link to missing repo-local files or missing markdown headings. It ignores external source/research links.
npm run audit:dead-code uses Knip with repo-specific Next route, script, and test entry points. It fails on unused files, unused dependencies, unlisted dependencies/binaries, or unresolved imports while allowing the expected Windows system helpers used by local smoke and cleanup scripts.
npm run audit:exports uses Knip’s exported-symbol audit, including entry exports, to keep public helper/type surfaces narrow after refactors.
If a local smoke/dev run is interrupted and memory or ports look stale, inspect project-owned process trees first:
npm run cleanup:project:dry-run
Then stop only this repo’s detected Next, smoke, test, release, and manual QA helper process trees:
npm run cleanup:project
The cleanup command requires this repo path plus a known Next or release-script command signature and excludes Codex/MCP infrastructure so other Codex conversations are not targeted.
To remove ignored local build and smoke artifacts after a verification run, inspect first:
npm run cleanup:artifacts:dry-run
Then remove only the generated .next, .local-workspace, out, build, coverage, tsconfig.tsbuildinfo, and next-env.d.ts artifacts:
npm run cleanup:artifacts
The artifact cleanup does not delete .env.local, node_modules, tracked docs, screenshots, or source files.
These checks cover device UI and auth launchers that the automated smoke runner intentionally does not click. Run them locally only, and do not paste API keys, account emails, organization names, OAuth paths, raw Claude profile folder names, or full home paths into screenshots or notes.
When the local app is already running, use the manual external QA helper to print the current sanitized readiness summary and remaining device/account checks:
npm run qa:manual
Set MANUAL_QA_BASE_URL=http://localhost:3000 when the app is running on a different local port. If no dev server is running, use the temporary-server helper instead:
npm run qa:manual:auto
Both helpers print only sanitized status plus the manual checklist, including why each gated check remains manual. They do not click native OS dialogs, launch login, send chat, or write evidence files.
The helper may print blocked readiness when the local provider is intentionally incomplete, such as API mode without an API key or Claude CLI mode without a usable account. Treat that as a sanitized setup snapshot, not as a failed automated smoke run. Finish the account-backed chat check only after you intentionally configure a provider you are allowed to use.
After running or intentionally deferring the manual checks, open Settings and mark the results in Manual QA Evidence. Use Passed only for checks you actually performed, Needs fix for observed failures, Skipped for checks intentionally not verified, and Pending for checks that still need a decision. Skipped is not a pass. The panel stores only status and timestamp in browser storage when available, or in memory for the current page if storage is restricted. It does not store prompts, screenshots, account names, profile paths, or auth output.
WORKSPACE_ROOT or SKILLS_DIR.Choose folder.Cancel and confirm Settings stays usable and the field is unchanged.Unsaved changes..env.local; otherwise reload the page.Pass criteria:
Open Login.Test CLI and confirm the result is sanitized.Pass criteria:
Run this only with an account or API key you are allowed to use. The automated smoke test verifies chat UI, streaming errors, retry, citations, and readiness states with mocked/local-safe inputs; this manual check proves the real selected provider can answer.
ANTHROPIC_API_KEY is present locally without copying its value into notes.Test CLI.Pass criteria:
Evidence to record:
Manual QA Evidence panel for session-local pass/fail/skipped state.npm test executes every src/**/*.test.ts file and release script helper test under scripts/**/*.test.mjs. Prefer the npm script for the full sweep so source and release-helper coverage stay in sync.
Before a checkpoint commit or release tag, run a static scan over tracked docs and source for:
Expected result: no private local paths, account identifiers, real API keys, provider auth paths, or bearer tokens. The authoritative privacy pattern lives in scripts/smoke/assertions/privacy.mjs and is run by the full release verifier.
npm run verify:release
Capture this evidence for a V1 release-candidate checkpoint:
/settings, /skills, /chat, /export, and /editor/guided./api/chat, /api/index, /api/chat/status, /api/release/readiness, and diagnostics zip generation.npm run release:draft -- --tag v1.0.0.Commit and push only after explicit approval.
The latest privacy-safe local evidence snapshot is tracked in latest-local-qa-evidence.md. Older detailed evidence is kept in qa-history.md.