self-verification-protocol
verified7be42bfd-729f-46b3-a876-ee6e7105bfe3
Before reporting any task done, actually run the code, read the diff, and confirm the output — never claim completion from memory. Use at the end of every task.
Metadata
Skill file
# Self-Verification Protocol
Use before reporting ANY task complete — actually run the code, read the diff, and confirm the output with evidence, rather than claiming done from memory.
## The Close-Out Ritual
Run these four steps, in order, before saying "done":
### 1. Re-run the tests / build
```bash
# Python
pytest -q # exit code 0? how many passed?
# JavaScript
npm test
# Generic — ALWAYS check the exit code
pytest -q; echo "Exit code: $?"
```
**Check the exit code explicitly.** A command that fails silently (or whose output you didn't read) is not a pass.
### 2. Read the diff in full
```bash
git diff # unstaged changes
git diff --cached # staged changes
git status # anything untracked or forgotten?
```
Read every hunk top-to-bottom. Ask: is this hunk justified by the task? Any debug prints, secrets, or stray edits?
### 3. Paste the real output
Capture and include the actual output — not a paraphrase:
```bash
# Capture the exact result
pytest -q 2>&1 | tail -20
# 12 passed in 1.34s
```
### 4. Confirm the artifact exists
```bash
ls -la path/to/deliverable.py # does the file exist and is it non-empty?
head -5 path/to/deliverable.py # does it contain what you think?
```
## The "Evidence Over Prose" Rule
| Prose claim (weak) | Evidence (strong) |
|---|---|
| "Tests pass" | `12 passed in 1.34s` (actual output) |
| "The file is written" | `ls -la` output + file size |
| "The API works" | `curl` response with status 200 |
| "It's fixed" | Before/after: `before: error X` → `after: 0 errors` |
Report what actually *executed* and its output — not what you intended to run.
## Catching the Silent-Failure Class
These failures look like success but aren't:
| Silent failure | How to catch it |
|---|---|
| Tests skipped (`pytest -k "not slow"` or `@skip`) | Read the summary line — `0 passed, 0 failed, 1 skipped` is NOT a pass |
| Command failed but exit code ignored | Always echo/check `$?` after each command |
| File written to the wrong path | `ls` the expected path; verify it's non-empty |
| `grep` found nothing (pattern wrong) | Check exit code 1 (no match) vs 0 (match) |
| API returned 200 with an error body | Parse the response body, not just the status |
| `git commit` committed nothing | `git log -1 --stat` — did the file actually get committed? |
```bash
# Explicitly checking for the silent-failure class
pytest -q
if [ $? -ne 0 ]; then echo "TESTS FAILED"; exit 1; fi
# Verify a test actually ran (not skipped)
pytest -q -rs # -rs shows skipped reasons
```
## Language-Specific Verify Commands
The "verify" step looks different per stack — pick the one that actually exercises the code:
| Stack | Verify command | What it proves |
|---|---|---|
| Python | `pytest -q && echo "exit $?"` | Tests pass, exit code checked |
| Python (single script) | `python script.py` + check output | Script runs without error |
| TypeScript/JS | `npm test` / `npx tsc --noEmit` | Tests + type-check |
| Go | `go test ./...` / `go build ./...` | Tests + compiles |
| Rust | `cargo test` / `cargo clippy` | Tests + lints |
| SQL migration | Run against a throwaway DB, `\d table` | Migration actually applies |
| Docker | `docker build . && docker run ...` | Image builds and starts |
| Shell script | `bash -n script.sh && ./script.sh` | Syntax + runs |
**Rule**: "verify" means *executing* the artifact, not just checking it exists.
## Worked Close-Out Example
Task: "Add a `--dry-run` flag to the deploy CLI."
```bash
# 1. Run the tests (after the final edit)
$ pytest -q
14 passed in 0.89s
$ echo "exit code: $?"
exit code: 0
# 2. Read the diff
$ git diff --stat
src/deploy.py | 18 +++++++++++++++---
tests/test_deploy.py | 12 ++++++++++++
2 files changed, 27 insertions(+), 3 deletions(-)
$ git diff
# ... read every hunk. The --dry-run logic is in place,
# the test covers it, no debug prints, no unrelated edits.
# 3. Actually exercise the feature
$ python -m deploy.cli --dry-run --env staging
[DRY RUN] Would deploy 4 services to staging (no changes applied)
$ echo "exit code: $?"
exit code: 0
# 4. Confirm the artifact
$ ls -la src/deploy.py
-rw-r--r-- 1 user user 4210 Jan 15 10:30 src/deploy.py
```
Now — and only now — is "done" an honest claim, backed by: test output, exit codes, the actual `--dry-run` output, and a confirmed file.
## Guardrails
- **Never** report "done" after writing code but before running it — writing ≠ working.
- **Never** trust a prior green run after further edits — re-run after the last change.
- **Never** say "it works" from memory — show the output or the exit code.
- **Always** check the exit code, not just the visible output (a command can print "success" and exit 1).
## Pitfalls
- **Reporting "done" after writing but before running**: "I wrote the function, it should work." — "should" is not evidence. Run it.
- **Trusting a prior green run after further edits**: Tests passed an hour ago, then you changed 5 files. The old green run proves nothing about the current tree.
- **Skipped tests counted as passes**: `0 passed, 1 skipped` reading as "tests are fine." A skip is a gap in coverage, not a pass.
- **Exit code ignored**: A build script prints "Build complete" but exits 1 (a sub-step failed). Read the exit code.
- **Paraphrasing instead of pasting**: "The tests passed" when the actual output said "1 failed, 11 passed." Paste the real output.
## Verify / Checklist
- [ ] Tests/build re-run AFTER the final edit (not before)
- [ ] Exit code explicitly checked (0 = success)
- [ ] `git diff` read in full — every hunk justified, no stray/debug/secret edits
- [ ] `git status` clean (or unexpected files explained)
- [ ] Real output captured and included in the report (not paraphrased)
- [ ] Artifact exists at the expected path and is non-empty
- [ ] Zero skipped/ignored tests unaccounted for
- [ ] The claim in the report matches the evidence in the outputAttached files
No attached files.