AI validation checklist
A checklist is a temporary step. Move each check below into CI as soon as it's stable. Decide which checks block a merge and which are advisory. Engineers ignore a list where everything blocks.
| Check | Prompt | Blocking? |
|---|---|---|
| Security review | "Review for OWASP top 10 vulnerabilities" | Yes for auth, payments, user data |
| Smoke testing | "Write smoke tests for critical paths" | Yes |
| Test coverage | "Identify untested code paths" | Advisory |
| Code reduction | "Simplify and remove unnecessary complexity" | Advisory |
| Dead code | "Find and remove unused code" | Advisory |
| Stress testing | "Identify bottlenecks under load" | Yes for hot paths |
| Observability | "Add logging, metrics, and tracing" | Yes for new services |
| Documentation | "Add docstrings for public APIs" | Advisory |
| 12 Factor | "Verify 12 Factor compliance" | Yes for new services |
Scale depth to impact. A config tweak and a payment flow shouldn't get the same pass. Track which checks actually catch defects. Drop the ones that never fire.
The 12 Factor App
The 12 Factor App is the baseline for services that scale and are easy to operate. AI-generated code drifts from it constantly (hardcoded config, in-memory state, log files), so check it explicitly.
| Factor | Principle | AI prompt |
|---|---|---|
| 1. Codebase | One codebase, many deploys | "Ensure no environment-specific code in main" |
| 2. Dependencies | Declare and isolate | "Check every dependency is declared" |
| 3. Config | Store in environment | "Move hardcoded values to environment variables" |
| 4. Backing services | Attached resources | "Make DB and cache connections configurable URLs" |
| 5. Build, release, run | Strict separation | "Separate build scripts from runtime code" |
| 6. Processes | Stateless | "Remove in-memory state, use external stores" |
| 7. Port binding | Export via port | "Bind to PORT, self-contained" |
| 8. Concurrency | Scale out via processes | "Design for horizontal scaling" |
| 9. Disposability | Fast start, graceful stop | "Handle SIGTERM gracefully" |
| 10. Dev/prod parity | Keep environments similar | "Use the same backing services in dev and prod" |
| 11. Logs | Event streams | "Log to stdout, not files" |
| 12. Admin processes | One-off processes | "Admin tasks as scripts, not endpoints" |
Reference: https://12factor.net/