A pre-flight key validation step at server start that tests each API key against its service's auth endpoint will catch 100% of stale/invalid key errors before the pipeline begins processing

Hypothesis

A pre-flight key validation step at server start that tests each API key against its service's auth endpoint will catch 100% of stale/invalid key errors before the pipeline begins processing

Changelog

Date	Summary
2026-04-06	Audited: added Changelog, domain tag career, stamped last_audited
2026-04-04	Initial creation

---

Hypothesis

The OpenRouter key pitfall demonstrated that stale API keys cause pipeline outages that mimic code bugs. The debugging instinct is to check code first, wasting time on valid code. The hypothesis is that a pre-flight validation script that tests each key against its service’s lightweight auth endpoint will catch 100% of key-related failures at startup, before any pipeline processing begins.

Method

1. Key manifest: enumerate all 17 environment variables required by jobs-apply (OPENROUTER_API_KEY, STRIPE_SECRET_KEY, GOOGLE_CLIENT_ID, etc.) with their validation endpoints
2. Validation endpoints: each service has a lightweight auth check:
   • OpenRouter: GET /api/v1/auth/key (returns key metadata, 401 on invalid)
   • Stripe: GET /v1/balance (returns balance, 401 on invalid)
   • Neon: GET /api/v2/projects (returns projects, 401 on invalid)
   • Resend: GET /emails (returns list, 401 on invalid)
   • Google OAuth: validate token format (no network call needed for client ID/secret format)
3. Build scripts/key-check.ts: async validation of all keys in parallel. Report pass/fail for each. Exit code 1 if any key fails.
4. Integrate: call key-check from the server bootstrap() function before socket listeners are registered. Block auto:start until all keys validate.
5. Error handler update: on any 401/403 from an API during runtime, re-run key-check and include results in the error message. This ensures the “check key first” protocol is enforced programmatically.
6. Simulation test: rotate OpenRouter key to an invalid value, start server, verify key-check catches it within 2 seconds (vs the 30+ minutes of debugging in the pitfall scenario).

Results

Pending. Will measure:

• Time-to-detection: key-check runtime for all 17 keys
• Coverage: percentage of key-related failures caught at startup
• False negatives: scenarios where key-check passes but key is still problematic (e.g., rate-limited but valid)

Findings

Pending.

Next Steps

If confirmed, extract the key validation pattern as a reusable module. Apply to oil (data provider keys) and any other project with API key dependencies. Consider adding a key-rotate command that updates the key in ~/.zshrc and runs validation in one step.