Skill

audit-pitfalls

March 30, 2026

auditdimension-health

Trigger

Audit the Pitfalls dimension for tagging consistency, cross-links, and knowledge graph density

Version: 260420

Changelog

260420: multiple edits

v_migrate: Changelog migrated from table to YYMMDD H3 format per versioning-standard rule 2 (V1.6 of skills upgrade plan)
v6: Added license, sources per V6.1/V6.2 of skills upgrade plan

260406: v1.3

v1.3: Added check 16 (PII and secrets scan per _config/pii-rules.yaml)
v2.2: Added checks 13-15 (voice & tone, tag completeness, graph connectivity)
v2.1: Added check 12 (content length & structure compliance per content-length-spec.yaml)

260403: v2.0

v2.0: Added checks 8-11 (research domain match, pitfall cluster detection, cross-project correlation, lifecycle chain completion score). Updated baseline to 29 pitfalls. Added research-informed idea creation and cluster-based gap filling procedures.
Added Agent to mounted_allowed_tools + post-audit visual enrichment trigger
Added Visual Enrichment section + self-improving-agent-patterns cross-reference

Systematic audit of the Pitfalls dimension: the origin layer of the vault’s lifecycle chain (pitfall → idea → experiment → skill → breakthrough). Pitfalls capture things that went wrong in real projects: bugs, deployment failures, architectural misjudgments, and process breakdowns. They are the raw material that drives the entire knowledge refinement pipeline.

This skill validates that every pitfall note is structurally complete, properly tagged, connected to its parent project, and linked forward to any ideas, experiments, or skills that address it. A healthy Pitfalls dimension means the vault’s learning loop has solid foundations: nothing learned the hard way gets lost.

Current baseline (v2.0): 29 pitfall notes across 8+ projects. 18 ideas exist, 3 of which reference pitfalls via addresses_pitfalls. 2 skills reference pitfalls via resolved_pitfalls. 147 research notes exist but are not yet connected to pitfalls. Lifecycle chain completion: 10% (3/29).

Interface

Trigger: Run when adding new pitfalls, after any vault restructuring, or on a periodic audit cadence (weekly recommended).

Inputs:

vault_path: root of the Obsidian vault (default ~/vault)
pitfalls_dir: directory containing pitfall notes (default topics/pitfalls/)
ideas_dir: directory containing idea notes (default ideas/)
skills_dir: directory containing skill notes (default skills/)
experiments_dir: directory containing experiment notes (default experiments/)

Outputs:

audit_report: per-pitfall pass/fail table covering all checks
gap_list: prioritized list of fixes needed, grouped by severity (P0/P1/P2)
graph_density_score: ratio of actual outbound wikilinks to the expected minimum (2 per pitfall)

Provenance

Extracted from manual vault-building sessions where pitfall notes were created from git history, session logs, and project retrospectives. The check criteria are derived from the vault’s type: topic schema and the lifecycle chain contract between dimensions.

Usage Notes

Run the full audit before claiming the Pitfalls dimension is healthy
The most common failure mode is orphaned pitfalls: notes with zero or one wikilink that sit disconnected from the knowledge graph
A pitfall without a forward-link to at least one idea is not yet “activated” in the lifecycle: it is knowledge captured but not yet leveraged
Pitfalls from older projects (pre-vault) often lack the project field: these need manual triage to assign the correct project

What to Check

1. Frontmatter Completeness (P0: blocks vault-bootstrap lint)

For every file in topics/pitfalls/*.md, verify:

Field	Required	Expected Value
`type`	yes	`topic`
`created`	yes	ISO date (YYYY-MM-DD)
`project`	yes	slug matching a `projects/{slug}/_index` path
`date`	recommended	same as `created` or the date the pitfall occurred
`status`	recommended	`fresh`, `addressed`, or `resolved`
`tags`	yes	array that includes `pitfall`

Check procedure:

Parse YAML frontmatter of each file
Confirm type: topic: reject any file claiming to be a pitfall without this
Confirm tags array contains the literal string pitfall
Confirm created is present and is a valid date
Confirm project is present and non-empty
Warn if status is missing (default assumption: fresh)

2. Body Structure (P1: required for knowledge extraction)

Each pitfall should have three narrative sections (exact heading text may vary):

Required Section	Acceptable Variants
`## What Happened`	`## What Goes Wrong`, `## What Went Wrong`, `## Problem`
`## Root Cause`	`## Why`, `## Cause`, `## Analysis`
`## Lesson Learned`	`## Lesson`, `## Fix`, `## How to Avoid`, `## Key Insight`, `## Prevention Rules`

Check procedure:

Extract all ## headings from the body (below frontmatter)
Match each against the acceptable variants (case-insensitive)
Flag any pitfall missing one or more of the three required narrative sections
A pitfall with ## What Happened + ## Root Cause + ## Fix passes (e.g., astro-carousel-js-ssr-mismatch.md)
A pitfall with ## What Goes Wrong + ## How to Avoid passes (combines root cause and lesson)

3. Project Backlink (P0: required for graph connectivity)

Every pitfall must contain a wikilink to its parent project. The link format is:

[projects/{project-slug}/_index](/projects/{project-slug}/_index)

or at minimum:

[{project-slug}](/{project-slug})

Check procedure:

Read the project field from frontmatter
Scan the body for [projects/{project}/_index](/projects/{project}/_index) or [{project}](/{project})
If neither exists, flag as missing project backlink: this is a P0 gap because it disconnects the pitfall from the project’s knowledge subgraph

4. Forward-Links to Ideas (P1: lifecycle activation)

Pitfalls that have been “addressed” should be linked from at least one idea’s addresses_pitfalls array. Conversely, pitfalls should ideally contain a forward-link to the idea that addresses them.

Check procedure:

For each idea in ideas/*.md, parse the addresses_pitfalls array
Build a reverse index: pitfall_path → [idea_paths]
For each pitfall, check if it appears in any idea’s addresses_pitfalls
If a pitfall IS referenced by an idea but the pitfall body does NOT contain a [ideas/{idea-slug}](/ideas/{idea-slug}) wikilink, flag as missing bidirectional link
Track the coverage ratio: pitfalls_with_idea_links / total_pitfalls

Current state (baseline): 3 of 21 pitfalls are referenced by ideas:

ratchet-phase-selection-bug ← cross-project-dqi-ratchet
linkedin-behavioral-detection ← gaussian-timing-for-all-automation
mcp-sdk-version-pinning-railway ← unified-mcp-gateway

5. Forward-Links to Skills (P1: lifecycle resolution)

Pitfalls that are fully “resolved” should appear in a skill’s resolved_pitfalls array. This closes the lifecycle loop.

Check procedure:

For each skill in skills/*.md, parse the resolved_pitfalls array
Build a reverse index: pitfall_path → [skill_paths]
For each pitfall, check if it appears in any skill’s resolved_pitfalls
If yes, the pitfall’s status should be resolved: flag mismatches
If a pitfall IS in a skill’s resolved_pitfalls but the pitfall body has no [skills/{skill-slug}](/skills/{skill-slug}) link, flag as missing bidirectional link

Current state (baseline): 2 pitfalls are resolved by skills:

ratchet-phase-selection-bug ← karpathy-ratchet
linkedin-behavioral-detection ← behavioral-anti-detection

6. Wikilink Density (P2: graph health)

Every pitfall should have at least 2 outbound wikilinks. The minimum expectation:

1 link to the parent project
1 link to a related note (idea, experiment, skill, or another pitfall)

Check procedure:

Count all [...](/...) patterns in each pitfall’s body
Flag any pitfall with fewer than 2 outbound wikilinks as low connectivity
Compute graph_density_score = sum(actual_links) / (total_pitfalls * 2)
Target: graph_density_score >= 1.0 (average of 2+ links per pitfall)

7. Orphan Detection (P2: no inbound links)

A pitfall is orphaned if no other note in the vault links TO it.

Check procedure:

Search the entire vault for [topics/pitfalls/{slug}](/topics/pitfalls/{slug}) references
Also search for [{slug}](/{slug}) short-form references
A pitfall with zero inbound links from outside topics/pitfalls/ is orphaned
Orphaned pitfalls should be linked from their parent project’s _index.md or a relevant idea/experiment

8. Research Domain Match (P1: lifecycle bridge)

Pitfalls should be connected to research notes that contain applicable methodologies or solutions. This is the missing JOIN between the pitfall dimension and the research dimension.

Check procedure:

For each “fresh” pitfall (no idea addresses it), extract its project and tags fields
Scan research/*.md for notes with matching domain tags or keywords from the pitfall’s ## Root Cause section
For each research note, check if its Applications section mentions the same project as the pitfall
Score matches using composite: (domain_alignment * 0.3) + (keyword_overlap * 0.3) + (temporal_proximity * 0.2) + (project_overlap * 0.2)
- domain_alignment: 1.0 if same domain tag, 0.5 if adjacent (ml/ai-agents), 0 otherwise
- keyword_overlap: count of shared substantive words between pitfall root cause and research key findings, normalized
- temporal_proximity: 1.0 if research < 7 days old, 0.5 if < 30 days, 0.25 if older
- project_overlap: 1.0 if research Applications mentions same project, 0 otherwise
Surface top-3 research matches per fresh pitfall in audit output

echo "=== Research Matches for Fresh Pitfalls ==="
for p in ~/vault/topics/pitfalls/*.md; do
  slug=$(basename "$p" .md)
  status=$(grep '^status:' "$p" | head -1 | sed 's/status: *//')

  # Only match fresh pitfalls (no idea addresses them)
  if [ "$status" = "fresh" ] || [ -z "$status" ]; then
    project=$(grep '^project:' "$p" | head -1 | sed 's/project: *//')
    # Extract root cause keywords (## Root Cause section)
    keywords=$(sed -n '/^## Root Cause/,/^##/p' "$p" | grep -oE '[a-z]{4,}' | sort -u | head -10 | tr '\n' '|')

    if [ -n "$keywords" ]; then
      echo "  $slug ($project):"
      # Search research notes for keyword matches
      grep -rl -E "$keywords" ~/vault/research/*.md 2>/dev/null | head -3 | while read -r match; do
        echo "    -> $(basename "$match" .md)"
      done
    fi
  fi
done

9. Pitfall Cluster Detection (P1: pattern extraction)

Individual pitfalls should be grouped into systemic clusters by root cause pattern. This reduces 29 individual problems to ~7 actionable anti-patterns and makes research matching more effective.

Defined clusters (update as new patterns emerge):

Cluster ID	Pattern	Expected Members
`DEPLOY`	Deployment mechanism proliferation	cloudflare-pages, vercel-deploy, railway-nixpacks, railway-deployment-saga, codex-merge-conflict, mcp-sdk-version-pinning
`DETECT`	Behavioral/detection evasion	linkedin-behavioral-detection, rubric-overfitting, linkedin-shadow-dom, agent-memory-bloat
`API`	API contract brittleness	lat-lon-swap, internal-tourism-api-post-hangs, openrouter-key, pirate-ship-provider-quarantine
`RENDER`	Frontend rendering state	html2canvas-live-dom, xterm-webgl-canvas, astro-carousel-ssr, screenshot-fifo-sort
`HOOKS`	Tooling/hook edge cases	unset-claudecode, hook-paths-absolute, sessionend-hook-timeout, bash-set-e-arithmetic
`STATE`	State management/logging	log-session-header, log-buffer-requeue, ratchet-phase-selection
`KNOWLEDGE`	Knowledge system integrity	circular-knowledge-corruption, agent-memory-bloat

Check procedure:

For each pitfall, attempt to assign it to a cluster based on keyword matching against cluster pattern descriptions
Pitfalls matching no cluster are flagged as “unclassified” (WARN)
Clusters with 3+ members and LOW research coverage are flagged as “research debt” (P1)
Output: cluster membership table + research coverage per cluster

echo "=== Pitfall Clusters ==="
# Define cluster keywords
declare -A clusters
clusters[DEPLOY]="deploy|railway|vercel|cloudflare|nixpacks|docker|merge"
clusters[DETECT]="detect|behavioral|bot|shadow|rubric|evasion"
clusters[API]="api|header|key|coordinate|provider|hang|timeout"
clusters[RENDER]="render|canvas|dom|ssr|screenshot|webgl|carousel"
clusters[HOOKS]="hook|claude|bash|path|timeout|env"
clusters[STATE]="log|buffer|state|session|ratchet|phase"
clusters[KNOWLEDGE]="knowledge|memory|corruption|circular|provenance"

for p in ~/vault/topics/pitfalls/*.md; do
  slug=$(basename "$p" .md)
  content=$(cat "$p" | tr '[:upper:]' '[:lower:]')
  matched=false

  for cluster in "${!clusters[@]}"; do
    if echo "$content" | grep -qE "${clusters[$cluster]}"; then
      echo "  $cluster: $slug"
      matched=true
      break
    fi
  done

  if ! $matched; then
    echo "  UNCLASSIFIED: $slug"
  fi
done

10. Cross-Project Pitfall Correlation (P2: universal solutions)

Pitfalls that repeat the same root cause pattern across 2+ projects indicate systemic issues that deserve universal solutions. Research that addresses a multi-project pattern is more valuable than project-specific fixes.

Check procedure:

From cluster analysis (check 9), extract unique projects per cluster
Flag clusters hitting 3+ projects as “systemic” (P1)
For systemic clusters, check if a matching research note with Applications section mentions all affected projects
If not, flag as “universal solution gap”: needs research or a cross-project idea

echo "=== Cross-Project Correlation ==="
echo "Cluster | Projects | Count | Systemic?"
echo "--------|----------|-------|----------"

# Count unique projects per cluster
for cluster in DEPLOY DETECT API RENDER HOOKS STATE KNOWLEDGE; do
  projects=""
  for p in ~/vault/topics/pitfalls/*.md; do
    content=$(cat "$p" | tr '[:upper:]' '[:lower:]')
    if echo "$content" | grep -qE "${clusters[$cluster]:-placeholder}"; then
      proj=$(grep '^project:' "$p" | head -1 | sed 's/project: *//')
      projects="$projects $proj"
    fi
  done
  unique=$(echo "$projects" | tr ' ' '\n' | sort -u | grep -v '^$' | wc -l | tr -d ' ')
  proj_list=$(echo "$projects" | tr ' ' '\n' | sort -u | grep -v '^$' | tr '\n' ',' | sed 's/,$//')
  systemic="no"
  [ "$unique" -ge 3 ] && systemic="YES"
  echo "$cluster | $proj_list | $unique | $systemic"
done

11. Lifecycle Chain Completion Score (P1: progress tracking)

Track what percentage of pitfalls have progressed through the lifecycle chain. This is the single most important health metric for the vault’s learning loop.

Chain stages:

Stage 0: Pitfall exists (captured): baseline
Stage 1: An idea references it in addresses_pitfalls: “addressed”
Stage 2: The idea was promoted to an experiment: “experiment in progress”
Stage 3: A skill lists it in resolved_pitfalls: “resolved”
Stage 4: A breakthrough note exists linked to the resolving experiment/skill: “complete”

Check procedure:

For each pitfall, determine highest chain stage reached
Compute aggregate: chain_completion = pitfalls_at_stage_1_or_higher / total_pitfalls
Target: >= 50%
Output: per-pitfall chain status + aggregate score

echo "=== Lifecycle Chain Completion ==="
total=0
stage0=0; stage1=0; stage2=0; stage3=0; stage4=0

# Build reverse indexes
idea_pitfalls=$(grep -rl 'addresses_pitfalls' ~/vault/ideas/*.md 2>/dev/null | xargs grep -ohP '\[\[topics/pitfalls/[^\]]+\]\]' | sed 's/\[\[topics\/pitfalls\///;s/\]\]//' | sort -u)
skill_pitfalls=$(grep -rl 'resolved_pitfalls' ~/vault/skills/*.md 2>/dev/null | xargs grep -ohP '\[\[topics/pitfalls/[^\]]+\]\]' | sed 's/\[\[topics\/pitfalls\///;s/\]\]//' | sort -u)

for p in ~/vault/topics/pitfalls/*.md; do
  slug=$(basename "$p" .md)
  total=$((total + 1))
  stage=0

  # Check if any idea addresses this pitfall
  if echo "$idea_pitfalls" | grep -qx "$slug"; then
    stage=1
    # Check if the addressing idea was promoted to experiment
    for idea in ~/vault/ideas/*.md; do
      if grep -q "\[\[topics/pitfalls/$slug\]\]" "$idea" 2>/dev/null; then
        if grep -q "^status: promoted" "$idea"; then
          stage=2
        fi
      fi
    done
  fi

  # Check if any skill resolved this pitfall
  if echo "$skill_pitfalls" | grep -qx "$slug"; then
    stage=3
    # Check for breakthrough note linked to resolving skill
    for b in ~/vault/breakthroughs/*.md; do
      if grep -q "$slug\|resolved_pitfalls" "$b" 2>/dev/null; then
        stage=4
      fi
    done
  fi

  case $stage in
    0) stage0=$((stage0 + 1)); icon="[ ][ ][ ][ ]" ;;
    1) stage1=$((stage1 + 1)); icon="[x][ ][ ][ ]" ;;
    2) stage2=$((stage2 + 1)); icon="[x][x][ ][ ]" ;;
    3) stage3=$((stage3 + 1)); icon="[x][x][x][ ]" ;;
    4) stage4=$((stage4 + 1)); icon="[x][x][x][x]" ;;
  esac

  echo "  $icon $slug"
done

addressed=$((total - stage0))
pct=$((addressed * 100 / total))
echo ""
echo "Chain Completion: $addressed/$total ($pct%): target >= 50%"
echo "  Stage 0 (captured only): $stage0"
echo "  Stage 1 (idea exists):   $stage1"
echo "  Stage 2 (experiment):    $stage2"
echo "  Stage 3 (skill):         $stage3"
echo "  Stage 4 (breakthrough):  $stage4"

12. Content Length & Structure Compliance (WARN)

Every topics/pitfalls/*.md note must have body content within 200-400 words (0.5-1.5 pages) and include all required sections. Word count excludes YAML frontmatter, ## Changelog section, and image embeds (![[...]]). See _config/content-length-spec.yaml for the canonical spec.

Required sections: What Happened, Root Cause, How to Avoid

How to check:

for f in "$VAULT"/topics/pitfalls/*.md; do
  [ -f "$f" ] || continue
  slug=$(basename "$f" .md)
  # Extract body: skip frontmatter, remove Changelog section, remove image embeds
  body=$(awk '/^---$/{n++; next} n>=2' "$f" | sed '/^## Changelog/,/^## [^C]/{ /^## [^C]/!d; }' | grep -v '^!\[\[')
  word_count=$(echo "$body" | wc -w | tr -d ' ')
  if [ "$word_count" -lt 200 ]; then
    echo "ERROR (stub): $slug is $word_count words (min 200)"
  elif [ "$word_count" -gt 480 ]; then
    echo "WARN (verbose): $slug is $word_count words (max 400)"
  fi
  # Check required sections
  for section in "What Happened" "Root Cause" "How to Avoid"; do
    if ! grep -q "^## $section" "$f"; then
      echo "MISSING SECTION: $slug lacks ## $section"
    fi
  done
done

Severity: ERROR if below min (stub content); WARN if above max x 1.2 (unfocused); ERROR if required section missing

Rationale: Pitfalls document failure patterns. Root cause and avoidance checklist are the high-value content. A verbose pitfall note will not be read when someone is about to repeat the mistake.

13. Voice & Tone Compliance (WARN)

Every topics/pitfalls/*.md note must follow the pitfalls voice spec: 70% clarity / 30% energy. The ## What Happened section tells a story: a real failure with stakes. The ## Root Cause section explains the mechanism clearly. The ## How to Avoid section (or equivalent) closes with a forward-looking rule: “We assumed X. That cost us [impact]. Here’s how to avoid it.”

Per-file checklist (flag WARN if 2+ items fail):

Opening of ## What Happened (or equivalent) names a concrete assumption, action, or decision: not a vague “there was a problem”
Jargon terms (deploy hooks, lint rules, behavioral detection thresholds) are defined inline on first use
## Root Cause (or equivalent) presents the specific mechanism before the general pattern (concrete before abstract)
The impact is quantified somewhere in the note: hours lost, deploys failed, sessions killed, or other measurable cost
## Root Cause uses active voice: “The hook timeout killed 76% of sessions” not “Sessions were being killed”
## How to Avoid (or equivalent) states a reusable rule, not just what was fixed this time: it should apply to future projects

Bash heuristic: passive voice and impact detection:

VAULT="${vault_path:-$HOME/vault}"
for f in "$VAULT"/topics/pitfalls/*.md; do
  [ -f "$f" ] || continue
  slug=$(basename "$f" .md)
  # Passive voice patterns
  passive=$(grep -ciE '\b(was|were|been|is|are) (assumed|missed|broken|introduced|lost|ignored|forgotten|overlooked)\b' "$f" || true)
  # Check for any quantified impact (numbers, percentages, durations)
  has_impact=$(grep -cE '\b[0-9]+\s*(%|minutes?|hours?|days?|seconds?|sessions?|deploys?|errors?|ms)\b' "$f" || true)
  [ "$passive" -gt 2 ] && echo "WARN: $slug has $passive passive-voice instances: rewrite What Happened/Root Cause"
  [ "$has_impact" -eq 0 ] && echo "WARN: $slug has no quantified impact: add a concrete cost (time lost, failure rate, etc.)"
done

Severity: WARN: a pitfall without a quantified cost cannot be prioritized against other pitfalls. A pitfall in passive voice fails to convey the urgency needed to prevent recurrence.

14. Tag Completeness (WARN)

Every topics/pitfalls/*.md note must have a well-formed tags array with minimum coverage to support cluster-based analysis (Check 9) and cross-dimension queries.

Requirements:

tags array must have at least 3 entries
Must include #pitfall
Must include at least one domain tag from the controlled vocabulary: ml, data-eng, quant-finance, game-dev, ops, career, omscs, real-estate, ai-agents, frontend
Must include at least one cluster tag from the defined clusters: DEPLOY, DETECT, API, RENDER, HOOKS, STATE, KNOWLEDGE

Bash check:

VAULT="${vault_path:-$HOME/vault}"
DOMAIN_TAGS="ml|data-eng|quant-finance|game-dev|ops|career|omscs|real-estate|ai-agents|frontend"
CLUSTER_TAGS="DEPLOY|DETECT|API|RENDER|HOOKS|STATE|KNOWLEDGE"
for f in "$VAULT"/topics/pitfalls/*.md; do
  [ -f "$f" ] || continue
  slug=$(basename "$f" .md)
  # Count tags
  tag_count=$(grep -A 20 '^tags:' "$f" | grep -c '^\s*-' || true)
  [ "$tag_count" -lt 3 ] && echo "WARN: $slug has $tag_count tag(s) (min 3)"
  # Check #pitfall tag
  if ! grep -A 20 '^tags:' "$f" | grep -q 'pitfall'; then
    echo "WARN: $slug missing required #pitfall tag"
  fi
  # Check domain tag
  if ! grep -A 20 '^tags:' "$f" | grep -qE "$DOMAIN_TAGS"; then
    echo "WARN: $slug missing domain tag (expected one of: $DOMAIN_TAGS)"
  fi
  # Check cluster tag
  if ! grep -A 20 '^tags:' "$f" | grep -qE "$CLUSTER_TAGS"; then
    echo "WARN: $slug missing cluster tag (expected one of: DEPLOY/DETECT/API/RENDER/HOOKS/STATE/KNOWLEDGE): run check 9 to determine cluster"
  fi
done

Severity: WARN: missing #pitfall tag removes the note from cross-dimension lifecycle queries. Missing cluster tag breaks the cluster detection check (Check 9) and prevents research matching (Check 8).

15. Graph Connectivity (WARN)

Every topics/pitfalls/*.md note must have a minimum of 2 outbound wikilinks. The lifecycle chain requires at least one inbound link from an idea (via addresses_pitfalls) and one link to the parent project.

Requirements:

Minimum 2 outbound [...](/...) wikilinks in the body
Must be referenced by at least 1 idea via addresses_pitfalls in ideas/*.md (enforced as WARN, not ERROR: new pitfalls may not yet have ideas)
Must link to the parent project ([projects/{slug}/_index](/projects/{slug}/_index) preferred, [{slug}](/{slug}) accepted)
Definition links for jargon terms in the note body are encouraged (link to a topics/ note for technical terms like file locking, shadow DOM, behavioral detection)
Cross-dimensional link: at least one link to an experiment, skill, or research note is encouraged where applicable

Bash check:

VAULT="${vault_path:-$HOME/vault}"
# Build reverse index: pitfall slug -> idea count
declare -A idea_coverage
for idea in "$VAULT"/ideas/*.md; do
  while IFS= read -r line; do
    if echo "$line" | grep -qE 'topics/pitfalls/'; then
      p_slug=$(echo "$line" | grep -oE 'topics/pitfalls/[^]]+' | sed 's|topics/pitfalls/||')
      idea_coverage[$p_slug]=$(( ${idea_coverage[$p_slug]:-0} + 1 ))
    fi
  done < "$idea"
done

for f in "$VAULT"/topics/pitfalls/*.md; do
  [ -f "$f" ] || continue
  slug=$(basename "$f" .md)
  proj=$(grep '^project:' "$f" | head -1 | sed 's/project: *//')
  # Count total wikilinks
  link_count=$(grep -oE '\[\[^\](/^\)+\]\]' "$f" | wc -l | tr -d ' ')
  [ "$link_count" -lt 2 ] && echo "WARN: $slug has $link_count wikilink(s) (min 2): low graph connectivity"
  # Check project backlink
  if ! grep -qE "\[\[projects/${proj}/_index\]\]|\[\[${proj}\]\]" "$f"; then
    echo "WARN: $slug missing project backlink ([projects/${proj}/_index](/projects/${proj}/_index) or [${proj}](/${proj}))"
  fi
  # Check inbound idea link
  idea_count=${idea_coverage[$slug]:-0}
  [ "$idea_count" -eq 0 ] && echo "WARN: $slug has no idea referencing it via addresses_pitfalls: lifecycle chain not started"
  # INFO: cross-dimensional link
  if ! grep -qE '\[\[(experiments|skills|research)/' "$f"; then
    echo "INFO: $slug has no cross-dimensional link (experiments/skills/research): add one when an experiment or resolution exists"
  fi
done

Severity: WARN for fewer than 2 outbound links, missing project backlink, or no idea covering the pitfall; INFO for missing cross-dimensional link.

16. PII and Secrets Scan (ERROR/WARN)

Every pitfall note must be free of real API keys, database credentials, and personal contact information. Env var names in documentation are fine; env var values are not. See _config/pii-rules.yaml for the canonical blocked and redact pattern lists.

Blocked patterns (ERROR): API keys (sk-, AIza, ghp_, xoxb-, AKIA, whsec_, sk_live_, sk_test_, re_), database connection strings with embedded passwords, env var assignments with real values.

Redact patterns (WARN): Personal email addresses, GCP project numbers, phone numbers, SSNs, credit card numbers, street addresses. Replace with placeholders (e.g., <personal-email>, <gcp-project-id>).

VAULT=~/vault
for f in "$VAULT"/topics/pitfalls/*.md; do
  [ -f "$f" ] || continue
  slug=$(basename "$f")
  if grep -qE 'sk-[a-zA-Z0-9]{20,}|AIza[a-zA-Z0-9_-]{35}|ghp_[a-zA-Z0-9]{36}|xoxb-|AKIA[A-Z0-9]{16}|whsec_|sk_live_|sk_test_|re_[a-zA-Z0-9]{20,}' "$f"; then
    echo "ERROR (PII): $slug contains a blocked secret pattern"
  fi
  if grep -qE 'postgres(ql)?://[^:]+:[^@]+@|mongodb(\+srv)?://[^:]+:[^@]+@' "$f"; then
    echo "ERROR (PII): $slug contains a database connection string with credentials"
  fi
  if grep -qE 'alexdgutierreza@gmail\.com|616560719313' "$f"; then
    echo "WARN (PII): $slug contains personal email or GCP project number: replace with placeholder"
  fi
  if grep -qE '\b[0-9]{3}-[0-9]{2}-[0-9]{4}\b' "$f"; then
    echo "ERROR (PII): $slug may contain an SSN"
  fi
done

Severity: ERROR for blocked secrets (must fix before commit). WARN for redact patterns (fix before any content flows to public destinations).

Reference: _config/pii-rules.yaml: canonical pattern list and zone-based escalation rules.

What Complete Looks Like

A fully healthy Pitfalls dimension meets all of these criteria:

Metric	Target	How to Measure
Tag consistency	21/21 have `#pitfall`	grep `pitfall` in each file’s `tags` array
`created` field	21/21 present	parse frontmatter
`project` field	21/21 present	parse frontmatter
Body structure	21/21 have all 3 narrative sections	heading scan
Project backlinks	21/21 link to `[projects/{slug}/_index](/projects/{slug}/_index)`	body wikilink scan
Idea coverage	>=50% (11/21) referenced by an idea	cross-reference `addresses_pitfalls` arrays
Skill resolution	all `resolved` pitfalls appear in a skill’s `resolved_pitfalls`	cross-reference skill frontmatter
Wikilink density	21/21 have >=2 outbound wikilinks	`[...](/...)` count per file
No orphans	0 pitfalls with zero inbound links	vault-wide backlink scan
Lint passes	`vault-bootstrap lint` reports 0 errors for pitfalls	run linter
Research matches surfaced	>=50% of fresh pitfalls have 1+ research match	Domain + keyword scan
Cluster membership	100% of pitfalls belong to a named cluster	Pattern grouping
Cross-project systemic	All 3+ project clusters have research coverage	Correlation matrix
Chain completion	>=50% (15/29) have at least idea link	Chain status tracking

Stretch goals:

Every pitfall has a ## Related section with categorized links
Pitfalls cluster by project (visible in Obsidian graph view with project-color grouping)
Temporal coverage: pitfalls exist for every project that has shipped code (check against projects/ directory)

How to Fill Gaps

Stamp last_audited

Every note you audit or create must have last_audited: YYYY-MM-DD in its frontmatter (today’s date). This enables vault-bootstrap stale to detect notes whose source files changed after the last audit. If the field is missing, add it. If it exists, update it to today.

Missing `#pitfall` tag

# Add to the tags array in frontmatter
tags:
  - existing-tag
  - pitfall  # <-- add this

Missing `project` field

Read the pitfall body for project context clues
Check the filename: many are prefixed with the project name (e.g., pirate-ship-*.md)
Cross-reference with git log: git log --all --oneline --grep="{pitfall-keyword}" in the relevant project repo
Add project: {slug} to frontmatter

Missing project backlink

Add a ## Related section at the bottom of the pitfall (if not already present), or append to the existing one:

## Related

- [projects/{project-slug}/_index](/projects/{project-slug}/_index): parent project

Missing bidirectional idea link

When an idea’s addresses_pitfalls references a pitfall, add a forward-link in the pitfall:

## Related

- [ideas/{idea-slug}](/ideas/{idea-slug}): idea that addresses this pitfall

Missing bidirectional skill link

When a skill’s resolved_pitfalls references a pitfall, add a forward-link in the pitfall and update the pitfall’s status:

status: resolved

## Related

- [skills/{skill-slug}](/skills/{skill-slug}): skill that resolved this pitfall

Low wikilink density (< 2 links)

Scan for related content:

Search other pitfalls for the same project value: link siblings
Search experiments for mentions of the pitfall’s topic
Check if the pitfall’s root cause applies to other projects: add cross-project links
Link to external resources if the pitfall involves a framework/library (Astro, Railway, Vercel, etc.)

Uncaptured pitfalls (expanding coverage)

Mine these sources for pitfalls not yet documented:

Git logs: git log --all --oneline | grep -iE 'fix|bug|revert|workaround|hotfix|patch' in each project repo
Claude Code session history: ~/.claude/projects/: search for sessions with repeated failures or pivots
Deployment logs: Vercel, Railway, Cloudflare dashboards for failed deploys
Issue trackers: GitHub Issues marked as bugs or incidents
CLAUDE.md files: Project root docs often mention known issues and workarounds

Research-informed idea creation

When a fresh pitfall matches research with a proven methodology (from check 8):

Create an idea note in ideas/ linking the pitfall (addresses_pitfalls) and the research note (source)
If the research methodology is experiment-ready (has concrete steps, parameters, or a pattern to apply), create an experiment stub in experiments/{project}/
Update the pitfall’s status from fresh to addressed
Add bidirectional links: pitfall -> idea, idea -> pitfall

Template for research-informed idea:

---
type: idea
title: "Apply {research-methodology} to address {pitfall-pattern}"
target_projects:
  - "[projects/{project}/_index](/projects/{project}/_index)"
addresses_pitfalls:
  - "[topics/pitfalls/{pitfall-slug}](/topics/pitfalls/{pitfall-slug})"
research_source: "[research/{research-slug}](/research/{research-slug})"
priority_score: {computed from cross-project count x severity}
source: audit
status: captured
promoted_to: null
created: {today}
tags: [{domain}]
last_audited: 2026-04-07
---

Cluster-based gap filling

When check 9 identifies a cluster with 3+ members and LOW research coverage:

Search research dimension for methodology notes that apply to the cluster pattern (not individual pitfalls)
If no matching research exists, flag as “research debt” in the audit report
Consider creating a meta-pitfall topic note: topics/anti-pattern-{cluster-name}.md that aggregates the cluster with structure: ## Pattern, ## Why It Happens, ## How to Detect, ## How to Fix, ## Instances

Updating pitfall status

fresh: captured but not yet addressed by any idea or experiment
addressed: at least one idea references it in addresses_pitfalls
resolved: a skill lists it in resolved_pitfalls, meaning the lesson is codified and reusable

Quality Checks

Automated checks (run after every audit)

Lint pass: vault-bootstrap lint must report 0 errors for all files in topics/pitfalls/. The linter enforces created field presence and type: topic for all topic files.
Dashboard render: The Pitfalls-Breakthroughs dashboard (if configured) should render all 21 pitfalls. If a pitfall is missing from the dashboard, it likely has a frontmatter parse error.

Wikilink integrity: Every [...](/...) target in a pitfall body must resolve to an existing file. Run a dead-link scan:

# Find all wikilinks in pitfall files and check targets exist
grep -ohP '\[\[([^\]]+)\]\]' topics/pitfalls/*.md | sort -u | while read link; do
  target=$(echo "$link" | sed 's/\[\[//;s/\]\]//')
  if [ ! -f "${target}.md" ] && [ ! -f "${target}" ]; then
    echo "DEAD LINK: $link"
  fi
done

Tag uniqueness: No pitfall should have duplicate tags. Parse each tags array and flag duplicates.

Manual checks (run quarterly)

Narrative quality: Skim each pitfall’s ## Root Cause section. It should explain WHY, not just restate WHAT. A root cause that just says “it was broken” is incomplete: it should identify the architectural or process failure.
Actionability of lessons: Each ## Lesson Learned (or equivalent) should contain a concrete, reusable rule: not just “be more careful.” Good: “Default to progressive enhancement in Astro: HTML/CSS first, then client:load directives.” Bad: “Test more before deploying.”
Temporal coverage: Compare the list of projects in projects/ against pitfalls’ project fields. Every project that has shipped production code should have at least one pitfall. If a project has zero pitfalls, either it is too new or its failures are undocumented.
Lifecycle chain integrity: Trace the full chain for at least 3 pitfalls:
- Pitfall exists and is tagged → check
- An idea references it in addresses_pitfalls → check
- The idea was promoted to an experiment → check promoted_to field
- The experiment produced results → check experiment notes
- A skill was extracted and lists it in resolved_pitfalls → check
- If all steps pass, the pitfall is fully resolved through the lifecycle

Audit output format

Produce a summary table after each audit run:

| Check | Pass | Fail | Coverage |
|-------|------|------|----------|
| #pitfall tag | 21 | 0 | 100% |
| created field | 21 | 0 | 100% |
| project field | 20 | 1 | 95% |
| Body structure | 19 | 2 | 90% |
| Project backlink | 15 | 6 | 71% |
| Idea forward-link | 3 | 18 | 14% |
| Skill resolution | 2 | 0 | 100%* |
| Wikilink density >=2 | 18 | 3 | 86% |
| No orphans | 17 | 4 | 81% |
| **Overall** | | | **82%** |

*Skill resolution is measured against pitfalls with status: resolved only.

Priority for remediation: P0 (frontmatter, tags, project backlink) → P1 (body structure, idea links) → P2 (wikilink density, orphan elimination).

Visual Enrichment

When this audit produces output that benefits from visualization:

Finding Type	Tool	Specification
Pitfall frequency by project	R viz (skills/r-visualization-pipeline)	Family: `DST`, Template: Journal
Pitfall creation rate	R viz (skills/r-visualization-pipeline)	Family: `TS`, Template: Journal
Pitfall-to-idea-to-experiment chain	Figma MCP (`generate_diagram`)	Type: Flowchart

See topics/visual-output-routing for the full routing decision framework.

Self-improvement context: This audit skill implements the lint cycle of Pattern 4 (Compiler Wiki) from skills/self-improving-agent-patterns. The vault’s audit skills collectively form the lint+heal loop described in research/2026-04-02-karpathy-llm-knowledge-base-pattern.

Post-Audit Visual Enrichment Trigger

After completing all quality checks, dispatch a Sonnet subagent to generate visuals for this dimension:

[!tip] Auto-generate visuals after audit Use the Agent tool to dispatch a subagent that runs skills/wiki-visual-enrichment for the pitfalls dimension. This generates R charts and Figma diagrams based on the Visual Enrichment specifications above. Content-hash dedup ensures only changed articles get new visuals.