6 Commits

Author SHA1 Message Date
Rene Fichtmueller
9bc1292bac fix: add rate-limiting semaphores to audit script
The audit script was flooding RIPE Stat and PeeringDB with unthrottled
parallel requests, causing 429 rate-limits that resulted in auth=0
false negatives (inflating the failure count).

Changes:
- Added threading.Semaphore for RIPE Stat (max 3) and PeeringDB (max 2)
- Added retry logic to _fetch_ripe (was fire-and-forget)
- Increased PDB retries from 2 to 3 with longer backoff (2s, 4s, 6s)
- Increased ASN stagger from 2s to 3s

Results: Accuracy 84% -> 87% (trend: 77% -> 87%, +10%)
2026-03-30 07:22:09 +02:00
Rene Fichtmueller
98b5cb1843 fix: prevent rate-limit 0-values under concurrent load
server.js: fetchPeeringDBWithRetry now does 3 attempts with exponential
backoff (2s, 5s) instead of 1 retry at 1.5s. Under audit load (9+
concurrent PDB requests), the longer delays let rate limits clear.

audit.py: stagger ASN submissions by 2s so PeerCortex's internal PDB
requests don't all fire simultaneously. Nightly audit takes ~8min
instead of 5min — acceptable for a midnight cron job.
2026-03-28 18:26:22 +13:00
Rene Fichtmueller
a9ee94466e perf: reduce audit concurrency to 3 to avoid PDB hammering 2026-03-28 15:16:42 +13:00
Rene Fichtmueller
711b89a09e feat: persistent known_issues tracking in ASN registry
When the same field fails 2+ consecutive audit runs, a known_issue
entry is written into the ASN's registry profile with:
- field name, description of what's wrong
- first_seen / last_seen dates, occurrence count
- last auth vs PC values
- status: open (stays until PeerCortex data matches)

Report shows KNOWN ISSUES section (all open issues across registry).
Issues auto-resolve when the ASN passes, or partially resolve when
individual fields are fixed. Also stores ASN name in registry.
2026-03-28 14:02:33 +13:00
Rene Fichtmueller
87ce2ed36a fix: audit.py — distinguish PDB fetch failure from 'not in PDB'
- pdb_present=True/False/None three-state (None = fetch failed)
- Skip IX/fac comparison when PDB fetch failed (avoid false positives)
- Add retry with backoff to _fetch_pdb (2 retries, 1.5s/3s delays)
- Fix datetime.utcnow() deprecation warning
- Report PDB fetch failures separately in summary
2026-03-28 13:22:25 +13:00
Rene Fichtmueller
2b0ba18e40 feat: daily accuracy audit system with rotating ASN batches
- audit/audit.py: nightly audit runs at midnight via cron
  * Rotates through all tracked ASNs (priority: errors > never > oldest)
  * Compares PeerCortex against RIPE Stat + PeeringDB (authoritative)
  * Uses PeeringDB API key (no rate limits)
  * Marks ASNs without PeeringDB entry as peeringdb_absent (fac=0 correct)
  * Self-heal retry on timeout
  * Tracks accuracy trend over time
  * JSON registry + daily reports + human-readable latest_report.txt
- audit/deploy_audit.sh: one-shot setup script (PM2 env fix + cron)
- .gitignore: exclude ecosystem.config.js (contains env secrets)
2026-03-28 12:50:52 +13:00