- Install better-sqlite3 for zero-latency local queries
- queryPeeringDBLocal() handles all major PDB API paths locally:
/net?asn=X, /netixlan, /netfac, /fac?id__in=, /ixfac, /ix, /ixlan
- fetchPeeringDB() now tries local SQLite first, falls back to live API
- Eliminates rate limits and reduces P99 response times dramatically
- Local DB synced daily at 03:48 via peeringdb-py cron on Erik
- Graceful fallback: if SQLite missing/corrupt, live API used transparently
- MANRS: replace broken Observatory API with public participants page scraping
(www.manrs.org/netops/participants/), 24h cache, returns pass/fail with member count
- /api/validate: add 'relationships' field (upstreams/downstreams/top_peers)
sourced from RIPE Stat asn-neighbours, no extra API calls needed
- /api/relationships?asn=X: new dedicated endpoint with resolved AS names,
full upstream/downstream/peer lists sorted by power score, 10min cache
- editorial: rebrand 'The ASN Newspaper' → 'The ASN News' across index-editorial.html
Root cause of neighbour=0 for large carriers (AS9002, AS3491, AS12956):
1. RIPE Stat asn-neighbours returns 5000+ entries for Tier-1 carriers,
exceeding the 30s timeout → fetchJSON returns null
2. null was cached in ripeStatCache for 15 minutes (the endpoint TTL)
3. All subsequent requests hit the null cache → perpetual 0 neighbours
Fixes:
- Never cache null results in ripeStatCache (only successful responses)
- Never persist null entries to disk cache
- Increase RIPE Stat timeout from 30s to 45s for prefix/neighbour queries
- Increase RIPE Stat semaphore from 10 to 15 concurrent requests
Verified: AS9002 up=146 down=2702, AS3491 up=90 down=710
- Phase 1: Parse ~400k ROAs from Cloudflare RPKI feed into local store
Eliminates ALL per-prefix RIPE Stat API calls (was 2000+ per lookup)
Binary search validation in <0.1ms instead of 1-20s HTTP roundtrip
Disk persistence (.roa-cache.json) for fast restart
- Phase 2: PeeringDB source cache (L2) for net/netixlan/netfac
6h TTL with LRU eviction (max 5000 entries per type)
Disk persistence (.pdb-source-cache.json) every 30min + SIGTERM
- Phase 3: RIPE Stat semaphore (max 10 concurrent) + response cache
Endpoint-specific TTLs (15min-24h based on change rate)
Max 2000 cached responses, disk persistence
- Phase 4: Extended /api/health with cache status, ASPA adoption metrics
Version bump to 0.6.0
Jittered refresh timers to prevent thundering herd
Graceful shutdown saves all caches
Expected: Audit accuracy 82% -> 95%+, lookup time 90s -> <8s
- index-editorial.html: floating \$_ terminal button (bottom-right)
- macOS-style title bar (traffic light dots), backdrop blur 18px
- Guided wizard: category → message → name → submit
- POST /api/feedback with ASN context auto-filled
- Safe DOM output builder (no innerHTML on user data)
- server.js: feedback API endpoints
- POST /api/feedback — stores entries to feedback.json
- GET /api/feedback?token=... — admin read (token-protected)
- OPTIONS preflight for CORS
- FEEDBACK_TOKEN + FEEDBACK_FILE constants from .env
- Host routing: shell.peercortex.org → shell.html
- public/shell.html: full-screen admin terminal
- login command → token auth via API
- list / list [category] — tabular overview
- show <n> — full entry detail
- stats — bar chart by category + top ASNs
- export — JSON file download
- refresh, logout, clear, help
- Upgrade from Leaflet to MapLibre GL JS 4.7.1 with OpenFreeMap dark base
- Add submarine cable layer (TeleGeography via /api/submarine-cables proxy, 24h cache)
- Add global datacenter layer (PeeringDB all facilities via /api/global-infra proxy)
- Layer toggles: ASN PoPs | Submarine Cables | Global Datacenters
- Dark-themed popup styling matching PeerCortex UI
- Server-side caching for both new data sources (24h TTL)
Root cause of 2.7GB RAM usage and 20+ OOM restart loops:
- server loaded all 825k ROAs from Cloudflare RPKI feed into a JS Map
- Every 10min refresh caused double-memory spike (old + new data) -> OOM kill
Solution:
- Remove rpkiRoaIndex Map, addRoaToIndex(), validateRPKILocal(), ipv4ToInt()
- fetchRpkiAspaFeed() now only loads ASPA objects (~1484, negligible RAM)
- Add validateRPKIWithCache(): calls RIPE Stat API per-prefix with a
5000-entry LRU cache (6h TTL) — same API already used by fetchRPKIPerPrefix()
- Update all 4 call sites: sync .map() -> await Promise.all()
Result: 2.7GB -> ~96MB RAM, no more OOM restarts
new URL() throws ERR_INVALID_URL on malformed inputs like XSS probe
requests (e.g. //brusEYkk%22%3E%3Cscript%3E...). Uncaught exception
caused memory leak and process restarts. Return HTTP 400 instead.
server.js: fetchPeeringDBWithRetry now does 3 attempts with exponential
backoff (2s, 5s) instead of 1 retry at 1.5s. Under audit load (9+
concurrent PDB requests), the longer delays let rate limits clear.
audit.py: stagger ASN submissions by 2s so PeerCortex's internal PDB
requests don't all fire simultaneously. Nightly audit takes ~8min
instead of 5min — acceptable for a midnight cron job.
- Add fetchPeeringDBWithRetry: 1 retry with 1.5s delay on null response
- Add fetchJSONWithRetry: 1 retry for RIPE Stat prefixes + neighbours
- Log HTTP 429 from PeeringDB instead of silently swallowing it
- Add &limit=1000 to netixlan/netfac queries (prevents truncation at 250)
- Fall back to asn= / local_asn= queries when PeeringDB net lookup fails
(previously: netId=null → IX=0, fac=0 for ~22 ASNs)
bgp.he.net scraper:
- Fixed prefix regex: "Prefixes Originated (v4): 147" format
- Fixed peer regex: "BGP Peers Observed (all): 274" format
- Added prefixes_all field
- AS6830: v4=147, v6=9, peers=274 (was all unavailable)
- Prefix cross-check now works: RIPE 151 vs HE 156 = 97% agreement
Peering Recommendations:
- Now filters out already-established peering sessions
- 3 categories: New Opportunities, Already Peering, No Shared IXP
- Uses BGP neighbour data to detect existing sessions
- Shows "Already peering with all top networks" when applicable
Dashboard: Added "Score Breakdown — Why X/100?" section showing:
- Per-check weight, earned points, and reason
- Total calculation with formula explanation
- Data source attribution
- "info" status excluded from scoring (e.g. MANRS API auth)
Security: try-catch around new URL() parser — malformed URLs from
scanner bots (XSS attempts) now return 400 instead of crashing server.
Was causing repeated crashes from automated vulnerability scanners.
- Route Server: threshold lowered from 20 to 10 IX for "bilateral policy" pass.
3-9 IX without RS = "info" (not warning). <3 IX = warning.
AS212635: 19 IX → pass (was warning)
- rDNS: sample size increased from 5 to min(20, total_prefixes)
Better coverage for large networks (AS13335: was 5/5621 = 0.09%)
- IX Route Server: always use asn= query (more reliable than net_id when PDB rate-limits)
AS212635: 0 → 19 IX connections correctly detected
AS212635 score: 98 → 100/100
- Geolocation: global networks (5+ facility countries) now get pass
even when MaxMind has no data (was warning)
- Route Server: uses ASN fallback when PeeringDB net_id unavailable
(was showing "0 IX connections" due to rate limiting)
- IX geocode fallback: CITY_COORDS map + IX_CITY_MAP for 70+ cities
AS49544 (i3D.net/Ubisoft): 100 IX connections correctly detected,
bilateral peering policy recognized, 27-country global presence pass
renderNetworkMap() was missing its closing } after the setTimeout(50)
callback. This caused a SyntaxError that prevented the entire script
from parsing — doLookup was undefined, Lookup button did nothing.
Also added deploy.sh backup script on Erik (auto-backup before restart,
keeps last 20 versions of server.js + index.html).
IXPs without PeeringDB facility coordinates now get geocoded via:
1. City name extraction from IX name (e.g. "France-IX Paris" → Paris)
2. Hard-coded IX ID → city map for 15 well-known IXPs (SwissIX→Zurich etc.)
3. 70+ major networking cities with lat/lon coordinates
AS8283 Coloclue: 9 → 12 IX locations (5 cities: AMS, FRA, Paris, Zurich, Meppel)
AS49544 i3D.net: 100 connections → 20 locations (16 cities worldwide)
- Leaflet map: double requestAnimationFrame after display:none removal
ensures container has real dimensions before L.map() init
- PeeringDB org cache: 24h disk cache (.pdb-org-cache.json) prevents
hammering PeeringDB API on server restarts (was causing 175 restarts)
- Check HTTP status before JSON.parse on PDB responses
- Leaflet.js (CDN) with CartoDB Dark Matter tiles matching Tokyo Night theme
- Cyan markers: facility/datacenter locations with name + city popup
- Orange markers: IX presence with IX name + speed popup
- Purple connecting lines between facilities in the same country
- Coordinates from PeeringDB facility API (batch lookup, chunked)
- IX locations via ixfac association + facility geocoding
- Auto-fit bounds, graceful degradation if no coordinates
- Collapsible card, XSS-safe popups via DOM API
- RPKI cross-check: Cloudflare RPKI feed + RIPE NCC Validator API (5 sample prefixes)
- Prefix cross-check: RIPE Stat vs bgp.he.net count comparison
- Neighbour cross-check: RIPE Stat vs bgp.he.net peer data
- Data Quality badge in dashboard (High/Medium/Low confidence)
- Hover tooltip: "Data Quality Report" with per-source agreement breakdown
- Added BETA tag to site header and version string (v0.5.0-beta)
- All UI text in English
RPKI Validation:
- Validate ALL prefixes (not sample of 10) using local Cloudflare RPKI feed
- Covers all 5 RIRs globally (RIPE, APNIC, ARIN, LACNIC, AFRINIC)
- Indexed ROA lookup (O(bucket) not O(824K)) for instant validation
- AS4739 now correctly shows 446/446 prefixes checked
ASPA Provider Detection:
- Only RIPE Stat "left" neighbours (verified upstreams) used as providers
- AS-path analysis used for frequency confirmation only, not as provider source
- Fixes false provider detection that included peers alongside upstreams
Multi-RIR Support:
- WHOIS/IRR queries all 5 RIR databases via RDAP in parallel
- RPSL validation checks RIPE + APNIC/ARIN/LACNIC/AFRINIC
- AS4739 (APNIC) now correctly found via rdap.apnic.net
Geolocation:
- Anycast/CDN networks (5+ facility countries or Content/NSP type) not flagged
- Only small networks with geo anomalies get warnings
Route Server Scoring:
- Networks with 20+ IX connections and no RS scored as "pass" (bilateral policy)
- Only small networks without RS get warnings
Error Handling:
- ASPA endpoints gracefully handle timeouts (show fallback instead of HTML parse error)
- Frontend checks Content-Type before JSON.parse
Reported by Philip Smith, Richard Steenbergen, Jared Mauch, Chris Malayter
- Compare: all API calls in single parallel batch (was sequential)
- Compare: RPKI sample reduced to 3+3 prefixes with 5s timeout cap
- Compare: response caching (5min TTL)
- Compare: AS name resolution parallel with 3s timeout
- Result: Compare from timeout (>20s) to ~5s first call, <1s cached
- Add response caching (5min TTL for lookups, 10min for ASPA)
- Add 8s timeout to all external API fetches
- RPKI validation: sample max 10 prefixes (5 v4 + 5 v6) instead of 50
- Run all PeeringDB + RIPE Stat calls in single parallel batch
- Resolve AS names in parallel with 3s timeout cap
- Add deploy/ directory with production server.js + index.html dashboard
- Landing page: Tokyo Night dark theme, interactive ASN search
- 15 API endpoints: lookup, aspa, aspa/verify, bgproutes, validate,
compare, peers/find, prefix/detail, ix/detail, topology, whois, health
- Features: RPKI per-prefix, RIPE Atlas probes, Network Health Report,
ASPA RFC verification engine, Provider Relationship Graph
- Query RIPE Atlas API for probes in the looked-up ASN
- Display probe count, connected/disconnected status, anchors
- Expandable probe detail table with links to atlas.ripe.net
- Connection ratio progress bar
- "Host a probe?" prompt for networks without Atlas presence
