From b3eff15fc402e723acbb2092ed9560fdb8a74824 Mon Sep 17 00:00:00 2001 From: Rene Fichtmueller Date: Sat, 18 Apr 2026 03:25:02 +0200 Subject: [PATCH] =?UTF-8?q?chore:=20changelog=20=E2=80=94=20daemon=20stabi?= =?UTF-8?q?lity,=20ATGBICS=20Playwright,=20health=20monitor=20accuracy?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CHANGELOG_PENDING.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG_PENDING.md b/CHANGELOG_PENDING.md index 03e85a2..0aa5e2f 100644 --- a/CHANGELOG_PENDING.md +++ b/CHANGELOG_PENDING.md @@ -6,6 +6,7 @@ Types: FEAT · FIX · UI · DATA · AI · INFRA {"d":"2026-04-18","t":"FIX","m":"FS.com Mac scraper: suppress Crawlee post-run ENOENT unhandledRejection — Crawlee's FileSystemStorage fires a final _isTaskReadyFunction call after run() resolves, reading a request .json that was already processed/cleaned-up. This ENOENT triggered process.exit(1) before Phase 2 completed, causing 7 days of missing FS.com price data. Fixed: targeted unhandledRejection handler in require.main block swallows ENOENT from request_queues paths while re-raising real errors."} {"d":"2026-04-18","t":"FIX","m":"FS.com Mac scraper: PID lock (/tmp/tip-fs-scraper.lock) added to run-fs-scraper-mac.sh — prevents concurrent instances when launchd 2am fire overlaps with a still-running earlier run. Previous concurrent instances caused rmSync(storage-fs-phase1) race (one instance deletes the storage dir while another is using it), crashing Phase 2."} {"d":"2026-04-18","t":"FIX","m":"Scraper health monitor: tiered alerts replacing false-positive 6h threshold. Old: fired every 3h for any vendor with 0 new prices (including stable prices). New: 🔴 CRITICAL (last price >7 days), 🟡 WARNING (last price 48h-7 days), ✅ STABLE (0 new prices but last price ≤48h — content hash dedup, scraper running OK). Shows pg-boss job state+time for faster root-cause."} +{"d":"2026-04-18","t":"FIX","m":"Daemon stability: global unhandledRejection handler in scheduler index.ts — Crawlee post-run lock-file ENOENT (request_queues path) was crashing the daemon process (process.exit(1)) which killed all active pg-boss jobs and triggered PM2 restart loops. Fix: swallow ENOENT from request_queues paths at the scheduler level; re-raise all other rejections. Also: FS.com scheduler worker now skips (SKIP_FS_SCRAPER=true env var) on Erik where Cloudflare WAF blocks datacenter IPs; Mac launchd handles FS.com scraping. Created missing Crawlee storage dirs: storage-fs-phase1, storage-fs-phase2, storage-ebay-transceivers, storage-fs. Health monitor pg-boss lookup extended from 12h → 26h; added completedMap; vendors with recent job completion + historical prices classified as STABLE (not CRITICAL) — eliminates ATGBICS/Fluxlight false-positive alerts."} {"d":"2026-04-18","t":"FIX","m":"Playwright chromium_headless_shell-1217 installed on Erik (/opt/playwright-browsers/): ATGBICS and FS.com PlaywrightCrawler were throwing BrowserLaunchError on every run since Crawlee browser-pool requires chromium_headless_shell binary, not regular chromium. Fixed by: PLAYWRIGHT_BROWSERS_PATH=/opt/playwright-browsers npx playwright install chromium --with-deps on Erik."} {"d":"2026-04-18","t":"FIX","m":"Crawlee withIsolatedStorage global env-var race condition eliminated: scheduler.ts removed all withIsolatedStorage() wrappers (were mutating process.env.CRAWLEE_STORAGE_DIR globally, causing concurrent scrapers to pick up wrong storage dirs). All Crawlee scrapers now use makeCrawleeConfig(name) instance-level Configuration. fs-com.ts migrated to fs-phase1/fs-phase2 storage names with rmSync cleanup before each phase. switch-assets-crawler.ts and switch-assets-playwright.ts now pass makeCrawleeConfig. Fixed: ATGBICS, community-issues, market-intel, FS.com."} {"d":"2026-04-18","t":"FIX","m":"FS.com Mac launchd scraper (org.tip.fs-scraper): was failing with exit code 126 (Operation not permitted) because macOS TCC blocks launchd agents from accessing ~/Desktop/Claude Code/ path. Fixed: script moved to ~/.tip/run-fs-scraper-mac.sh, plist WorkingDirectory changed to ~/.tip. FS.com is now scraping from residential Mac IP again."}