Rene Fichtmueller
6be2c131d3
fix: add PID lock to run-fs-scraper-mac.sh — prevent simultaneous instances
...
Adds /tmp/tip-fs-scraper.lock PID file to prevent launchd from running
a second instance while the previous one is still active (e.g. 2am
schedule fires, runs past 10am when launchd fires again). Without this,
concurrent instances caused rmSync(storage-fs-phase1) in one instance
to delete the Crawlee storage dir while the other was still using it,
resulting in ENOENT crashes.
2026-04-18 02:43:28 +02:00
Rene Fichtmueller
5b35b2b8be
feat(scraper+api): warehouse stock data pipeline — FS.com v2, SmartOptics v2, Stock API
...
Scraper changes:
- fs-com.ts v2: Playwright stealth patches + www.fs.com/de/ URL fix (de.fs.com DNS NXDOMAIN).
Extracts DE-Lager, Global-Lager, Nachlieferung, units_sold, compatible_brands, price_net.
Mac-side runner (run-fs-scraper-mac.sh) via SSH tunnel for residential IP access.
Fast-fail connectivity check on datacenter IPs that are blocked by Cloudflare.
- smartoptics.ts v2: WooCommerce REST API fallback + 8 catalog categories + relative URL fix.
Was finding only 8 products, now discovers 18+ with multi-category crawl.
DB layer:
- db.ts: add upsertStockObservation() — writes 10 new stock_observations columns
(warehouse_de_qty, warehouse_global_qty, backorder_qty, units_sold, compatible_brands,
price_net, product_url, delivery dates) with dedup check.
API:
- routes/stock.ts: GET /api/stock, /api/stock/summary, /api/stock/:id
Warehouse breakdowns per transceiver/vendor with top-sellers and vendor summary.
- routes/review.ts: equivalence review queue (approve/reject/bulk-approve).
- index.ts: register /api/stock and /api/review routes.
Dashboard:
- index.html: 🏭 Stock tab with stat cards (DE-Lager, Global-Lager, Nachlieferung totals),
top-sellers table, vendor breakdown, recently-restocked events, part-number lookup.
SQL migrations:
- 034: blog-review-tag, 035: price-observations is_anomalous, 036: transceiver-equivalences.
2026-04-17 10:45:59 +02:00
