Stock dashboard (index.html):
- Replace all [DEMO]/demo badges on warehouse data with "FS.com" source labels
(data was always real scraper data, never demo in the DB)
- Update subtitle: "Scraper-Lagermengen: DEMO DATA" → "Wettbewerber-Marktdaten"
- "Recently Restocked" badge: DEMO DATA → SCRAPER DATA
Switch detail (queries.ts):
- Fix getFlexoptixSuggestions: wavelength_nm → wavelength_tx_nm,
price_verified_usd → street_price_usd (column mismatch with live schema)
- DS5000 and other OSFP switches now show all 62 Flexoptix OSFP transceivers
with direct shop links in the detail modal
Hot Topics (hot-topics.ts):
- NOG Talks + News Article clusters now fetch summary/mentioned_vendors/
mentioned_products/mentioned_standards from news_articles table
- description field builds bullet-point list per article with summaries,
key vendors/standards (vs. 3 bare titles joined with "|" before)
- buildTopicBriefing() rewritten as structured LLM document with sections:
Market Signals (bullets), Recommended Angle, Market Context (buy signal,
technologies, impact horizon), Writing Instructions (600-900 words,
actionable, opinionated, no generic summaries)
- DELETE 2133 rows from reorder_signals WHERE is_demo_data = true
- atgbics.ts: add upsertStockObservation (confidence=1, binary available
boolean from Shopify API; quantityAvailable 1/0 for in/out stock)
- optcore.ts: add upsertStockObservation (confidence=1, WooCommerce text
stock level parsed via parseStockLevel; quantityAvailable 1/0)
- Both scrapers already run every 2h on Erik scheduler
- FS.com: already captures full warehouse breakdown (DE+Global+backorder)
3x/day from Mac (02:00/10:00/18:00) at confidence=3 — no change needed
- QSFPTEK: already captures real quantities at confidence=2 — no change
- sfpcables/prolabs/wiitek: no meaningful stock signal, not modified
- sql/108: normalize form_factor across all vendors (SFP-Plus → SFP+, etc.)
and round speed_gbps for consistent matching
- sql/109: document 30→90 day matcher window change
- robots/catalog-reconcile.ts: new bulk-reconcile robot — matches all
Flexoptix products against all competitors without 30-day time limit
- scheduler.ts: register catalog:reconcile job (monthly + on-demand),
fix nightly matcher 30→90 day window, UPPER() form_factor matching,
ROUND() speed_gbps matching
Fixes: ATGBICS/NADDOD/10Gtek/ShopFiber24 had 0 Flexoptix equivalences
due to stale price_observations being filtered out. Expected coverage
improvement: 22% → 45-60% after first reconcile run.
CODEX-TASK-flexoptix-reference-matching.md — comprehensive plan to fix
zero-match gap for ATGBICS/NADDOD/10Gtek/ShopFiber24 (8.260+ products
with 0 Flexoptix equivalences).
Root cause: 30-day price_observation window excludes vendors whose
scrapers ran >30 days ago. Solution: catalog-reconcile robot (full
bulk match, no time limit), form_factor normalization (SQL 108),
30→90 day window fix in nightly matcher, on-demand API endpoint.
Expected: coverage from 22% → 45-60% after one reconcile run.
End-to-end deployment record for the 127→227 article corpus expansion:
- Gitea push (transceiver-db@f311e08)
- Magatama pool reconciliation (magatama@0e42de9)
- Erik sync via scp
- RunPod training trigger (job 0141303c, lane fo_blogllm, 500 iters)
Documents the continuous evolution plan (per-article + quarterly refresh)
and quality gates going forward.
Rewrite Python code samples to read credentials from environment
via load_credentials_from_env() helper rather than literal kwargs.
Avoids triggering pre-commit secrets scanners on 'password=' pattern
in training data while improving security guidance shown to readers.
Replace literal 'apipass'/'admin' placeholder credentials with
explicit <USER>/<PASSWORD> placeholders. Prevents false-positive
secrets scan detection in Magatama pre-commit hooks.
No real credentials were ever present — these are training
data code examples showing API connection patterns.
- index-pi.ts: removed Playwright scrapers (FS.COM, eBay enricher, switch assets)
added NADDOD (fetch-based, benefits from residential IP)
now 32 fetch-only queues safe for ARM/Pi without Chromium
- index-fs-only.ts: new dedicated FS.COM + NADDOD worker for Erik
routes through Pi SOCKS5 via PROXY_URLS=socks5://10.10.0.6:1080
Crawlee ProxyConfiguration automatically applies to Playwright crawler
- pi-scraper-setup.sh: removed inline index-pi.ts override (repo version now authoritative)
- CODEX-TASK-pi-scraper-deploy.md: full 9-step Codex spec for Pi fleet setup
covers WireGuard keypair, Erik peer config, setup script, ecosystem.config.js
- CODEX-TASK-zero-manual-review.md: deterministic equivalence matcher spec
