Close the missing feedback loop. Detection (v_scraper_health, v_data_quality_alerts,
check_switch_quality) was accurate but inert; dispatch was blind fixed-cadence
(boss.send() is never called from any health path). tip-dqo is a separate, read-mostly
control plane that closes DETECT -> PRIORITIZE -> DISPATCH -> VERIFY -> RECHECK against
observation-table freshness (never pg-boss job state), with DB-enforced guardrails.
- sql/129: source_registry (signal->queue map + per-source SLA + guardrail contract +
global kill-switch), self_heal_dispatch append-only ledger, scraper_run_log,
acknowledged_exceptions, v_dqo_targets ranked view.
- orchestrator/: control-loop, self-heal-guards (pure, unit-tested), escalation-sink
(ntfy + no-op fallback), index entrypoint. Reuses the verification-robots dispatch
idiom (extracted shared dispatchQueues) and the quality views as-is.
- Guardrails: kill-switch, plan-mode default, per-source cooldown/budget/backoff/
circuit-breaker, load-guard, erik-safe profile cap, delta-gated escalation. Root-cause
routing escalates code_fix/blocked (e.g. ATGBICS silent stock-write break) to a human
instead of looping retries; only staleness is auto-dispatched.
- security: remove hardcoded 'tip_dev_2026' DB password fallback across 6 sites ->
fail-closed requireDbPassword() / dbConnectionString().
SELF_HEAL_PLAN_ONLY defaults true (dispatches nothing; writes plan rows for review).
Verified against the live DB: detects ATGBICS-stock dead/54d -> escalate, Flexoptix-stock
stale/30d -> plan-dispatch; guards + ledger correct; 8/8 guard unit tests pass.
One article (The Next Platform) had published_at 2026-07-15 while scraped
2026-06-19 -- an article cannot be published after it was scraped. One-time
clamp to the scrape date + BEFORE INSERT/UPDATE trigger to prevent recurrence.
- trg_compat_speed_guard: BEFORE INSERT/UPDATE on compatibility forces
algorithmic (spec_match/community) rows whose transceiver is faster than the
switch max port speed to status=incompatible. Curated rows never touched.
Makes the "no physically-impossible compatibility" invariant structural, so
it survives a scraper regression.
- check_switch_quality(): on-demand proof; CI gate is
`select bool_and(pass) from check_switch_quality() where enforced` (must be
true). Data-source gaps (coding coverage, speed<=0, datasheet) tracked as
enforced=false so they do not fail the gate before the data exists.
Root cause: the form-factor fallback in flexoptix-compat matched transceivers by
form factor only, ignoring speed, so a 400G OSFP port matched every OSFP module
incl. 1.6T (5686+ impossible rows across 266 switches).
- flexoptix-compat: fallback now filters transceiver speed <= switch max port
speed (derived from ports_config + stored max_speed_gbps). Prevents regeneration.
- sql/125: evidence-based effective-ceiling (stored/ports/curated) views;
flag 5811 impossible spec_match rows status=incompatible (curated rows never
touched, understated switches never false-flagged); backfill max_speed_gbps for
327 switches from ports_config; re-derive speed for 306 speed<=0 transceivers
(form-factor capped, high-confidence only). v_switch_quality_alerts is the task
list; v_switch_understated_ports surfaces switches needing datasheet enrichment.
Reading packages/scraper showed QSFPTEK/ATGBICS populate quantity_available
100%; they simply lack FS.com-only units_sold/warehouse-split. The 123 alert
keyed on those FS.com fields and false-positived QSFPTEK. Rekey to
quantity_available OR warehouse_qty so the alert list drains to empty; surface
pct_qty_available/pct_vendor_ts in v_stock_completeness.
FS.com renders per-warehouse availability as a bare word-date on each line
("2 Stk. im DE-Lager, 8. Juli 2026 Lieferbar"). The old regex only matched a
"Lieferung:"/"erwartet:" prefix or numeric dd.mm.yyyy, so warehouse_de/global
delivery_date + backorder_estimated_date were 0% populated across 85k rows, and
lead_time_days was never written at all.
- new fs-com-parse.ts: pure, unit-tested parsers (13 node:test cases).
- extractDeliveryDate finds the bare word/numeric date per warehouse, bounded to
its own line so a date never bleeds to a neighbouring warehouse.
- computeLeadTimeDays = min(delivery_date - today), floored at 0; threaded into
upsertStockObservation (now writes lead_time_days) and upsertPriceObservation.
- parseGermanQty: K/M abbreviations now treat '.' as a decimal ("4.8K"=4800,
"1.4M"=1.4M) instead of stripping it (was "1.4M"->14,000,000 — the outlier).
- units_sold sanity gate at ingest (FS_MAX_UNITS_SOLD, default 2,000,000).
- sql/122: purge impossible historical units_sold, re-document columns as collected.
sql/119: product_type enum (module|dac|aoc|aec|switch|nic|accessory) computed by
trigger trg_transceivers_biu from part_number/category/reach on every insert/update.
Backfilled 41008 rows (36284 module, 2791 dac, 1835 aoc, 29 aec, 27 accessory,
26 switch, 16 nic). Cables and switches/NICs no longer counted as modules;
BASE-T/RJ45 copper modules protected from cable demotion; AOC/DAC keyword beats
the optical-token guard. Consumers filter product_type=module.
sql/120: conservative form_factor recovery. Corrects only provably-wrong rows
(form_factors.max_speed < speed_gbps) via an explicit title token; 87 fixed
(e.g. 400G cables mislabelled SFP+ to QSFP-DD), 947 opaque rows left untouched
(never guessed). Scraped original preserved in new column form_factor_raw.
sql/121: document lead_time_days as not-collected (0 percent populated on all
time-series tables) so no consumer fabricates a default.
scraper/utils/db.ts: note that product_type/form_factor are trigger-maintained.
- sql/108: normalize form_factor across all vendors (SFP-Plus → SFP+, etc.)
and round speed_gbps for consistent matching
- sql/109: document 30→90 day matcher window change
- robots/catalog-reconcile.ts: new bulk-reconcile robot — matches all
Flexoptix products against all competitors without 30-day time limit
- scheduler.ts: register catalog:reconcile job (monthly + on-demand),
fix nightly matcher 30→90 day window, UPPER() form_factor matching,
ROUND() speed_gbps matching
Fixes: ATGBICS/NADDOD/10Gtek/ShopFiber24 had 0 Flexoptix equivalences
due to stale price_observations being filtered out. Expected coverage
improvement: 22% → 45-60% after first reconcile run.
- Migration 100: adds `description` column to standards (plain-language
DE·EN for non-technical colleagues), fills all 63 standards incl.
complete 200G tier (SR4/DR4/FR4/LR4/ER4/CR4), copper DAC variants,
PON family (GPON/XG-PON1/NG-PON2/25G-PON), 1.6T emerging standard
- Migration 101: new form_factors table — 20 entries covering SFP family
(SFP→SFP112), QSFP family (QSFP+→QSFP-DD800), OSFP family (OSFP→OSFP224),
CFP family, legacy XFP/CXP with full_name, speed, channels, status,
supersedes chain, and bilingual plain-language descriptions
- GET /api/form-factors — new endpoint, returns all form factors with
transceiver_count join
- GET /api/form-factors/:name — single form factor detail
Dashboard Standards tab:
- DB description shown as subtitle in standards table rows
- Full DE + EN description in standard detail panel
- New Form Factors grid section with status badges, speed, channel info,
family color coding, supersedes chain
- openFormFactorDetail() panel with full specs + transceiver link
- Search extended to match description + notes fields
- Migration 099: flexoptix_internal_demand table with RLS + v_demand_by_speed view
- Import script: AES-256-CBC decrypt → parse 8585 SKUs → upsert with velocity class
- 279 SKUs cross-referenced to transceiver catalog; 1288 with real demand data
- New /api/internal/demand/* routes (by-speed, velocity, hype-weights, forecast-input)
— protected by JWT auth + localhost/LAN IP restriction middleware
- Forecast engine calibrated with real Flexoptix run-rates (demand_calibrated flag)
- Dashboard: real Flexoptix Sales Velocity panel replaces DEMO DATA in Warehouse tab
with momentum indicators, velocity class breakdown, trend arrows
- Security: data stays on private server; RLS enforces is_internal=TRUE at DB layer
7 models: AS7312-54X, AS7312-54XS, AS7326-56X, AS7716-32X, AS7816-64X,
AS9716-32D, AS7512-32X. All from SONiC HCL Accton/Edgecore vendor.
Sources: edge-core.com official WP CDN + stordis.com + epsglobal.com.
Also adds scripts/apply-pending-migrations.sh for Erik DB catchup.
A9K-4HG-FLEX-FC, A9K-8X100G-LB-TR, A9K-4X100GE base: all share identical
hardware with previously-covered -TR/-SE siblings. FC/SE/TR are Cisco
software license suffixes on the same physical line card. Coverage: 595 → 598.
