Close the missing feedback loop. Detection (v_scraper_health, v_data_quality_alerts,
check_switch_quality) was accurate but inert; dispatch was blind fixed-cadence
(boss.send() is never called from any health path). tip-dqo is a separate, read-mostly
control plane that closes DETECT -> PRIORITIZE -> DISPATCH -> VERIFY -> RECHECK against
observation-table freshness (never pg-boss job state), with DB-enforced guardrails.
- sql/129: source_registry (signal->queue map + per-source SLA + guardrail contract +
global kill-switch), self_heal_dispatch append-only ledger, scraper_run_log,
acknowledged_exceptions, v_dqo_targets ranked view.
- orchestrator/: control-loop, self-heal-guards (pure, unit-tested), escalation-sink
(ntfy + no-op fallback), index entrypoint. Reuses the verification-robots dispatch
idiom (extracted shared dispatchQueues) and the quality views as-is.
- Guardrails: kill-switch, plan-mode default, per-source cooldown/budget/backoff/
circuit-breaker, load-guard, erik-safe profile cap, delta-gated escalation. Root-cause
routing escalates code_fix/blocked (e.g. ATGBICS silent stock-write break) to a human
instead of looping retries; only staleness is auto-dispatched.
- security: remove hardcoded 'tip_dev_2026' DB password fallback across 6 sites ->
fail-closed requireDbPassword() / dbConnectionString().
SELF_HEAL_PLAN_ONLY defaults true (dispatches nothing; writes plan rows for review).
Verified against the live DB: detects ATGBICS-stock dead/54d -> escalate, Flexoptix-stock
stale/30d -> plan-dispatch; guards + ledger correct; 8/8 guard unit tests pass.
The script hardcoded the pre-rotation POSTGRES_PASSWORD (retired 2026-06-04,
already invalid). Source the current password from ~/.tip/.env (git-ignored,
exported) instead — same pattern as the FS.com residential runner.
One article (The Next Platform) had published_at 2026-07-15 while scraped
2026-06-19 -- an article cannot be published after it was scraped. One-time
clamp to the scrape date + BEFORE INSERT/UPDATE trigger to prevent recurrence.
- trg_compat_speed_guard: BEFORE INSERT/UPDATE on compatibility forces
algorithmic (spec_match/community) rows whose transceiver is faster than the
switch max port speed to status=incompatible. Curated rows never touched.
Makes the "no physically-impossible compatibility" invariant structural, so
it survives a scraper regression.
- check_switch_quality(): on-demand proof; CI gate is
`select bool_and(pass) from check_switch_quality() where enforced` (must be
true). Data-source gaps (coding coverage, speed<=0, datasheet) tracked as
enforced=false so they do not fail the gate before the data exists.
Root cause: the form-factor fallback in flexoptix-compat matched transceivers by
form factor only, ignoring speed, so a 400G OSFP port matched every OSFP module
incl. 1.6T (5686+ impossible rows across 266 switches).
- flexoptix-compat: fallback now filters transceiver speed <= switch max port
speed (derived from ports_config + stored max_speed_gbps). Prevents regeneration.
- sql/125: evidence-based effective-ceiling (stored/ports/curated) views;
flag 5811 impossible spec_match rows status=incompatible (curated rows never
touched, understated switches never false-flagged); backfill max_speed_gbps for
327 switches from ports_config; re-derive speed for 306 speed<=0 transceivers
(form-factor capped, high-confidence only). v_switch_quality_alerts is the task
list; v_switch_understated_ports surfaces switches needing datasheet enrichment.
Reading packages/scraper showed QSFPTEK/ATGBICS populate quantity_available
100%; they simply lack FS.com-only units_sold/warehouse-split. The 123 alert
keyed on those FS.com fields and false-positived QSFPTEK. Rekey to
quantity_available OR warehouse_qty so the alert list drains to empty; surface
pct_qty_available/pct_vendor_ts in v_stock_completeness.
FS.com renders per-warehouse availability as a bare word-date on each line
("2 Stk. im DE-Lager, 8. Juli 2026 Lieferbar"). The old regex only matched a
"Lieferung:"/"erwartet:" prefix or numeric dd.mm.yyyy, so warehouse_de/global
delivery_date + backorder_estimated_date were 0% populated across 85k rows, and
lead_time_days was never written at all.
- new fs-com-parse.ts: pure, unit-tested parsers (13 node:test cases).
- extractDeliveryDate finds the bare word/numeric date per warehouse, bounded to
its own line so a date never bleeds to a neighbouring warehouse.
- computeLeadTimeDays = min(delivery_date - today), floored at 0; threaded into
upsertStockObservation (now writes lead_time_days) and upsertPriceObservation.
- parseGermanQty: K/M abbreviations now treat '.' as a decimal ("4.8K"=4800,
"1.4M"=1.4M) instead of stripping it (was "1.4M"->14,000,000 — the outlier).
- units_sold sanity gate at ingest (FS_MAX_UNITS_SOLD, default 2,000,000).
- sql/122: purge impossible historical units_sold, re-document columns as collected.
sql/119: product_type enum (module|dac|aoc|aec|switch|nic|accessory) computed by
trigger trg_transceivers_biu from part_number/category/reach on every insert/update.
Backfilled 41008 rows (36284 module, 2791 dac, 1835 aoc, 29 aec, 27 accessory,
26 switch, 16 nic). Cables and switches/NICs no longer counted as modules;
BASE-T/RJ45 copper modules protected from cable demotion; AOC/DAC keyword beats
the optical-token guard. Consumers filter product_type=module.
sql/120: conservative form_factor recovery. Corrects only provably-wrong rows
(form_factors.max_speed < speed_gbps) via an explicit title token; 87 fixed
(e.g. 400G cables mislabelled SFP+ to QSFP-DD), 947 opaque rows left untouched
(never guessed). Scraped original preserved in new column form_factor_raw.
sql/121: document lead_time_days as not-collected (0 percent populated on all
time-series tables) so no consumer fabricates a default.
scraper/utils/db.ts: note that product_type/form_factor are trigger-maintained.
A switch presents transceiver compatibility ONLY if ports_verified=true (its port
config confirmed against the manufacturer datasheet). Unverified switches return
NO suggestions rather than potentially-wrong data. New columns: ports_verified,
ports_verified_source, ports_verified_at. Verified so far against datasheets:
HPE Aruba CX 10000-48Y6C, NVIDIA SN3700 (corrected 100G->200G QSFP56), SN4700,
SN5400, SN5600. 262 switches still pending datasheet verification.
Chassis switches use ports_config keys like 'max_per_slot_100G_QSFP28' (speed in
the middle), not just '100G_QSFP28'. The '^[0-9.]+G' anchor failed -> port_speed
NULL -> 0 suggestions (e.g. Cisco 8818 router). Now extracts the speed token
wherever it sits before the cage. Cisco 8818: 0 -> 55, all Cisco-codeable, max 400G.
Physical fit alone is not real compatibility — a module that seats in the cage
still won't run unless Flexoptix can code it for the switch's platform. Added a
coding requirement: each suggestion must have an fx_compatibilities entry for the
switch's mapped vendor (HPE/Aruba->aruba, Cisco->cisco, NVIDIA->mellanox, etc.;
whitebox/unmapped -> MSA Standard fallback).
CX 10000-48Y6C: 629 physically-fitting -> 283 actually-codeable-for-Aruba. All
283 verified to carry an Aruba coding. Every suggestion is now catalog-confirmed,
datasheet-accurate, physically fitting AND guaranteed codeable to run in the switch.
nominalSpeedFromSpecs missed '1.6T Ethernet' protocol entries (only matched 'G'),
returning 800 for twin-port 1.6T parts instead of 1600. Added Terabit handling +
guarded against 'Nx 800G' aggregate partial-matches. After re-deriving all 805
catalog-confirmed FX parts from their stored datasheet specs, DB now matches the
Flexoptix datasheet 100%: Form Factor 802/802, Speed 801/801, 0 deviations.
~107 parts were misattributed to the Flexoptix vendor but do not exist in the
live Flexoptix catalog (FX uses dot-SKUs like S.1606.28.KD; these use dash-SKUs
like FOT-AX-200G). They polluted 'BEI FLEXOPTIX BESTELLEN' with non-orderable
phantom parts carrying name-guessed specs. Now require fx_specifications IS NOT
NULL — every suggestion is confirmed in Flexoptix's API with authoritative
datasheet specs. CX 10000-48Y6C: 801 -> 630 suggestions, all catalog-confirmed,
0 physically impossible, 0 phantoms.
Root cause of unreliable speeds: the bulk Flexoptix products API returns NO speed
or form-factor field, so the catalog sync guessed them by parsing the product
NAME (e.g. 'FO-109010-CWDM' -> 100000G). Cryptic FX codes like 'O.164HG2.2.C:Sx'
are unparseable, producing garbage.
The detail enricher already pulls per-SKU specifications=1 (rate-limited) but only
wrote secondary fields. Now it also derives:
form_factor <- structured 'Form Factor' spec (authoritative datasheet value)
speed_gbps <- highest Ethernet rate in 'Supported Protocols', fallback to the
'Bandwidth' line-rate upper bound mapped to nominal
Both OVERWRITE the corrupt bulk values (COALESCE(spec, existing)). Never derived
from the product name. Verified: 100/100 freshly-enriched FX parts now have
physically-consistent form_factor/speed (0 contradictions), incl. uncrackable
codes correctly resolved to OSFP/800G, QSFP-DD800/800G etc.
getFlexoptixSuggestions matched ONLY by form factor, discarding the speed encoded
in each ports_config key (e.g. '100G_QSFP28'). Corrupt transceiver speed_gbps
values (400G/200G/128G/100000G mislabeled as QSFP28) leaked through, so a 100G
switch showed impossible '400G QSFP28' / '100T QSFP28' suggestions.
Now parses (speed, form_factor) from each port key and requires every suggested
module to (a) mechanically seat in the cage — precise port-FF -> accepted-module-FF
map, a QSFP28 cage takes QSFP+/28/56 but never QSFP-DD — and (b) have
speed_gbps <= the port's speed. CX 10000-48Y6C (25G SFP28 + 100G QSFP28) now
returns only valid <=25G SFP / <=100G QSFP modules; 0 physically impossible
entries (was 4 garbage groups). Belt-and-suspenders: even with corrupt speed data,
nothing oversized can reach a customer-facing suggestion.
reorder_signals grew to 4.49M rows / 1.19GB — the compute job INSERTed a fresh
row per transceiver every 4h run but never deleted old ones (24h TTL filtered
them at read time via DISTINCT ON + expires_at, but they were never purged).
4.37M rows were already expired dead weight.
Fix: DELETE existing rows for a transceiver before inserting its new signal, so
the table holds exactly one (latest) row per transceiver. Cleaned up to 18,175
rows / 4.5MB (99.6% reclaimed, VACUUM FULL). Backup: reorder_signals_keep_bak_20260606.
Verified: re-running compute:reorder-signals keeps count stable at 18,175.
Root cause of the persistent sync:flexoptix-catalog HTTP 401: line 397 used
'?? null' which only coerces null/undefined. With FLEXOPTIX_API_TOKEN='' (empty
string set in .env), token stayed '' and line 485's 'token ?? getBearerToken()'
returned '' instead of performing the username/password login — sending an empty
'Bearer ' header that the products endpoint rejected with 401.
Fix: '|| null' coerces empty string to null so the bearer-login fallback fires.
Verified: sync now completes (username/password -> customer token -> products 200,
3 products/price/stock writes on limit=50). Credentials were correct all along.
The Research Robot panel showed only an LLM assessment (info, no action). Now:
API (research-robot.ts):
- GET enriches response with recommendations[] computed from live pgboss state:
classifies each persistently-failing job (auth/401, network, no-handler, other)
into severity + concrete advice + offered actions.
- POST /action {action,job}: dispatch (enqueue one run), pause (remove from
schedule with backup to research_robot_paused_schedules), resume (restore).
All validated against the pgboss.queue whitelist.
Dashboard:
- Renders each recommendation as a card with severity colour, cause, last error,
and action buttons (Jetzt auslösen / Pausieren / Fortsetzen / Token-Anleitung).
- Verified: sync:flexoptix-catalog -> critical auth (HTTP 401), offers
token-help + pause. Dispatch/pause/resume roundtrip tested green.
The 30d-vs-60d price momentum aggregated AVG/median across whatever SKUs
happened to be in a speed/form-factor bucket each period. New expensive SKUs
entering the catalog (NVIDIA switches at 30k USD, AOC cables) faked huge jumps
— 400G OSFP showed +151% when matched-SKU reality was 0%.
Now: compute per-transceiver median price in each period, keep only SKUs present
in BOTH periods (>=2 obs each), report the median of per-SKU pct deltas. Also
excludes non-transceiver form factors, AOC/DAC cables, switch SKUs, price>15k,
and anomalous observations. Result: 400G OSFP +151%->0%, signals 21->8, and the
ones that remain (NVIDIA MFA7U10 +84% same-SKU) are genuine price moves.
Adds parseWarehouseStock() to decode the HTML-entity-encoded warehouse_stock JSON
(us/nl/sg/cn per-region array). When the static page has warehouse data, writes:
warehouse_de_qty ← nl (EU-closest warehouse)
warehouse_global_qty ← sum(us+nl+sg+cn), or falls back to quantity_available
stock_confidence ← 3 (L3) when warehouse breakdown available, else 2
Note: per-warehouse quantities require JS execution to populate (API-loaded);
static HTML has [0,0] placeholders. The fallback ensures NADDOD global totals
appear in the competitor-by-tech dashboard comparison.
Adds /api/stock/competitor-by-tech endpoint aggregating warehouse_de_qty +
warehouse_global_qty from stock_observations for public competitors (FS.COM
etc.) per technology class. Dashboard velocity table gets two new columns
FS.COM DE + FS.COM Global with traffic-light coloring vs. monthly demand.
Clicking any signal card opens a modal with a 180-day SVG line chart
per source vendor (multi-line, colour-coded), x-axis dates, y-axis price,
current best price summary. Uses existing /api/price-history/:id endpoint.
No external chart library — pure inline SVG.
Group by part_number instead of transceiver_id (eliminates OEM duplicate rows).
Use PERCENTILE_CONT median instead of AVG to reduce single-outlier impact.
Add CV-filter (stddev/avg <= 0.35 over 2x window) to exclude high-variance
sources like Mouser quantity-tier pricing that produces artificial swings.
Blog LLM client probes BLOG_OLLAMA_URL (primary, WireGuard tunnel to Mac
Studio loopback Ollama) and falls back to BLOG_OLLAMA_URL_FALLBACK
(Cloudflare tunnel) when the primary transport is unreachable. Re-probed
at startup and every 60s; prefers primary when available. Both tunnels
terminate on the Mac loopback over independent transports, so the blog
keeps reaching fo-blog regardless of which transport drops.
Blog auto-discovery + generation now use BLOG_OLLAMA_URL (-> Mac Studio
192.168.178.213:11434 over the Erik<->home WireGuard tunnel), falling
back to OLLAMA_URL. Search/embeddings stay on Erik-local OLLAMA_URL
(nomic-embed-text). Fixes blog model not-found after OLLAMA_URL was
repointed to Erik-local for the search fix.
