fix(dqo): route vendor pricing/stock to pi-fetch (residential Pi cluster)
The Adapter (sql/129) seeded all vendor sources as erik-safe, but TIP data is fetched by the 3 residential Pi 5 workers (tashi/portia/ester), all now Playwright-capable — not Erik (datacenter IP, blocked by FS.com/ATGBICS). 14 sources -> pi-fetch (fixes gbics/sfpcables never-dispatching under erik-safe). Config only; loop stays in Plan-Mode. 10gtek stays proxmox-heavy.
This commit is contained in:
parent
332319975a
commit
201f5d3a47
36
sql/130-adapter-pi-topology.sql
Normal file
36
sql/130-adapter-pi-topology.sql
Normal file
@ -0,0 +1,36 @@
|
||||
-- sql/130-adapter-pi-topology.sql
|
||||
-- Teach the TIP Adapter (tip-dqo, the data-quality orchestrator from sql/129) the REAL
|
||||
-- fetch topology, verified 2026-07-06: TIP data is fetched primarily by three residential
|
||||
-- Raspberry Pi 5 nodes — ContextX-Pi01/tashi(.204, 16GB), Pi02/portia(.212, 4GB),
|
||||
-- Pi03/ester(.211, 4GB) — all live pg-boss workers, all now Playwright-capable
|
||||
-- (chromium v1208, launch-verified). NUC(.115) is the scheduler; Erik(.127) holds DB+API.
|
||||
--
|
||||
-- The other session seeded every vendor pricing/stock source as profile='erik-safe'. That
|
||||
-- is wrong for the fetch fleet: Erik is a datacenter IP (blocked by FS.com/ATGBICS & co.),
|
||||
-- and 'erik-safe' only dispatches ERIK_SAFE_QUEUES — so gbics/sfpcables would NEVER dispatch
|
||||
-- (their own notes flagged this). The residential Pis are the correct fetchers.
|
||||
--
|
||||
-- Fix: route all vendor pricing/stock sources to profile='pi-fetch' (recognized by the
|
||||
-- orchestrator: queuesForProfile → all non-heavy/non-discovery queues; cap 10). This is
|
||||
-- CONFIG ONLY — no scraper fires (the loop stays in Plan-Mode until SELF_HEAL_PLAN_ONLY=false).
|
||||
-- 10gtek stays 'proxmox-heavy' (its queue is in HEAVY_QUEUES). __global__ stays 'erik-safe'
|
||||
-- (config row, excluded from v_dqo_targets). Idempotent.
|
||||
--
|
||||
-- OPEN (code-level, belongs to the tip-dqo owner — NOT changed here to avoid colliding with
|
||||
-- their in-flight go-live): the HEAVY Playwright queues (scrape:pricing:fs, prolabs, compat:*)
|
||||
-- are now runnable on the residential Pi cluster too (Pis have Playwright). queuesForProfile
|
||||
-- currently excludes HEAVY from pi-fetch, so heavy IP-blocked sources (FS.com) still can't
|
||||
-- route to the Pis. A code change (let pi-fetch — or a new pi-heavy profile — dispatch HEAVY
|
||||
-- to the residential cluster) would let the cluster take those over from the Mac runners.
|
||||
|
||||
update source_registry
|
||||
set profile = 'pi-fetch',
|
||||
notes = coalesce(notes || ' | ', '')
|
||||
|| 'topology-fix 130 (2026-07-06): residential Pi cluster is the fetcher, not Erik.',
|
||||
updated_at = now()
|
||||
where source_key <> '__global__'
|
||||
and profile = 'erik-safe';
|
||||
|
||||
insert into _migrations (filename, applied_at)
|
||||
select '130-adapter-pi-topology.sql', now()
|
||||
where not exists (select 1 from _migrations where filename = '130-adapter-pi-topology.sql');
|
||||
Loading…
x
Reference in New Issue
Block a user