dd2c917f05
sql/119: product_type enum (module|dac|aoc|aec|switch|nic|accessory) computed by trigger trg_transceivers_biu from part_number/category/reach on every insert/update. Backfilled 41008 rows (36284 module, 2791 dac, 1835 aoc, 29 aec, 27 accessory, 26 switch, 16 nic). Cables and switches/NICs no longer counted as modules; BASE-T/RJ45 copper modules protected from cable demotion; AOC/DAC keyword beats the optical-token guard. Consumers filter product_type=module. sql/120: conservative form_factor recovery. Corrects only provably-wrong rows (form_factors.max_speed < speed_gbps) via an explicit title token; 87 fixed (e.g. 400G cables mislabelled SFP+ to QSFP-DD), 947 opaque rows left untouched (never guessed). Scraped original preserved in new column form_factor_raw. sql/121: document lead_time_days as not-collected (0 percent populated on all time-series tables) so no consumer fabricates a default. scraper/utils/db.ts: note that product_type/form_factor are trigger-maintained.
36 lines
2.3 KiB
SQL
36 lines
2.3 KiB
SQL
-- Migration 121: document lead_time_days as NOT COLLECTED
|
|
-- ─────────────────────────────────────────────────────────────────────────────
|
|
-- Verified 2026-07-04: lead_time_days is 0 % populated on every time-series table
|
|
-- price_observations : 1,378,984 rows → 0 filled
|
|
-- stock_observations : 85,406 rows → 0 filled
|
|
-- stock_snapshots : 0 rows (LLM-crawler path is dormant)
|
|
--
|
|
-- The plumbing exists (crawler-llm stock-schema captures lead_time_days and
|
|
-- writes it to stock_snapshots; upsertPriceObservation accepts a leadTimeDays
|
|
-- param) but the ACTIVE ingestion path never supplies a value, and there is no
|
|
-- evidence any tracked vendor page exposes a machine-readable lead time.
|
|
--
|
|
-- Rather than fabricate a default, this migration documents the column as
|
|
-- not-collected so no consumer invents one. Lupo already renders "k.A." / "n/a"
|
|
-- and blends only the two signals it actually measures (availability, breadth).
|
|
-- Populating this field is a SCRAPER-side task (extend the active crawlers to
|
|
-- read a delivery/lead-time badge) — out of scope for a schema migration.
|
|
-- ─────────────────────────────────────────────────────────────────────────────
|
|
|
|
COMMENT ON COLUMN price_observations.lead_time_days IS
|
|
'NOT COLLECTED (verified 2026-07-04): 0 % populated. The active scraper path never '
|
|
'supplies a lead time. Do NOT default this — treat NULL as "unknown / not captured". '
|
|
'Populating it requires scraper work (read a delivery-time badge on the vendor page).';
|
|
|
|
COMMENT ON COLUMN stock_observations.lead_time_days IS
|
|
'NOT COLLECTED (verified 2026-07-04): 0 % populated. Treat NULL as "unknown". See '
|
|
'price_observations.lead_time_days.';
|
|
|
|
COMMENT ON COLUMN stock_snapshots.lead_time_days IS
|
|
'Captured by the crawler-llm path (stock-schema.lead_time_days) but that path is '
|
|
'currently dormant (0 rows). Treat NULL as "unknown / not captured".';
|
|
|
|
INSERT INTO _migrations (filename)
|
|
VALUES ('121-lead-time-not-collected-doc.sql')
|
|
ON CONFLICT (filename) DO NOTHING;
|