Rene Fichtmueller
|
297dc46f2b
|
feat(crawler-llm): intelligent vendor discovery pipeline + TIPLLM training data
- spec-validator.ts: physical plausibility checks (form factor↔speed matrix,
wavelength↔fiber consistency, IEEE standard cross-check, reach limits).
Outputs tier (high/medium/low/rejected) + confidence_delta for LLM scores.
- training-data-writer.ts: converts validated crawler extractions to SFT JSONL
training pairs (spec_qa / crawl_reasoning / validation / discovery types).
Auto-commits and pushes to Gitea tip-training-data repo in batches of 50.
- vendor-discovery-crawler.ts: PlaywrightCrawler pipeline — catalog URL →
LLM extraction (scrapeWithLLM) → spec validation → DB persist +
Gitea SFT training pairs. 8 vendor configs registered
(Cisco/Juniper/Arista/FS.com/Flexoptix/Nokia/Huawei/II-VI).
- scheduler.ts: 8 weekly discover:vendor:* jobs added (Sun 20:00–Mon 10:00 UTC).
Total registered jobs: 102.
- Gitea repo created: gitea.context-x.org/rene/tip-training-data
|
2026-04-28 23:46:34 +02:00 |
|
Rene Fichtmueller
|
681da54523
|
feat: Procurement Intelligence Engine (WS0c)
- Migration 019: stock_snapshots, abc_classification, reorder_signals,
product_lifecycle_events, market_intelligence, crawler_llm_log tables
- Seeded 7 market intel events (OFC 2026, AWS/Azure CapEx, Coherent lead times,
EU TED tenders, ECOC 2026, IEEE 802.3df)
- Seeded 4 lifecycle events (Cisco SFP-10G-LR EOL, Juniper EOL,
400ZR ratified, 800G MSA draft)
- Crawler LLM: core.ts (Ollama-based extractor), stock-schema.ts (typed schemas
+ vendor profiles for Flexoptix/FS.com/10Gtek/ATGBICS/ProLabs/Farnell/Mouser),
validator.ts (rule-based sanity checks + cross-validation)
- market-intelligence.ts scraper: OFC/ECOC, LightReading, IEEE 802.3, EU TED,
Farnell/Mouser lead times, FierceTelecom — weekly via pg-boss
- computeAbcClassification(): dynamic A/B/C classification from price obs +
compat count + vendor breadth
- computeReorderSignals(): buy_now/wait/hold/monitor with reasons + signal strength
- API: GET /api/procurement/overview|signals|signals/:id|abc|market-intel|
stock-trends/:id|lifecycle
- Dashboard: Procurement Intel tab with Reorder Signals, ABC table,
Market Intel cards, Lifecycle Events
|
2026-04-01 22:04:33 +02:00 |
|