Rene Fichtmueller
|
a6f7968393
|
feat: Phase 3 — Norton-Bass Hype Cycle Engine
Implements the full Norton-Bass Multigenerational Diffusion Model for
transceiver technology lifecycle forecasting.
Math: Bass diffusion F(t) + logistic adoption S(t) = L / (1 + e^(-k(t-t0)))
Parameters: p (innovation ~0.03), q (imitation ~0.3-0.5), m (market potential)
Phase Classification Engine (composite score):
30% Port shipment share + 20% ASP decline rate + 15% Standards maturity
+ 15% Interop validation + 10% Vendor trajectory + 10% Media sentiment
11 technologies tracked: 1G → 10G → 25G → 40G → 100G → 400G → 800G → 1.6T
+ CPO, LPO, 400ZR Coherent
5-year adoption forecast per technology
API: GET /api/hype-cycle (all) + GET /api/hype-cycle/:tech (detail)
Live: https://transceiver-db.context-x.org/api/hype-cycle
|
2026-03-27 23:35:57 +13:00 |
|
Rene Fichtmueller
|
b43bdd3060
|
feat: TIP Phase 0+1 — monorepo, DB schema, API, scraper engine
Phase 0 - Foundation:
- Restructure into npm workspace monorepo (packages/core, api, scraper)
- PostgreSQL 17 + TimescaleDB schema (15 tables incl. hypertables)
- Docker Compose for local dev (PostgreSQL on 5433 + Qdrant)
- Express 5 API on port 3200 with 6 routes
- Seed script to migrate 159 transceivers + 42 standards from npm package
- Erik server setup script + PM2 ecosystem config
Phase 1 - Scraper Engine:
- Crawlee + Playwright framework with pg-boss scheduler
- FS.com scraper (PlaywrightCrawler, anti-bot workaround)
- Optcore.net scraper (WP REST API enumeration + PlaywrightCrawler)
- Uses /wp-json/wp/v2/product to get 2000+ product URLs
- Playwright renders individual product pages for price extraction
- Cisco TMG Matrix scraper (compatibility data)
- News RSS aggregator (optics.org, SPIE, Network World, Nature Photonics)
- Keyword relevance scoring for transceiver/fiber topics
- xml2js with malformed XML sanitization
- SHA-256 content hashing for change detection (skip unchanged records)
- pg-boss v10 with explicit queue creation before scheduling
|
2026-03-27 16:27:31 +13:00 |
|