b43bdd3060
Phase 0 - Foundation: - Restructure into npm workspace monorepo (packages/core, api, scraper) - PostgreSQL 17 + TimescaleDB schema (15 tables incl. hypertables) - Docker Compose for local dev (PostgreSQL on 5433 + Qdrant) - Express 5 API on port 3200 with 6 routes - Seed script to migrate 159 transceivers + 42 standards from npm package - Erik server setup script + PM2 ecosystem config Phase 1 - Scraper Engine: - Crawlee + Playwright framework with pg-boss scheduler - FS.com scraper (PlaywrightCrawler, anti-bot workaround) - Optcore.net scraper (WP REST API enumeration + PlaywrightCrawler) - Uses /wp-json/wp/v2/product to get 2000+ product URLs - Playwright renders individual product pages for price extraction - Cisco TMG Matrix scraper (compatibility data) - News RSS aggregator (optics.org, SPIE, Network World, Nature Photonics) - Keyword relevance scoring for transceiver/fiber topics - xml2js with malformed XML sanitization - SHA-256 content hashing for change detection (skip unchanged records) - pg-boss v10 with explicit queue creation before scheduling
38 lines
715 B
TypeScript
38 lines
715 B
TypeScript
/**
|
|
* transceiver-db — Open-source optical transceiver database
|
|
*
|
|
* 159 products, 42 IEEE/MSA standards, 16 form factors, 9 speed tiers.
|
|
* From 1G SFP to 800G OSFP. Zero dependencies.
|
|
*/
|
|
|
|
export {
|
|
transceivers,
|
|
searchTransceivers,
|
|
getByFormFactor,
|
|
getBySpeed,
|
|
getByReach,
|
|
getByCategory,
|
|
getById,
|
|
} from "./database";
|
|
|
|
export { standards, getStandard, searchStandards } from "./standards";
|
|
|
|
export { competitors, getCompetitor } from "./market";
|
|
|
|
export { breakouts } from "./breakouts";
|
|
|
|
export type {
|
|
Transceiver,
|
|
Standard,
|
|
Competitor,
|
|
Breakout,
|
|
FormFactor,
|
|
FiberType,
|
|
ConnectorType,
|
|
TempRange,
|
|
ProductCategory,
|
|
PriceTier,
|
|
MarketStatus,
|
|
VendorCompat,
|
|
} from "./types";
|