rene/transceiver-db
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
425 Commits 5 Branches 9 Tags
Commit Graph

11 Commits

Author SHA1 Message Date
Rene Fichtmueller
de179c4c7c fix: remove DEMO labels from real stock data; fix switch Flexoptix suggestions; enrich Hot Topics LLM context 
Stock dashboard (index.html):
- Replace all [DEMO]/demo badges on warehouse data with "FS.com" source labels
  (data was always real scraper data, never demo in the DB)
- Update subtitle: "Scraper-Lagermengen: DEMO DATA" → "Wettbewerber-Marktdaten"
- "Recently Restocked" badge: DEMO DATA → SCRAPER DATA

Switch detail (queries.ts):
- Fix getFlexoptixSuggestions: wavelength_nm → wavelength_tx_nm,
  price_verified_usd → street_price_usd (column mismatch with live schema)
- DS5000 and other OSFP switches now show all 62 Flexoptix OSFP transceivers
  with direct shop links in the detail modal

Hot Topics (hot-topics.ts):
- NOG Talks + News Article clusters now fetch summary/mentioned_vendors/
  mentioned_products/mentioned_standards from news_articles table
- description field builds bullet-point list per article with summaries,
  key vendors/standards (vs. 3 bare titles joined with "|" before)
- buildTopicBriefing() rewritten as structured LLM document with sections:
  Market Signals (bullets), Recommended Angle, Market Context (buy signal,
  technologies, impact horizon), Writing Instructions (600-900 words,
  actionable, opinionated, no generic summaries)
 2026-05-14 00:33:45 +02:00
Rene Fichtmueller
1af4f090f7 fix: harden TIP verification cleanup 2026-05-09 22:16:29 +02:00
Rene Fichtmueller
a43e572946 fix: advance TIP product verification robots 2026-05-09 20:19:19 +02:00
Rene Fichtmueller
a2492d833b feat: Flexoptix order section per switch + reject generic/logo images 2026-04-20 23:31:36 +02:00
Rene Fichtmueller
c0cd0dc1ca feat: compatibility panel — verification_method, competitor prices, spec-match collapsible 
- API: getCompatibleTransceivers() returns verification_method, orders vendor_compat first
- Dashboard: Flexoptix section splits vendor-tested vs spec-match (collapsed)
- Dashboard: Competitor section shows vendor-tested with prices, spec-match as chips
 2026-04-20 22:52:49 +02:00
Rene Fichtmueller
e9fcda2811 feat: wire finder.ts + switch-docs + Ollama LLM tools to MCP server 
MCP Server (packages/mcp-server/src/index.ts):
- Register registerSwitchDocTools (switch-docs.ts) — switch documentation lookup
- Register finderTools dynamically (finder.ts) — find_flexoptix_for_switch, get_competitor_alerts
- Add analyze_market_with_llm tool: qwen2.5:14b via Ollama, enriched with live hype cycle + pricing + news
- Add generate_blog_post tool: fo-blog-v5 (fine-tuned) with qwen2.5:14b fallback, enriched with live pricing data
- OLLAMA_BASE_URL env var (default: https://ollama.fichtmueller.org)

Also includes scraper improvements (ascentoptics, atgbics, gbics, skylane, ebay-enricher),
API route updates (blog, blog-sll, health, hot-topics, transceivers, queries),
and dashboard hot-topics refresh.
 2026-04-18 00:21:58 +02:00
Rene Fichtmueller
e6d042f827 fix: resolve merge conflict in index.ts + add untracked blog-sll, news, sql migration 2026-04-05 11:51:07 +02:00
Rene Fichtmueller
4020ec77d9 feat: product intelligence layer — eBay enricher, community issues, datasheets+manuals API 
- Migration 020: product_issues table, condition/marketplace on price_observations, features JSONB
- eBay enricher: switch features/description/refurb prices + transceiver condition pricing
- Community issues scraper: Reddit/ServeTheHome/Arista/Cisco community bug reports
- 7 pre-seeded issues (DCS-7800R3, SG350, QFX5120, CRS326, USW-Pro etc.)
- API: /switches/:id/issues + /switches/:id/documents endpoints
- Dashboard switch modal: features from DB, description, eBay refurb price, issues+docs async
- Datasheet finder for Arista/Cisco/Juniper/HPE vendor pages
- Scheduler: 4 new jobs (ebay enrichment nightly, community issues weekly)
 2026-04-01 22:46:27 +02:00
Rene Fichtmueller
f91d2a15b9 feat: switch Flexoptix recommendations, switch verified labels, stronger verification check 
- getCompatibleTransceivers: adds vendor_name, price, verification fields; Flexoptix sorted first
- Switch detail: data quality bar (Image/Product Page/Datasheet confirmed)
- Switch detail: Flexoptix Recommended section with prices, verified badges, shop links
- Switch detail: other vendors section shows 100% badge on slugs
- Transceiver detail: verification condition explicit === true (cache-safe)
- Transceiver detail: fallback text when no verification data exists yet
 2026-04-01 20:59:30 +02:00
Rene Fichtmueller
615a7e50c7 fix: remove non-existent vendor URL columns, fix text=uuid cast in transceiver lookup 2026-03-30 07:49:54 +02:00
Rene Fichtmueller
e9fb50a248 feat: TIP Phase 0+1 — monorepo, DB schema, API, scraper engine 
Phase 0 - Foundation:
- Restructure into npm workspace monorepo (packages/core, api, scraper)
- PostgreSQL 17 + TimescaleDB schema (15 tables incl. hypertables)
- Docker Compose for local dev (PostgreSQL on 5433 + Qdrant)
- Express 5 API on port 3200 with 6 routes
- Seed script to migrate 159 transceivers + 42 standards from npm package
- Erik server setup script + PM2 ecosystem config

Phase 1 - Scraper Engine:
- Crawlee + Playwright framework with pg-boss scheduler
- FS.com scraper (PlaywrightCrawler, anti-bot workaround)
- Optcore.net scraper (WP REST API enumeration + PlaywrightCrawler)
  - Uses /wp-json/wp/v2/product to get 2000+ product URLs
  - Playwright renders individual product pages for price extraction
- Cisco TMG Matrix scraper (compatibility data)
- News RSS aggregator (optics.org, SPIE, Network World, Nature Photonics)
  - Keyword relevance scoring for transceiver/fiber topics
  - xml2js with malformed XML sanitization
- SHA-256 content hashing for change detection (skip unchanged records)
- pg-boss v10 with explicit queue creation before scheduling
 2026-03-27 16:27:31 +13:00