37 Commits

Author SHA1 Message Date
Rene Fichtmueller
270bd12382 feat(dashboard): clickable LLM model selector — switch blog engine at runtime
- client.ts: BLOG_LLM_PROVIDER/OLLAMA_LLM_MODEL as mutable state (setLlmProvider/
  getLlmProvider). Reads blog-llm-settings.json on startup for persistence.
  All generate()/checkHealth()/chat() calls use dynamic provider() + llmModel()
  — no restart required for switches.

- blog.ts: POST /api/blog/llm/switch endpoint — validates provider, calls
  setLlmProvider(), writes settings file, returns previous+active state.

- index.html: all 4 model cards now clickable (cursor:pointer, hover fade).
  switchBlogLlm(provider, model) — disables cards during switch, shows
  green/red feedback toast, auto-refreshes status. SSH instruction removed.
2026-04-29 01:15:45 +02:00
Rene Fichtmueller
b5decc517f fix: hard-cap blog generation to 800-1100 words
LLM_OPTS.maxTokens 8192 → 1600, LLM_REFINE 6144 → 1800,
Step 4 master draft 8192 → 1600. Added explicit word-count
constraint to STEP4_MASTER_DRAFT prompt: HARD LIMIT 800-1100 words.

Root cause: no token ceiling → fo-blog-v6 produced 4000-5000w articles.
Generated-by label updated to fo-blog-engine-v6-length-fix.
2026-04-28 22:45:48 +02:00
Rene Fichtmueller
d7144731e0 feat(scraper): add 100+ OEM seed scrapers + tip-llm-guided inference layer
New OEM transceiver seed scrapers (94 cron-scheduled, 24/7):
- Media/Broadcast: Evertz, Grass Valley, Haivision, Viasat
- Asian Optical: FiberHome, Oplink, Accelink, Hisense Broadband
- Optical Mfrs: Lumentum, II-VI/Coherent, Source Photonics, O-Net,
  InnoLight, AOI, Sumitomo Electric, NeoPhotonics
- Industrial: GE Grid, Schweitzer, Moxa Industrial, Cisco IE,
  Phoenix Contact, Beckhoff, Omron, ABB, Siemens, Schneider, Rockwell, Belden
- Enterprise/DC: Arista, Pica8, Pluribus, DriveNets, Cisco (Meraki/Catalyst/Nexus/ASR)
- Cloud: AWS, Azure, Google Cloud, Meta
- Storage: NetApp, Pure Storage, HPE Storage, IBM Storage, Dell Storage, Hitachi Vantara
- 5G/RAN: Samsung Networks, Nokia AirScale, Ericsson RAN, Mavenir
- Security: Check Point, Barracuda, Fortinet, Palo Alto
- Telecom Optical: ADVA, PacketLight, FiberHome, Accelink, Hisense

API: tip-llm-guided inference layer (strict schema + repair-retry + safe fallback)
- POST /api/tip-llm/infer|research-plan|extract|finding|health
- Hard JSON schema enforcement, create_finding=false on empty evidence
- Confidence gate (>= 0.4), validation with consistency check

Build: added incremental=true to scraper tsconfig (OOM prevention)
Scheduler: 87 → 94 registered workers
2026-04-27 00:00:14 +02:00
Rene Fichtmueller
62d97a783c feat: add claude-code LLM provider + update dashboard to fo-blog-v5
- client.ts: add claude-code provider routing BLOG_LLM_PROVIDER=claude-code
  to claude-bridge (flat-rate, no API billing via Claude Code subscription)
- checkHealth() now pings /health on claude-bridge for real availability check
- Default OLLAMA_LLM_MODEL changed from qwen2.5:14b to fo-blog-v5
- Dashboard: add claude-code card (EMPFOHLEN), rename fo-blog-v3 → fo-blog-v5
- loadBlogLLMStatus() handles all 3 providers: claude-code/anthropic/ollama
- Grid expanded from 3 to 4 columns to accommodate new card
- ecosystem.config.js + .env on Erik: OLLAMA_LLM_MODEL=fo-blog-v5 confirmed
2026-04-18 20:45:14 +02:00
Rene Fichtmueller
e9fcda2811 feat: wire finder.ts + switch-docs + Ollama LLM tools to MCP server
MCP Server (packages/mcp-server/src/index.ts):
- Register registerSwitchDocTools (switch-docs.ts) — switch documentation lookup
- Register finderTools dynamically (finder.ts) — find_flexoptix_for_switch, get_competitor_alerts
- Add analyze_market_with_llm tool: qwen2.5:14b via Ollama, enriched with live hype cycle + pricing + news
- Add generate_blog_post tool: fo-blog-v5 (fine-tuned) with qwen2.5:14b fallback, enriched with live pricing data
- OLLAMA_BASE_URL env var (default: https://ollama.fichtmueller.org)

Also includes scraper improvements (ascentoptics, atgbics, gbics, skylane, ebay-enricher),
API route updates (blog, blog-sll, health, hot-topics, transceivers, queries),
and dashboard hot-topics refresh.
2026-04-18 00:21:58 +02:00
Rene Fichtmueller
51af249361 Merge remote-tracking branch 'github/main'
# Conflicts:
#	packages/api/src/llm/fo-blog-pipeline.ts
#	packages/api/src/routes/blog.ts
#	packages/scraper/src/scheduler.ts
#	packages/scraper/src/scrapers/fs-com.ts
#	packages/scraper/src/scrapers/gbics.ts
2026-04-06 18:03:36 +02:00
Rene Fichtmueller
4acf293690 fix(llm): checkHealth uses key presence check, not live API call
Live Anthropic API call during health check causes 429 when the pipeline
is actively running, blocking all subsequent regenerate requests.
2026-04-06 04:07:21 +02:00
Rene Fichtmueller
72033ff5c5 fix(blog): fix claudeQueue deadlock from recursive 429 retry
The generateClaude() function was recursively calling itself inside
enqueueClaude(), creating a circular Promise dependency that permanently
deadlocked the claudeQueue. Any 429 rate-limit response would poison
the queue, blocking all future Claude API calls until server restart.

Fixes:
- Split retries into claudeApiCall() which is called from enqueueClaude
  (not re-entering the queue on retry = no circular dependency)
- Max 3 retries with increasing backoff (10s/30s/60s)
- Add resetClaudeQueue() exported function
- Add 15-minute auto-reset stall detection to enqueueClaude
- Expose resetClaudeQueue in POST /api/blog/llm/reset-queue endpoint
- Fix merge conflict markers in index.ts (duplicate scraperRouter import)
2026-04-06 02:51:28 +02:00
Rene Fichtmueller
b6928265bf fix: serialize Claude API calls via queue to prevent 429 rate-limit spam
Tier-1 Anthropic API has 40K TPM — with ~20K tokens per pipeline step,
concurrent calls immediately hit the limit. enqueueClaude() serializes
all generateClaude() calls so only one runs at a time, eliminating
the flood of 429-retry-429-retry loops.
2026-04-06 00:57:03 +02:00
Rene Fichtmueller
cf04549b1b feat: add Anthropic Claude provider to blog LLM client
- Auto-routes to Claude API when BLOG_LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY set
- Fallback to Ollama queue when key not present
- Add rate-limit retry (429 → 10s backoff) for Claude API
- Add STEP_TECHNICAL_SANITY, STEP_SELF_HEAL, STEP_TITLE_CONTRACT_CHECK prompts
- Fix STEP_LINKEDIN_POST angle-specific hooks, remove Gold Reference repetition
2026-04-06 00:21:48 +02:00
Rene Fichtmueller
c43e1f881a fix(blog): hard story blacklist in STEP4 + LinkedIn — ban 2AM/dirty connector/lab-vs-prod stories
10 specific story patterns banned directly in draft prompt.
LinkedIn banned hooks: 'Everything looks fine', 'CRC creeping', 'Same optics same setup'.
Title Contract now injected into STEP4 as binding constraint.
2026-04-05 23:55:56 +02:00
Rene Fichtmueller
3913372f10 feat(blog): Title Contract + Technical Sanity Check + Self-Heal + angle-aware LinkedIn generator
Pipeline now has 21 steps:
- STEP0: Title Contract binds LLM to headline promise
- STEP19: Technical Sanity Check (optical engineering accuracy)
- STEP20: Self-Heal (auto-fix technical errors preserving tone)
- STEP21: Title Contract Verification (final gate check)
- LinkedIn generator is now angle-aware (no more default Physical Layer hook)
2026-04-05 23:11:16 +02:00
Rene Fichtmueller
8e0eda6c41 fix(blog): anti-repetition engine — 6 angle types, forbidden structures, existing article context injection 2026-04-05 22:47:15 +02:00
Rene Fichtmueller
438225cf7c fix(blog): raise word target to 1200-1600, fix power-budget false positive in validateArticle 2026-04-05 20:49:22 +02:00
Rene Fichtmueller
e4dfd2a2db feat(blog): AEM/APM pipeline steps + SLL context builder + LinkedIn v2 prompts 2026-04-05 01:26:09 +02:00
Rene Fichtmueller
fea0b0fb66 feat: blog engine v5 — Auto-Kill Layer, 16-step pipeline, longer content
Upgrades FO Blog Pipeline from 14 to 16 steps:
- NEW Step 8d: Auto-Kill Layer v1.0 (10 systematic categories A-J)
- NEW Step 15: Auto-Kill Scoring (cleanliness, narrative, non-AI, relevance)
- Updated banned phrases from Gold-standard editorial feedback
- Soft Delete List for conditional phrases
- Auto-Kill categories: spec blocks, formulas, section leakage,
  generic transitions, repeated concepts, SKU mentions, false authority,
  over-explained basics, whitepaper tone, fake precision

Content length changes per user feedback:
- Blog target: 1,200-2,000 words (was 700-1,000) — thorough and detailed
- LinkedIn target: 2,000-2,800 chars (was 350-600) — use maximum length
- Reduction pass: 25-30% cut (was 15-25%) — remove weak, keep depth
2026-04-04 11:02:45 +02:00
Rene Fichtmueller
ede4f5b966 feat: blog engine v3 — 8-stage pipeline with Auto-Kill Layer
Complete rewrite of blog prompts and pipeline based on editorial
Gold-standard feedback. Replaces 3-pass system with 8-stage pipeline:

1. Master generation (narrative voice, no spec dumps)
2. Narrative Control (kill visible structure, enforce flow)
3. Auto-Kill Layer (remove AI phrases, spec residue, repetition)
4. Reduction Engine (cut 40% — keep strongest ideas only)
5. Depth pass (add specifics where vague, no spec dumps)
6. Quality Control (hard delete list validation)
7. Procurement layer (optional, sales audience)
8. LinkedIn post generation (new)

Key changes:
- System prompt rewritten with Hard Delete List (29 banned phrases)
- Soft Delete List for conditional phrases
- Auto-Kill categories A-J (spec blocks, formulas, whitepaper tone, etc.)
- Master prompts enforce continuous narrative, no section headings
- Word count targets reduced (800-1200 instead of 1500+)
- Scoring pass added (cleanliness, narrative, non-AI feel, relevance)
- LinkedIn companion post auto-generated
- Context data injection reduced (fewer items, no dump instructions)
2026-04-04 10:52:31 +02:00
Rene Fichtmueller
c509251109 feat(blog): Spec dump hard fail + Gold Standards 6 + LinkedIn v2
- System prompt: SPEC DUMP ABSOLUTE HARD FAIL block (before FORMAT rules)
  TX/RX tables, multi-optic comparison blocks, repeated sections = hard fail
  Behavioral prose rule: "what actually happens" not "what the spec says"

- STEP9 QA: check 12a SPEC DUMP — removes datasheet blocks, flags
  duplicate sections (e.g. "fiber types" twice), spec-heavy intros

- Gold Standard 6: 400G/800G deep dive corrected (8.8→10/10)
  zero spec tables, pure behavioral narrative, 3 core ideas max,
  ending is reframe not checklist

- LinkedIn Gold Example 2: sharper short format (346 chars vs 700)
  reframe hook, short beats without bullet markers, no emoji, 4 hashtags

- STEP_LINKEDIN_POST: rewritten with new gold format
  optimal 350-600 chars, beat rhythm, no bullet markers, gold example inline

- WRONG PATTERNS: +7 new entries (spec dump, duplicate section,
  LinkedIn bullet list, LinkedIn "excited to share" hook, LinkedIn >800 chars)
2026-04-04 09:32:01 +02:00
Rene Fichtmueller
4f631fc61e feat(blog): Reduction Engine v1.0 + LaTeX/connector hard fails
- Replace STEP8b_REDUCTION with 5-pass Reduction Engine:
  Pass 1: Repetition Kill (one concept, one home)
  Pass 2: Tech Prune (LaTeX hard delete, SKU removal, formula prose replacement)
  Pass 3: Flow Rebuild (close gaps after cuts, no new content)
  Pass 4: Weight Correction (title/content alignment throughout)
  Pass 5: Humanization (rhythm variation, hedge removal, punch ending)
  Target: 700-1000 words (600-1300 range, warnings outside)

- System prompt + STEP9 QA: add hard fails for
  LaTeX formulas (\[...\], \frac{}, \text{} etc) — destroys blog flow
  DR4 connector error (DR4=MPO-12, not LC duplex; FR4=LC duplex)
  Title/content mismatch (title topic must be the spine, not just the intro)

- Gold Standard 5: market alert / pricing article template
  (correct title alignment, no LaTeX, DR4=MPO-12, ending on topic)

- WRONG PATTERNS extended with 4 new entries covering above failures

- blog.ts: step log messages updated to 11-14/14; word count
  output shows % reduction and range warning (>1300 or <600)
2026-04-04 08:57:21 +02:00
Rene Fichtmueller
1e19365e96 feat: blog engine v5 — narrative control + linkedin post + min words fix
- STEP4b_NARRATIVE_CONTROL: new pipeline step after draft; detects wrong
  narrative (technology blamed instead of processes), applies anti-FUD filter,
  reality reframe ("this becomes a problem when..."), Flexoptix voice check
- System prompt: NARRATIVE CONTROL RULE added as absolute rule #1
- Gold Standard 4: corrected "compatible vs OEM" article added as reference
- Minimum words: STEP4 raised from 1500 to 2500 words (final output was 750)
- Reduction pass: 25-35% → 15-25%, target 1500-2000 words final
- STEP_LINKEDIN_POST: generates LinkedIn post ≤2800 chars (hard limit 3000);
  stores in blog_drafts.linkedin_post + linkedin_char_count column
- Pipeline now 14 steps: v5-narrative-control
- Migration 024: linkedin_post + linkedin_char_count columns in blog_drafts
2026-04-04 08:30:27 +02:00
Rene Fichtmueller
f616e0ebbe feat: blog engine v4 (reduction+style-lock passes) + flexoptix scraper fixes
Blog engine (fo-blog-pipeline.ts):
- Add STEP8b_REDUCTION: cuts article 25-35%, removes repeated concepts
- Add STEP8c_STYLE_LOCK: enforces tone consistency, fixes scope/OPM confusion,
  removes inline SKUs from article flow
- Add Gold Standard 3 to calibration (Style B troubleshooting example 2026-04-04)
- Pipeline now 12 steps (was 10), version bumped to v4-reduction-stylelock

blog.ts:
- Wire STEP8b and STEP8c into pipeline between Kill-AI-Tone and QA Check
- Update progress tracking to 12 total steps
- Update pipeline_version to 'v4-reduction-stylelock'

flexoptix-catalog.ts:
- Fix contentHash call: pass object directly, not JSON.stringify(object)

db.ts:
- price_verified=true set in content_hash early-return path (no new observation)
- image_verified=true auto-set in findOrCreateScrapedTransceiver on INSERT/UPDATE
2026-04-04 07:50:01 +02:00
Rene Fichtmueller
abea0cd8fa fix: remove orphaned floating text causing TypeScript build error in fo-blog-pipeline
Dead code leftover from STEP4_MASTER_DRAFT rewrite was sitting outside
any template literal, causing compilation failure. Removed duplicate
CONTEXT DATA RULES block and orphaned `{{OUTLINE}}`/`{{CONTEXT_DATA}}`
placeholders that were not wrapped in a string.
2026-04-03 00:51:59 +02:00
Rene Fichtmueller
3a61723126 fix(blog): complete pipeline rewrite — eliminate sections/bullets, fix DR4 wavelength, fix scope description
Core changes:
- HARD RULES rewritten: zero tolerance for ## headers, #### Scenario: patterns, bullet sections
- Gold article added as reference standard in STEP4_MASTER_DRAFT
- MANDATORY SECTIONS removed — replaced with continuous prose requirement
- STEP3_OUTLINE: now a flow plan (3-4 beats), not a section list
- STEP5_REALITY_INJECTION: no longer adds 'What Breaks' sections — injects into prose
- STEP9_QA_CHECK: format violations now primary HARD FAIL, above content checks
- DR4 wavelength fix: 1310nm = 0.35 dB/km (not 1550nm = 0.22 dB/km)
- Scope description fix: visual inspection tool ≠ loss measurement device
- Invented firmware version numbers now explicit HARD FAIL
2026-04-03 00:43:14 +02:00
Rene Fichtmueller
c81b9f5375 feat: hot topics now uses market_intelligence + LLM queue reset
Hot Topics:
- SOURCE 3b: market_intelligence table (15 items, 0.6+ relevance)
  with urgency mapping per intel_type + buy signal angles
- Fix news_articles: url → source_url (correct column name)
- Fix template literals: ${year} in string literals → backticks
- Increase limit: 6 → 15 topics returned
- Lower news cluster threshold: 2 → 1 article to form topic
- More research topics per day: 2 → 3
- More evergreen topics per day: 3 → 4
- Result: 24 total topics, 15 shown (was 8 total, 6 shown)

LLM Queue:
- Add resetOllamaQueue() export + auto-reset after 15min stall
- Add getQueueDepth() for monitoring
- New endpoints: GET /api/blog/llm/status, POST /api/blog/llm/reset-queue
2026-04-02 22:23:21 +02:00
Rene Fichtmueller
174078efdb feat: 100% verified data — no invented prices, part numbers, or designations
gatherBlogData():
- Fetches real prices from price_observations (last 30 days) per product
- Filters transceivers by speed extracted from topic keywords
- Enriches every product with verified_prices array + has_verified_price flag
- Joins DB products with vector search results (DB first — they have real prices)

contextData injection (blog.ts):
- [PRODUCT] lines: exact standard_name, form_factor, speed, reach, connector, dBm specs, Watts
- [VERIFIED PRICE] lines: real EUR/USD price, vendor, observed date, source URL
- [NO VERIFIED PRICE IN DB]: explicit tag — LLM must not invent a number
- [NO PRODUCT DATA AVAILABLE]: fallback when DB returns nothing

fo-blog-pipeline.ts system prompt:
- DATA INTEGRITY RULES block: prices/part numbers/vendors ONLY from context
- Never approximate with ~€350 or 'typically $200-600' for specific products
- Power specs only from [PRODUCT] data or REFERENCE VALUES

STEP4 context instructions:
- Explicit rules on how to use [VERIFIED PRICE] vs [NO VERIFIED PRICE]
- Invented data = HARD FAIL in QA

STEP9 QA — 3 new hard fail checks (30, 31, 32):
- Check 30: invented prices → remove or replace with flexoptix.net reference
- Check 31: invented part numbers → remove, use class name instead
- Check 32: invented vendor names → remove if not in known list
2026-04-01 17:27:55 +02:00
Rene Fichtmueller
580df8be01 blog: calibration v8 — AI phrasing blacklist, STEP8 6-step rewrite, Flexoptix author identity
- STRICTLY FORBIDDEN section: comprehensive AI phrasing blacklist (leverage, utilize,
  Furthermore, robust, seamless, delve into, in conclusion, etc.)
- STEP8 rewritten into 6 structured steps: hunt AI words, break rhythm, add human element,
  fix label formats, structural cleanup, ensure Flexoptix identity
- Banned sentence structures added (parallel triplets, same-length paragraphs)
- STEP6 author identity: reader should know this came from Flexoptix
- Version bump to 0.3.0
2026-04-01 00:43:38 +02:00
Rene Fichtmueller
52a04129e2 blog: calibration v7 — remove Cause/Fix/Example labels, integrate as prose narrative 2026-04-01 00:10:50 +02:00
Rene Fichtmueller
6b77b18842 fix(blog): extract article from QA, status badge ready/step X/10, calibration v6 Flexoptix balance 2026-03-31 23:52:56 +02:00
Rene Fichtmueller
ef0b0bb148 fix(llm): add 429 retry with exponential backoff + ollamaQueue concurrency guard 2026-03-31 21:45:46 +02:00
Rene Fichtmueller
01ad16464d blog: calibration v5 — anti-consulting-prose, correct loss budget math, vendor lock-in specifics 2026-03-31 21:26:39 +02:00
Rene Fichtmueller
45abd15fe4 blog: calibration v4 — technical accuracy + structure limits
- Fix SR4/DR4 fiber count: both use 8 fibers (4TX+4RX), difference is MMF vs SMF
- Fix power per port: 400G=10-15W/port, 800G=15-25W/port (not 1kW/port)
- Fix pricing context: always distinguish OEM ($1-5K) vs compatible ($200-600)
- Add HARD RULES 15-21: fiber count, power, pricing, no markdown headers,
  max 6 sections, no repeated topics, flow over format
- Add QA CALIBRATION FAILS 16-21: same rules enforced at QA step
- Add fiber/power reference tables with correct values
- Strip markdown (##/###/####/**) from all output — plain text only
- Add Style B gold example (10/10 validated prose article)
- Update STEP5 reality injection with correct SR4→DR4 description
- Update STEP8 kill-AI-tone to strip markdown headers + merge duplicates
2026-03-31 17:27:51 +02:00
Rene Fichtmueller
315a988775 feat(blog): add Style B prose calibration — 10/10 narrative flow standard
- CALIBRATION_GOLD_STANDARD now covers two validated styles: A (structured) and B (prose)
- Style B: no headers, no bullets, 1-3 sentence paragraphs, reframe ending
- STEP8_KILL_AI_TONE: prose conversion option for over-structured articles
- STEP4_MASTER_DRAFT: explicit style choice instruction (A vs B based on angle)
- Gold standard includes exact prose rhythm patterns from 10/10 human-reviewed article
- Wrong patterns expanded: symmetric sections, checklist endings, transition clichés
2026-03-31 16:48:10 +02:00
Rene Fichtmueller
f71ef2b20c feat(blog): regenerate button, SEO hashtags, calibration engine v2
- POST /api/blog/:id/regenerate — re-runs full 10-step LLM pipeline on existing draft
- Regenerate button visible when quality_issues present or status=review
- SEO keywords now displayed as clickable #hashtags (copy-to-clipboard)
- fo-blog-pipeline: added PoE misuse, DR4 mislabeling, ZR/DR4 conflation as hard QA fails
- fo-blog-pipeline: 14 hard rules in system prompt (was 10)
- fo-blog-pipeline: CALIBRATION_GOLD_STANDARD + withCalibration() from 10/10 human review
- System prompt now includes gold standard example on every pipeline run
2026-03-31 16:46:25 +02:00
Rene Fichtmueller
c01d69e02e fix(blog): harden pipeline prompts based on v0.2.1 blog review feedback
System prompt: 10 HARD RULES (non-negotiable, article fails QA without them)
- Mandatory WHAT BREAKS IN PRODUCTION section (2+ specific failures with symptoms/cause/fix)
- Mandatory HIDDEN COSTS section (cleaning, troubleshooting time, cabling redesign, training)
- Mandatory WHEN NOT TO USE section for every recommendation
- Absolute statement rule: NEVER without conditions/context
- Cabling reality: MPO polarity, SR4→DR4 migration, cleaning requirements
- Brutal hook requirement: not 'If you're still...' but 'You're about to sign a PO. Stop.'
- Minimum 2500 words (was 2000)

Step 5 (Reality Injection): Now checks for ALL mandatory sections and adds if missing
Step 9 (QA Check): Hard fail checks — article is NOT publishable without production failures + hidden costs
Feedback source: Human expert review scoring 7.5/10, targeting 9.5+
2026-03-31 09:24:08 +02:00
Rene Fichtmueller
d1d23ce31d feat(v0.2.1): data confidence tracking + validation + blog feedback system
- Migration 016: data_confidence column (vendor_verified/enriched_estimated/scraped_unverified)
- Migration 015: blog_feedback table with 8 quality scores + free text
- Validation script: 8 physics-based rules (wavelength↔fiber, reach plausibility, power limits)
- Blog feedback API: POST /api/blog/:id/feedback + training data export
- FO Blog Pipeline v3: 10-step Flexoptix Style prompts (Less bullshit. More engineering.)
- Auto-fix: wavelength↔fiber mismatches corrected automatically
2026-03-31 09:12:37 +02:00
Rene Fichtmueller
814325b349 feat: dashboard v2, blog expansion, market/cable MCP tools, switch asset scrapers, scraper utilities 2026-03-30 08:07:12 +02:00
Rene Fichtmueller
70447def02 feat: massive scraper expansion + hype cycle engine + lifecycle prediction
New scrapers:
- GBICS.com (BigCommerce, GBP prices, 10 categories, 78 products)
- Juniper HCT (Next.js SSR parser, 475 transceivers with specs/EOL)
- SFPcables.com (Magento store, 16 categories, 78 products)
- Fluxlight (BigCommerce, 6 pages, 118 products)
- Champion ONE (compatible vendor scraper)

Scraper fixes:
- 10Gtek: rewritten to parse HTML spec tables (152 products)
- Flexoptix: fix price extraction from Magento Hyva HTML
- Register all scrapers in CLI (--gbics, --juniper, --sfpcables, etc.)

Hype Cycle Engine enhancements:
- Data-driven enrichment from scraped vendor/price data
- Revenue lifecycle prediction (peak year, decline, revenue index)
- Regional adoption model (NA, China, APAC, Europe, RoW with lag coefficients)
- New API endpoints: /enriched, /lifecycle, /regional/:tech

DB growth: 89 → 1,168 transceivers, 0 → 416 prices, 6 vendors
Qdrant: 1,162 products embedded with nomic-embed-text

Research: Norton-Bass model, standards-to-market timelines, hype signals
2026-03-28 02:30:19 +13:00