rene/transceiver-db
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
381 Commits 5 Branches 9 Tags
Commit Graph

381 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Rene Fichtmueller
3f7395ea8d feat: sub-tabs Standards/Formfaktoren + rich form factor detail panel 
Standards tab now has two sub-tabs:
  - Standards: existing table (default)
  - Formfaktoren: full-page grid with search/family/status filters

Form factor detail panel (openFormFactorDetail) rebuilt as async:
  - Mini inline hype cycle curve SVG with dot at exact position
  - Buy signal per form factor (BUY_NOW / CONSIDER / WAIT / HOLD / AVOID)
  - Typed use-case bullets per form factor (20 mapped)
  - Top-10 transceiver mini-list (live fetch from /api/transceivers?form_factor=)
  - Clicking a transceiver row opens its full detail panel
  - Supersedes chain as clickable badges (navigate to that form factor)
  - flexoptix.net search link

FF_HYPE data: curated hype cycle positions for all 20 form factors
FF_USE_CASES: tailored use case lists per form factor
filterFormFactors: now also filters by search text + status dropdown
 2026-04-25 21:19:13 +02:00
Rene Fichtmueller
6eca121125 chore: update changelog for standards/form-factors expansion 2026-04-25 20:59:07 +02:00
Rene Fichtmueller
bfb43809a8 feat: standards audit + form factors reference 
- Migration 100: adds `description` column to standards (plain-language
  DE·EN for non-technical colleagues), fills all 63 standards incl.
  complete 200G tier (SR4/DR4/FR4/LR4/ER4/CR4), copper DAC variants,
  PON family (GPON/XG-PON1/NG-PON2/25G-PON), 1.6T emerging standard

- Migration 101: new form_factors table — 20 entries covering SFP family
  (SFP→SFP112), QSFP family (QSFP+→QSFP-DD800), OSFP family (OSFP→OSFP224),
  CFP family, legacy XFP/CXP with full_name, speed, channels, status,
  supersedes chain, and bilingual plain-language descriptions

- GET /api/form-factors — new endpoint, returns all form factors with
  transceiver_count join
- GET /api/form-factors/:name — single form factor detail

Dashboard Standards tab:
  - DB description shown as subtitle in standards table rows
  - Full DE + EN description in standard detail panel
  - New Form Factors grid section with status badges, speed, channel info,
    family color coding, supersedes chain
  - openFormFactorDetail() panel with full specs + transceiver link
  - Search extended to match description + notes fields
 2026-04-25 20:58:45 +02:00
Rene Fichtmueller
ff4bc34930 fix: remove IP restriction from internal-demand (Cloudflare tunnel breaks req.ip) 
JWT auth + RLS is the real security boundary. IP check was blocking legitimate
dashboard users accessing via Cloudflare tunnel (X-Forwarded-For = real user IP,
not 127.0.0.1). Added /api/internal/demand/stock-analysis: demand × live stock
combined analysis with reorder_urgency, coverage_days_de, momentum_ratio.
Dashboard: new Demand × Live Stock Analyse panel with critical/low/ok/overstock chips.
 2026-04-25 20:41:35 +02:00
Rene Fichtmueller
f162e03978 feat: Flexoptix internal demand intelligence + real forecast calibration 
- Migration 099: flexoptix_internal_demand table with RLS + v_demand_by_speed view
- Import script: AES-256-CBC decrypt → parse 8585 SKUs → upsert with velocity class
- 279 SKUs cross-referenced to transceiver catalog; 1288 with real demand data
- New /api/internal/demand/* routes (by-speed, velocity, hype-weights, forecast-input)
  — protected by JWT auth + localhost/LAN IP restriction middleware
- Forecast engine calibrated with real Flexoptix run-rates (demand_calibrated flag)
- Dashboard: real Flexoptix Sales Velocity panel replaces DEMO DATA in Warehouse tab
  with momentum indicators, velocity class breakdown, trend arrows
- Security: data stays on private server; RLS enforces is_internal=TRUE at DB layer
 2026-04-25 17:44:20 +02:00
Rene Fichtmueller
0611091c8f fix: migration 097 correct vendor slug ragile → ragile-networks 2026-04-25 16:37:16 +02:00
Rene Fichtmueller
6492522b11 data: image backfill migrations 094/097/098 — Cisco ASR9K/8K-MPA/NC55, Edgecore AS7946, Ragile 
Migration 094 (fixed): 12 Cisco models — 8K-MPA-4D/16H/16Z2D, A9K-8HG-FLEX-FC/SE/TR,
A9K-400G-DWDM-TR, N9348Y12C-SE1, NC55-36X100G-A-SE, ASR-9000V-24-A/DC-E, ASR-9922-RP-TR.
Fixed 4 bad URLs replaced with verified eBay CDN + NetworkTigers Shopify CDN sources.

Migration 097: +2 whitebox models — Ragile RA-B6510-48V8C (unixsurplus.com BigCommerce,
Micas M2-W6510-48V8C OEM equivalent), Edgecore AS7946-74XKDB (eBay CDN, 2024 TH5 800G).

Migration 098: +5 Cisco ASR 9903/9900 line cards — A9903-8HG-PEC, A9903-8HG-PEC-FC,
A9903-20HG-PEC-FC, A99-12X100GE-FC, A99-32HG-FC. All via eBay CDN (171–302KB JPEGs).
Coverage: 663 → 668/677 (98.7%). Remaining 9 models confirmed no accessible public images.
 2026-04-25 13:38:11 +02:00
Rene Fichtmueller
8c625ff1d2 feat(training): add TIP selflearning workflows 2026-04-25 12:21:56 +02:00
Rene Fichtmueller
ba998f4c01 fix: vendor_compat 0%→100%, price denorm, wiitek disabled, price-denorm scheduler 
- Migration 094: images for 12 Cisco 8K MPA + A9K-8HG-FLEX + ASR-9000V models
- Migration 095: price denorm refresh (EUR 679→1376, USD 166→835 with 180d window)
- Migration 096: bulk vendor_compat by form_factor — all 9013 transceivers now
  have OEM compatibility patterns (was 0/9013 because all slugs are scraped-*)
- wiitek.ts: disable dead scraper (wiitek.com unreachable since 2026-04, EAI_AGAIN)
- scheduler.ts: add compute:price-denorm job (daily 05:30 UTC) to keep
  street_price_usd/price_verified_eur fresh without manual migration runs
- seed-from-npm.ts: ON CONFLICT now also updates vendor_compat (was only updated_at)
 2026-04-25 08:55:21 +02:00
Rene Fichtmueller
108b2687d6 data: migration 093 — NCS 5500 main line card images (+6 models, 95.1%→96.0%) 
NC55-18H18F: it-market.com JPEG.
NC55-24X100G-SE: networkgenetics.net BigCommerce CDN 317KB PNG.
NC55-32T16Q4H-A: hummingbirdnetworks.com BigCommerce CDN 32KB.
NC55-36X100G-S: dedicatednetworksinc.com WordPress CDN 1.2MB PNG.
NC55-MOD-A-S: stack-systems.com Magento CDN.
NC55-MOD-A-SE-S: core92.com Odoo CDN 52KB.
 2026-04-25 07:51:15 +02:00
Rene Fichtmueller
3722cb4b10 data: migration 092 — Cisco image backfill (+48 models, 88.2%→95.1%) 
A99 ASR9900 line cards: router-switch.com CDN + SQL subquery reuse from
A9K counterparts (identical hardware) + NetworkOutlet/ZionNetworking/
NetworkTigers for models not on router-switch.
Catalyst 9600 LC/SUP: NetworkTigers + ITBargainCenter CDN.
NC55-MPA adapters: router-switch.com (157KB each).
NC55-24H12F-SE: networkgenetics.net BigCommerce CDN.
A900-IMA modules: NetworkTigers + router-switch.
A9903-20HG-PEC: TopParagonResource webp.
ASR-9922-RP-SE: NetworkTigers.
A9K-4HG-FLEX-X variants: SQL subquery from A9K-4HG-FLEX counterparts.
 2026-04-25 07:48:57 +02:00
Rene Fichtmueller
75ab607467 feat(data): migration 091 — Arista/Juniper SONiC HCL images (+7 models) 
7060CX-32S, 7050QX-32, 7050QX-32S, 7170-32CD, 7280CR3-32D4 (arista)
QFX5200-32C-S, QFX5210-64C (juniper)
All via NetworkTigers Shopify CDN (full-res, 40–120KB)
Estimated coverage: 616 → 623
 2026-04-21 23:09:46 +02:00
Rene Fichtmueller
2894013684 feat(data): migration 090 + apply script — Edgecore AS-series SONiC switches 
7 models: AS7312-54X, AS7312-54XS, AS7326-56X, AS7716-32X, AS7816-64X,
AS9716-32D, AS7512-32X. All from SONiC HCL Accton/Edgecore vendor.
Sources: edge-core.com official WP CDN + stordis.com + epsglobal.com.
Also adds scripts/apply-pending-migrations.sh for Erik DB catchup.
 2026-04-21 23:02:25 +02:00
Rene Fichtmueller
7cc910ce16 feat(data): migration 089 — Arista 7800R4/Cisco 8000+Catalyst+Nexus/Juniper PTX images 
8 models: 7800R4-36D2-LC, 8101-32FH, 8111-32EH, C9300X-24Y, C9500-48Y4C,
N9K-C93108TC-FX3P, PTX10001-36MR, PTX10004.
Coverage: 601 → 609 (89.6% → 90.8%).
 2026-04-21 22:39:06 +02:00
Rene Fichtmueller
f6218ad698 data: add image migration 088 — Ubiquiti USW + Phoenix Contact (3 models) 
USW-Enterprise-48-PoE + USW-Aggregation (cdn.ecomm.ui.com official Ubiquiti
ecommerce CDN), FL SWITCH 7528-2S (rspsupply.com distributor CDN, product
ID 2891026, same pattern as migration 079). Coverage: 598 → 601 (89.6%).
 2026-04-21 22:23:13 +02:00
Rene Fichtmueller
85b9803463 data: add image migration 087 — Cisco A9K license-variant reuse (3 models) 
A9K-4HG-FLEX-FC, A9K-8X100G-LB-TR, A9K-4X100GE base: all share identical
hardware with previously-covered -TR/-SE siblings. FC/SE/TR are Cisco
software license suffixes on the same physical line card. Coverage: 595 → 598.
 2026-04-21 22:20:19 +02:00
Rene Fichtmueller
cd8d7e816f data: add image migration 086 — Asterfusion/Arista/Ruckus/misc (8 models) 
CX732Q-N + CX564P-N (Asterfusion, official WP CDN), 7130-48LB (Arista,
Alta Technologies reseller), ICX 7850-48FS (Ruckus, networktigers.com),
E810-CQDA2 (Intel, ESAITech CDN), FSP 3000 CloudConnect (Adtran/ADVA,
nwrusa.com), MediorNet MicroN UHD (Riedel, riedel.net fileadmin CDN),
nGeniusONE InfiniStreamNG (Netscout, netscout.com Pantheon CDN).
Coverage: 587 → 595 (87.5% → 88.7%).
 2026-04-21 22:13:54 +02:00
Rene Fichtmueller
219b00523f data: add image migration 085 — mixed straggler batch (9 switches) 
New coverage: N540-24Q2C2DD-SYS (Cisco TD CDN), Apollo 9900 Series /
Ribbon (ribboncommunications.com), Seastone2 / Midstone-200i / Celestica
(servethehome.com + celestica.com/uploadedImages), RA-B6510-48V8C / Ragile
(Micas ODM equiv, BigCommerce CDN), QuantaMesh T7064-IX1D / QCT (IX4
EOL family image). CDN upgrade: DS3000/DS4000/DS5000 foleon → official
celestica.com/uploadedImages. Coverage: 581 → 587 (86.6% → 87.5%).
 2026-04-21 22:03:08 +02:00
Rene Fichtmueller
6357145316 data: add image migrations 082-084 — Cisco A9K line cards, NC57/NCS1K14, A9K bulk 
Migration 082 (13 models): A9K-4HG-FLEX-TR/SE, A9K-4T-B/E/L, A9K-4T16GE-SE/TR,
A9K-4X100GE-FC/SE/TR, A9K-MOD400-SE/TR, A9K-MOD80-SE/TR.
Sources: networktigers.com Shopify CDN + router-switch.com.

Migration 083 (15 models): NC57-18DD-SE/24DD/36H-SE/36H6D-S/48Q2D-S/48Q2D-SE-S/
MOD-S/MOD-SE-S + NC57-MPA-12L-S/1FH1D-S/2D4H-S + NCS1K14-2.4T-K9/L-K9/X-K9/TXL-K9.
Sources: Cisco TD CDN (numbered IDs) + datasheet-c78-742016 JCR renditions.

Migration 084 (62 models): A9K-8T/4-B/E/L (slash-format fix), A9K-16T/8-B,
1X/2X/8X/16X 100GE, 20HG-FLEX-FC/SE/TR, 24X10GE variants, 2T20GE-B/E/L,
36X10GE-SE/TR, 40GE-B/E/L/SE/TR, 48X10GE variants, RSP440/RSP880, SIP-700,
all MPA modules (11 non-FC + 3 dedicated FC + 6 FC reuse non-FC hardware image).
Sources: networktigers.com + cdn.shopify.com + router-switch.com + cloudappliances.co.uk.

Coverage after 084: 519 → ~581 (77.3% → ~86.6%).
 2026-04-21 21:20:47 +02:00
Rene Fichtmueller
772af1b6fb data: add image migrations 080-081 — Avaya, Advantech, Cisco N540/N540X/N560-IMA/ASR-9900-RP 
Migration 080 (3 images): Avaya ERS 4950GTS-PWR+ (planetrefurbished.com),
Advantech EKI-7720G-4FI and EKI-9516G-4GMXP (advdownload.advantech.com CDN).

Migration 081 (26 UPDATEs): Cisco N560 IMA modules (4 models, Cisco TD CDN
524xxx range), NCS 540 fixed-port (6Z/FH-AGG/FH-CSR/24Q8L2DD, TD CDN + manualslib),
NCS 540X (10 models, TD CDN 521-522xxx + eBay CDN + signellent.com),
ASR-9900-RP-SE/TR (brightstarsystems.com); upgrades ASR-9902/9903 and NCS1001/1002-K9
to official Cisco support CDN. Coverage: 466 → 491 (69.4% → 73.2%).
 2026-04-21 16:00:36 +02:00
Rene Fichtmueller
50f0e738c9 feat(data): image migrations 078-079 — Cisco NCS1K4/ASR9000v, WAGO, WatchGuard, ADTRAN, Phoenix Contact 
Migration 078: 21 Cisco models — ASR-9001-S, ASR-9000V-AC/DC-A, A9KV-V2-DC-A/DC-E
  (specific images), NCS1001/1002/1K-EDFA, all 13 NCS1K4 chassis variants.
  Sources: networktigers.com + router-switch.com

Migration 079: WAGO 852-1505, WatchGuard Firebox M5800, ADTRAN NetVanta 1560-48P,
  Phoenix Contact FL SWITCH 4808E-16FX-4GC (4 models).
  Sources: gilautomation.com Shopify, watchguard.com help-center, portal.adtran.com

Coverage: 443 → 466 images (66.0% → 69.4%)
 2026-04-21 15:01:39 +02:00
Rene Fichtmueller
356add2e49 feat(data): image migrations 075-077 — Cisco NCS 9300 SE1, QCT, QNAP, Sophos, Barracuda, Peplink, Westermo 
Migration 075: Cisco Nexus 9300 SE1 (7 models) — poster-image + support CDN
Migration 076: QuantaMesh (3), QNAP QSW-M5216-1T, Sophos CS210-48FP (5 models)
Migration 077: Barracuda F900, Peplink SD Switch 24-Port, Westermo Lynx 5612 (3 models)

Coverage: 428 → 443 images (63.8% → 66.0%)
 2026-04-21 14:49:29 +02:00
Rene Fichtmueller
ba5ab9c502 chore: redact internal IP from CHANGELOG before public push 2026-04-21 14:31:07 +02:00
Rene Fichtmueller
5d7a7a9876 data: add switch images migrations 073-074 (Cisco ASR9k/NCS540 + multi-vendor) 
Migration 073 — Cisco ASR 9000 + NCS 540 Series (18 entries):
  ASR-9001/9901/9902/9903 + FC fabric-card variants
  A9KV-V2-AC/DC-A/DC-E satellite shelf
  N540-28Z4C, N540-24Z8Q2C, N540-12Z20G, N540-ACC, N540X-16Z4G8Q2C
  Sources: networktigers.com Shopify CDN, tempestns.com WP CDN
  Coverage: 400 → 418 (62.3%)

Migration 074 — Extreme / Ruijie / Ruckus / ZTE / Edgecore (10 entries):
  Extreme SLX 9740-40C, X695-48Y-8C, 5520-48T
  Ruijie RG-S6920-4C, RG-S5760C-24SFP/8GT8XS-X
  Ruckus ICX 7150-48PF, ICX 7550-48ZP
  ZTE ZXR10 5960-56PM-H, ZXR10 9908
  Edgecore AS7712-32X
  Coverage: 418 → 428 (63.8%)

All URLs verified HTTP 200. A/D power variants share chassis images.
 2026-04-21 14:30:42 +02:00
Rene Fichtmueller
4e927acf55 data: switch image coverage 065-072 — Cisco/Juniper/Arista/NVIDIA/Huawei/Nokia/Dell/Extreme/HPE/Ubiquiti/Supermicro/Celestica/Asterfusion/FS.com/Edgecore 
Migrations 065-072: 72 verified image URLs across all 17 tier-1 vendors.
- 065: Cisco 8000/Catalyst/Nexus/NCS (14) — cisco.com/c/dam doc CDN
- 066: Juniper EX/MX/QFX (10) — juniper.net/content/dam image library
- 067: Arista remaining 7000-series (11) — arista.com QSG front-panel PNGs
- 068: NVIDIA Mellanox SN-series (5) — docscontent.nvidia.com dims4 CDN
- 069: Huawei CloudEngine/NE40E (5) + Nokia IXR-D3L/7750 SR-14s (2)
- 070: Dell PowerSwitch ON-series (5) + Extreme Networks 8720/X465 (2)
- 071: HPE Aruba CX 6300M/8100/8360 (3) + Ubiquiti USW (3) + Supermicro (2)
- 072: Celestica DS3000/4000/5000 (3) + Asterfusion CX-N (3) + FS.com (2) + Edgecore (2)

All URLs verified HTTP 200 (2026-04-21). 5 models skipped (no public image found):
Arista 7280R3A-48D5, 750-36Y; NVIDIA SN3750-SX; Nokia 7750 SR-1 (hotlink-protected), 7220 IXR-H4.
 2026-04-21 11:34:38 +02:00
Rene Fichtmueller
382de40d5d data: Avaya/NetApp/Keysight/A10/Evertz/RAD/Ekinops/DrayTek/Fujitsu/Broadcom/Calix/Citrix images (migration 064) 2026-04-21 10:18:51 +02:00
Rene Fichtmueller
3cd6d44ea5 data: Sophos XGS 6500 image + Zyxel XS3800-28 URL fix (migration 063) 2026-04-21 10:16:51 +02:00
Rene Fichtmueller
b7e8097a57 data: Cambium/Gigamon/SonicWall/Planet/PaloAlto/Westermo/Zyxel/Synology/TRENDnet/Waystream/Kemp/LANCOM images (migrations 061-062) 2026-04-21 10:10:32 +02:00
Rene Fichtmueller
906619e602 docs: update CHANGELOG_PENDING with migrations 055-060 2026-04-21 09:57:06 +02:00
Rene Fichtmueller
d3eb53ad1b data: HPE Aruba CX + Extreme Networks images 0%→100% (migration 060) 2026-04-21 09:56:11 +02:00
Rene Fichtmueller
6827f6081f data: Arista/Edgecore/Fortinet/Dell/Huawei/D-Link/Netgear/Check Point/Ruckus images (migrations 057-059) 2026-04-21 09:55:22 +02:00
Rene Fichtmueller
3b42f0aa90 data: ALE/H3C/Hirschmann/Ciena/Netberg images 0%→100% (migration 056) 2026-04-21 09:50:46 +02:00
Rene Fichtmueller
b9ab664a56 data: MikroTik CRS/CCR CDN images + NVIDIA ConnectX-7 (migration 055) 2026-04-21 09:48:30 +02:00
Rene Fichtmueller
e2061c6c4d data: direct image injection for Nokia, F5, Delta Networks, Siemens, TP-Link 
Migration 051: TP-Link TL-SG3452XP + TL-SX3016F via static.tp-link.com CDN
Migration 052: Nokia 6/6 — 7220 IXR-D3L/H4 (docs.nokia.com graphics),
  7250 IXR-10 + 7750 SR-1 (tempestns.com), SR-14s (telecomcauliffe.com),
  SR-1e (docs hardwareBanner — no standalone public image available)
Migration 053: F5 BIG-IP i5800/i10800 (wtit.com), i15800 (blueally CDN)
Migration 054: Delta Networks 4/4 (hardwarenation.com + manualslib),
  Siemens SCALANCE 4/4 — X-200/X-300/X-500 via images.sw.cdn.siemens.com

All 14 URLs verified HTTP 200 with correct image content-type (2026-04-21).
CHANGELOG_PENDING.md updated for all 4 migrations.
 2026-04-21 08:42:14 +02:00
Rene Fichtmueller
466cda285b data: image coverage improvements — QCT, Allied Telesis + CHANGELOG update 
- sql/046: QCT QuantaMesh T3048-LY8 direct image injection
- sql/050: Allied Telesis 3/3 (AT-x530-28GSX, AT-x530L-52GPX, AT-x950-28XSQ)
  via alliedtelesis.com Drupal static files CDN (og:image, all 200 PNG)
- CHANGELOG: document filter pattern fixes + all 6 new vendor migrations
  (Moxa/UfiSpace/Brocade/NVIDIA/Allied Telesis — 19 new images total)
 2026-04-21 08:11:57 +02:00
Rene Fichtmueller
bbc6f560dd fix: add image filter patterns and direct URL migrations for 6 vendors 
- switch-image-playwright.ts + switch-image-fetcher.ts: add filter patterns
  for /webimage-404/ (Netgear 404 hero), /Brand/ + /cybersecurity.png/
  (Moxa brand marketing images not product photos)
- sql/047: Moxa 4/4 models — CDN getattachment paths (hotlink-protected,
  Referer: moxa.com required; R2 proxy needed for production display)
- sql/048: UfiSpace 6/6 models — ufispace.com/image/<hash>/ direct PNGs;
  Brocade G720+G730 — broadcom.com og:image; ICX 7850-48FS — CommScope/Ruckus
  vistancenetworks.com ImageServer (rand param is cache-bust only, not auth)
- sql/049: NVIDIA SN-series 6/6 — docscontent.nvidia.com (SN2201/3700/4700)
  and S3 direct (SN5400/5600); SN3750-SX via uvation reseller CDN
 2026-04-21 07:57:55 +02:00
Rene Fichtmueller
b65e4452db fix: add error-graphic, icon-library, illustration filters to GENERIC_IMAGE_PATTERNS 
- /404[-_]error/i, /error[-_]graphic/i — Broadcom 404-ERROR-GRAPHIC.png
- /\/icon[-_]library\//i — D-Link navigation/icon-library path images
- /[-_]illustration[._]/i — Arista Cloud-Legacy_Illustration and similar diagrams
- Nokia banner, Huawei marketing, banners/ path patterns (Playwright scraper)
- Cookie consent patterns synced to switch-image-fetcher.ts (was only in Playwright)
 2026-04-21 07:38:01 +02:00
Rene Fichtmueller
f4afe14af4 feat: add 12 new vendor URL builders to Playwright image scraper 
- Nokia, Huawei, Ciena, Moxa, D-Link, Alcatel-Lucent Enterprise,
  Asterfusion, Brocade: passthrough builders (use stored product_page_url)
- NVIDIA Networking: SN-series URL builder (sn5600 → /ethernet-switching/sn5600/)
- Netgear: lowercase model slug builder for /business/wired/switches/fully-managed/
- UfiSpace: hardcoded sitemap-verified URL map (all 6 S9xxx models)
- QCT: hardcoded URL map for T3048-LY8 and T7032-IX1
- Add Nokia banner / Huawei marketing image patterns to GENERIC_IMAGE_PATTERNS
 2026-04-21 07:24:11 +02:00
Rene Fichtmueller
6a4b4700cb data: inject Edgecore product images directly (Playwright blocked by 403) 
Edgecore blocks headless browsers (Playwright 403) but serves og:image via
plain HTTP. 5 models resolved via direct curl extraction:
- DCS204, DCS510, DCS810, EPS203 → edge-core.com/product/<slug>/
- Minipack2 → minipack-as8000-open-modular-platform product page
AS7535/AS7726/AS7946/AS9516 not on Edgecore's public WooCommerce site.
 2026-04-21 07:07:24 +02:00
Rene Fichtmueller
8f36eff956 fix(scraper): filter OneTrust/cookie-consent images + skip in img fallback 
cdn.cookielaw.org logos appear as the largest DOM image on Dell/Extreme
product pages when the cookie consent overlay is present. Added to both
GENERIC_IMAGE_PATTERNS (isGenericImage filter) and img fallback skipPattern
so the next-largest actual product image can be found.
 2026-04-21 06:45:41 +02:00
Rene Fichtmueller
d67fbe31da fix(scraper): fall through to img fallback when og:image is generic/logo 
Previously: if og:image existed (even as a Dell logo URL), page.evaluate() returned
early and the img fallback was never tried. Now: meta tags are extracted first, then
isGenericImage() is checked in Node.js, and the img fallback runs if meta image is null
or generic. This allows vendors like Dell (og:image = logo) to still get product images
via the DOM fallback.
 2026-04-21 06:36:12 +02:00
Rene Fichtmueller
09d3a60b7c fix(scraper): fix Edgecore/Extreme URL builders, broaden img fallback, fix ENOENT 
- buildEdgecoreUrl: /product/<slug>/ (WooCommerce, no .html) with EDGECORE_SLUG_MAP
  for AS7712-32X→as7712-32x-ec, Minipack2→minipack-as8000-open-modular-platform
- buildFortinetUrl: returns null (all pages redirect to generic, no usable og:image)
- buildExtremeUrl: direct product URL (extremenetworks.com/product/<slug>)
- img fallback: remove strict 'product/switch/router/hardware' path requirement;
  now takes largest image >=200x150px excluding flags/icons/spinners — isGenericImage()
  filters hero/banner/logo afterward
- ENOENT fix: unique per-run Crawlee storage dir (timestamp suffix) prevents
  stale request-queue file contamination between back-to-back vendor runs
 2026-04-21 06:33:32 +02:00
Rene Fichtmueller
87b9416592 fix(scraper): fix Arista series-level URL builder + bypass Crawlee URL deduplication 
- buildAristaUrl() now extracts series prefix (7060X5-32QS → 7060x5-series)
  instead of individual model URLs that lack og:image
- Strip trailing sub-variant 'A' so R3A → R3 series page
- Add uniqueKey: row.id to each request — prevents Crawlee from deduplicating
  models that share the same series URL (e.g. 7060x5-series)
- For Arista: always prefer fresh builder URL over stored product_page_url
  so stale individual-model URLs don't override correct series pages
 2026-04-21 06:22:41 +02:00
Rene Fichtmueller
18a9e1346e feat: Playwright image scraper for bot-blocked vendors (Arista/Dell/Edgecore/Fortinet/Extreme) 2026-04-21 06:16:05 +02:00
Rene Fichtmueller
653824f23b fix: Cisco line card URL mapping (8800/84/86 → 8000 family page, skip ASR9K logo-only) 2026-04-21 00:49:32 +02:00
Rene Fichtmueller
c9333ab5ea fix: MikroTik hardcoded slug map for + models (crs305/312/317/326) 2026-04-21 00:45:41 +02:00
Rene Fichtmueller
9618a4f0e0 fix: Cisco 8000 builder URL + MikroTik lowercase + new vendor builders 
URL builder fixes:
- Cisco 8000: update to new /site/us/en/ URL scheme (family page, not per-model)
- MikroTik: fix to lowercase+underscore format (was uppercase, caused 404)
- Fortinet: set to null — JS-rendered pages, all redirect to generic page
- Alcatel-Lucent Enterprise slug added to dispatcher (was missing, caused 0 hits)
- Add Quanta, Allied Telesis, Ufispace, Netgear URL builders
- NVIDIA: skip ConnectX/BlueField non-switch models

Migration 044:
- Clear 35 wrong NCS-5500 URLs from Cisco 8000-series models
- Pre-set correct 8000-series family URL for 21 models without images
 2026-04-21 00:41:31 +02:00
Rene Fichtmueller
f340482752 fix: monitor-erik.sh — correct Erik SSH target + fix awk header skip 2026-04-21 00:34:28 +02:00
Rene Fichtmueller
9e6be570a3 feat: more switch image coverage + system health metrics + Erik monitor 
switch-image-fetcher:
- Add Fortinet URL builder (11 FortiSwitch models)
- Add Quanta Cloud Technology, Allied Telesis, Ufispace, Netgear URL builders
- Fix alcatel-lucent-enterprise slug missing from URL_BUILDERS dispatcher
- Fix NVIDIA builder to skip ConnectX/BlueField adapters (not switches)
- Add aruba slug alias for hpe-aruba

health endpoint:
- Add system metrics: CPU load (1/5/15m), memory usage, disk usage
- Add load_status indicator (ok/busy/overloaded)
- Expose process RSS memory
- Used by external monitors

scripts/monitor-erik.sh:
- Cron-ready health check script for Claudi (.82) and Raspberry Pis
- Checks TIP API health endpoint (load, memory, disk, DB latency)
- Checks PM2 process state via SSH (errored/stopped detection)
- ntfy.sh push notifications (set NTFY_TOPIC env var)
- Includes systemd service + timer unit comments for auto-install
 2026-04-21 00:31:43 +02:00
Rene Fichtmueller
823b64bd24 perf: load-aware scraper guard + higher rate limits + /tmp crawlee storage 2026-04-20 23:35:02 +02:00