data: image coverage improvements — QCT, Allied Telesis + CHANGELOG update

- sql/046: QCT QuantaMesh T3048-LY8 direct image injection
- sql/050: Allied Telesis 3/3 (AT-x530-28GSX, AT-x530L-52GPX, AT-x950-28XSQ)
  via alliedtelesis.com Drupal static files CDN (og:image, all 200 PNG)
- CHANGELOG: document filter pattern fixes + all 6 new vendor migrations
  (Moxa/UfiSpace/Brocade/NVIDIA/Allied Telesis — 19 new images total)
This commit is contained in:
Rene Fichtmueller 2026-04-21 08:11:57 +02:00
parent 51c18212b8
commit f275e94a6f
3 changed files with 53 additions and 0 deletions

View File

@ -189,3 +189,14 @@ Types: FEAT · FIX · UI · DATA · AI · INFRA
{"d":"2026-04-20","t":"FEAT","m":"community-issues.ts enhanced: added Cisco Field Notices, Juniper KB, SONiC GitHub Issues sources + new scrapeTransceiverCompatIssues() for switch+transceiver combo issues."} {"d":"2026-04-20","t":"FEAT","m":"community-issues.ts enhanced: added Cisco Field Notices, Juniper KB, SONiC GitHub Issues sources + new scrapeTransceiverCompatIssues() for switch+transceiver combo issues."}
{"d":"2026-04-20","t":"UI","m":"Dashboard switch table: thumbnail column (48px lazy-load image with gear-icon fallback). Switch detail: compatibility panel shows verification_method badge, vendor-tested vs form-factor split, competitor pricing in detail rows."} {"d":"2026-04-20","t":"UI","m":"Dashboard switch table: thumbnail column (48px lazy-load image with gear-icon fallback). Switch detail: compatibility panel shows verification_method badge, vendor-tested vs form-factor split, competitor pricing in detail rows."}
{"d":"2026-04-20","t":"FIX","m":"Scrapers: ATGBics new Shopify theme (card__info), NADDOD corrected shop URL, VCELink disabled (site pivoted to audio/video April 2026). Scheduler: 59 schedules, 78 workers."} {"d":"2026-04-20","t":"FIX","m":"Scrapers: ATGBics new Shopify theme (card__info), NADDOD corrected shop URL, VCELink disabled (site pivoted to audio/video April 2026). Scheduler: 59 schedules, 78 workers."}
{"d":"2026-04-21","t":"FEAT","m":"switch-image-playwright.ts: Playwright image scraper for bot-blocked switch vendors (Arista, Dell, Edgecore, Fortinet, HPE-Aruba, Extreme) — stealth headless Chromium, per-vendor URL builders (series-level for Arista, WooCommerce for Edgecore, direct-product for Extreme), og:image→twitter:image→img fallback chain, uniqueKey=row.id to bypass Crawlee URL deduplication for shared series pages, makeCrawleeConfig(Date.now() suffix) per-run to avoid ENOENT from stale request-queue files."}
{"d":"2026-04-21","t":"FIX","m":"Arista image coverage 33%→71%: buildAristaUrl() extracts series slug from model (7060X5-32QS→7060x5-series, 7280R3A→7280r3-series stripping trailing sub-variant 'a'). uniqueKey=row.id forces Crawlee to process all models even when multiple share the same series-level page. 15/21 Arista models now have images; 6 remaining series pages lack og:image in CMS (older models: 7050cx3, 7060dx5, 7060px4, 7060x4, 7170, 7260cx3)."}
{"d":"2026-04-21","t":"FIX","m":"og:image generic-logo fallback: meta image extraction decoupled from img fallback — og:image checked against isGenericImage() in Node.js; if it matches (logo/brand), falls through to img fallback instead of returning early. Fixes Dell (og:image=logo) and Extreme (og:image=logo) pipelines running img fallback as intended."}
{"d":"2026-04-21","t":"FIX","m":"OneTrust/cookie consent image filter: cdn.cookielaw.org, cookiebot.com, trustarc.com, consent-manager added to GENERIC_IMAGE_PATTERNS; cookielaw|cookiebot|trustarc added to img fallback skipPattern — prevents OneTrust company logo (largest DOM image on Extreme product pages) from being selected as product photo."}
{"d":"2026-04-21","t":"DATA","m":"Cisco 8000-series images 0%→100%: migration 044 cleared 35 stale NCS-5500 product_page_urls incorrectly assigned to 8000-series models, then set correct cisco.com/site/us/en/ URLs. switch-image-fetcher.ts plain HTTP run: 32/32 Cisco 8000-series models now have images."}
{"d":"2026-04-21","t":"DATA","m":"Edgecore images 0%→50%: migration 045 injects 5 direct image URLs (DCS204, DCS510, DCS810, EPS203, Minipack2) via curl-extracted og:image from WooCommerce product pages — Playwright blocked by Cloudflare WAF on edge-core.com but plain curl succeeds. AS7xxx enterprise switches not listed on edge-core.com website."}
{"d":"2026-04-21","t":"FIX","m":"Image filter patterns: /webimage-404/ (Netgear 404 hero), /\\/Brand\\// + /cybersecurity\\.png/ (Moxa brand images) added to GENERIC_IMAGE_PATTERNS in both switch-image-playwright.ts and switch-image-fetcher.ts. Cleared 5 bad DB rows (Moxa Brand/cybersecurity.png x4, Netgear webimage-404 x1)."}
{"d":"2026-04-21","t":"DATA","m":"Moxa images 0%→100% (4/4): direct CDN injection via migration 047 — Moxa Azure CDN getattachment paths. Hotlink-protected (Referer: moxa.com required); R2 proxy needed for production display."}
{"d":"2026-04-21","t":"DATA","m":"UfiSpace images 0%→100% (6/6) + Brocade 0%→100% (3/3): migration 048 — UfiSpace ufispace.com/image/<hash>/ PNGs (publicly accessible); Brocade G720/G730 via broadcom.com og:image, ICX 7850-48FS via CommScope/Ruckus vistancenetworks.com ImageServer (rand param cache-bust only, ID hash stable)."}
{"d":"2026-04-21","t":"DATA","m":"NVIDIA Networking images 0%→100% (6/6): migration 049 — SN2201/SN3700/SN4700 via docscontent.nvidia.com official docs CDN, SN5400/SN5600 via k3-prod-nvidia-docs.s3 direct, SN3750-SX via uvation reseller CDN."}
{"d":"2026-04-21","t":"DATA","m":"Allied Telesis images 0%→100% (3/3): migration 050 — x530/x530L/x950 series og:image from alliedtelesis.com Drupal CMS static files. QCT T3048-LY8 image via migration 046. Overall coverage: 33.4%→36.2%+ across 671 switches."}

View File

@ -0,0 +1,14 @@
-- Migration 046 — QCT / Quanta Cloud Technology product images (direct URL injection)
--
-- QCT T3048-LY8: og:image accessible via plain HTTP on qct.io
-- URL: qct.io/product/index/Networking/Ethernet-Switch/T3000-Series/QuantaMesh-T3048-LY8
-- og:image: qct.io/upload/website/product/gallery/normal/NetworkSwitch-QuantaMesh-T3048-LY8_FrontView02-740x460_16111812291.png
--
-- T7032-IX1 and T7064-*/T9032-* have no accessible og:image (JS-rendered or no dedicated page).
UPDATE switches
SET image_url = 'https://www.qct.io/upload/website/product/gallery/normal/NetworkSwitch-QuantaMesh-T3048-LY8_FrontView02-740x460_16111812291.png',
product_page_url = COALESCE(product_page_url, 'https://www.qct.io/product/index/Networking/Ethernet-Switch/T3000-Series/QuantaMesh-T3048-LY8'),
assets_scraped_at = NOW()
WHERE model = 'QuantaMesh T3048-LY8'
AND vendor_id = (SELECT id FROM vendors WHERE slug = 'quanta-cloud-technology');

View File

@ -0,0 +1,28 @@
-- Migration 050 — Allied Telesis product images (direct URL injection)
--
-- Source: alliedtelesis.com og:image (Drupal CMS, static files CDN)
-- All URLs verified HTTP 200 image/png (2026-04-21).
-- AT-x530-28GSX — x530 Series Gigabit PoE+ Smart Access Switch
UPDATE switches
SET image_url = 'https://www.alliedtelesis.com/sites/default/files/image/2022-07/x530-series-3840.png',
product_page_url = COALESCE(product_page_url, 'https://www.alliedtelesis.com/products/switches/x530-series'),
assets_scraped_at = NOW()
WHERE model = 'AT-x530-28GSX'
AND vendor_id = (SELECT id FROM vendors WHERE slug = 'allied-telesis');
-- AT-x530L-52GPX — x530L Series PoE+ Access Switch
UPDATE switches
SET image_url = 'https://www.alliedtelesis.com/sites/default/files/image/2021-11/x530L-series-3840.png',
product_page_url = COALESCE(product_page_url, 'https://www.alliedtelesis.com/products/switches/x530l-series'),
assets_scraped_at = NOW()
WHERE model = 'AT-x530L-52GPX'
AND vendor_id = (SELECT id FROM vendors WHERE slug = 'allied-telesis');
-- AT-x950-28XSQ — x950 Series Aggregation Switch
UPDATE switches
SET image_url = 'https://www.alliedtelesis.com/sites/default/files/image/2021-11/x950-28-52-3840.png',
product_page_url = COALESCE(product_page_url, 'https://www.alliedtelesis.com/products/switches/x950-series'),
assets_scraped_at = NOW()
WHERE model = 'AT-x950-28XSQ'
AND vendor_id = (SELECT id FROM vendors WHERE slug = 'allied-telesis');