rene/transceiver-db
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
274 Commits 5 Branches 9 Tags
Commit Graph

53 Commits

Author SHA1 Message Date
Rene Fichtmueller
108b2687d6 data: migration 093 — NCS 5500 main line card images (+6 models, 95.1%→96.0%) 
NC55-18H18F: it-market.com JPEG.
NC55-24X100G-SE: networkgenetics.net BigCommerce CDN 317KB PNG.
NC55-32T16Q4H-A: hummingbirdnetworks.com BigCommerce CDN 32KB.
NC55-36X100G-S: dedicatednetworksinc.com WordPress CDN 1.2MB PNG.
NC55-MOD-A-S: stack-systems.com Magento CDN.
NC55-MOD-A-SE-S: core92.com Odoo CDN 52KB.
 2026-04-25 07:51:15 +02:00
Rene Fichtmueller
3722cb4b10 data: migration 092 — Cisco image backfill (+48 models, 88.2%→95.1%) 
A99 ASR9900 line cards: router-switch.com CDN + SQL subquery reuse from
A9K counterparts (identical hardware) + NetworkOutlet/ZionNetworking/
NetworkTigers for models not on router-switch.
Catalyst 9600 LC/SUP: NetworkTigers + ITBargainCenter CDN.
NC55-MPA adapters: router-switch.com (157KB each).
NC55-24H12F-SE: networkgenetics.net BigCommerce CDN.
A900-IMA modules: NetworkTigers + router-switch.
A9903-20HG-PEC: TopParagonResource webp.
ASR-9922-RP-SE: NetworkTigers.
A9K-4HG-FLEX-X variants: SQL subquery from A9K-4HG-FLEX counterparts.
 2026-04-25 07:48:57 +02:00
Rene Fichtmueller
75ab607467 feat(data): migration 091 — Arista/Juniper SONiC HCL images (+7 models) 
7060CX-32S, 7050QX-32, 7050QX-32S, 7170-32CD, 7280CR3-32D4 (arista)
QFX5200-32C-S, QFX5210-64C (juniper)
All via NetworkTigers Shopify CDN (full-res, 40–120KB)
Estimated coverage: 616 → 623
 2026-04-21 23:09:46 +02:00
Rene Fichtmueller
2894013684 feat(data): migration 090 + apply script — Edgecore AS-series SONiC switches 
7 models: AS7312-54X, AS7312-54XS, AS7326-56X, AS7716-32X, AS7816-64X,
AS9716-32D, AS7512-32X. All from SONiC HCL Accton/Edgecore vendor.
Sources: edge-core.com official WP CDN + stordis.com + epsglobal.com.
Also adds scripts/apply-pending-migrations.sh for Erik DB catchup.
 2026-04-21 23:02:25 +02:00
Rene Fichtmueller
7cc910ce16 feat(data): migration 089 — Arista 7800R4/Cisco 8000+Catalyst+Nexus/Juniper PTX images 
8 models: 7800R4-36D2-LC, 8101-32FH, 8111-32EH, C9300X-24Y, C9500-48Y4C,
N9K-C93108TC-FX3P, PTX10001-36MR, PTX10004.
Coverage: 601 → 609 (89.6% → 90.8%).
 2026-04-21 22:39:06 +02:00
Rene Fichtmueller
f6218ad698 data: add image migration 088 — Ubiquiti USW + Phoenix Contact (3 models) 
USW-Enterprise-48-PoE + USW-Aggregation (cdn.ecomm.ui.com official Ubiquiti
ecommerce CDN), FL SWITCH 7528-2S (rspsupply.com distributor CDN, product
ID 2891026, same pattern as migration 079). Coverage: 598 → 601 (89.6%).
 2026-04-21 22:23:13 +02:00
Rene Fichtmueller
85b9803463 data: add image migration 087 — Cisco A9K license-variant reuse (3 models) 
A9K-4HG-FLEX-FC, A9K-8X100G-LB-TR, A9K-4X100GE base: all share identical
hardware with previously-covered -TR/-SE siblings. FC/SE/TR are Cisco
software license suffixes on the same physical line card. Coverage: 595 → 598.
 2026-04-21 22:20:19 +02:00
Rene Fichtmueller
cd8d7e816f data: add image migration 086 — Asterfusion/Arista/Ruckus/misc (8 models) 
CX732Q-N + CX564P-N (Asterfusion, official WP CDN), 7130-48LB (Arista,
Alta Technologies reseller), ICX 7850-48FS (Ruckus, networktigers.com),
E810-CQDA2 (Intel, ESAITech CDN), FSP 3000 CloudConnect (Adtran/ADVA,
nwrusa.com), MediorNet MicroN UHD (Riedel, riedel.net fileadmin CDN),
nGeniusONE InfiniStreamNG (Netscout, netscout.com Pantheon CDN).
Coverage: 587 → 595 (87.5% → 88.7%).
 2026-04-21 22:13:54 +02:00
Rene Fichtmueller
219b00523f data: add image migration 085 — mixed straggler batch (9 switches) 
New coverage: N540-24Q2C2DD-SYS (Cisco TD CDN), Apollo 9900 Series /
Ribbon (ribboncommunications.com), Seastone2 / Midstone-200i / Celestica
(servethehome.com + celestica.com/uploadedImages), RA-B6510-48V8C / Ragile
(Micas ODM equiv, BigCommerce CDN), QuantaMesh T7064-IX1D / QCT (IX4
EOL family image). CDN upgrade: DS3000/DS4000/DS5000 foleon → official
celestica.com/uploadedImages. Coverage: 581 → 587 (86.6% → 87.5%).
 2026-04-21 22:03:08 +02:00
Rene Fichtmueller
6357145316 data: add image migrations 082-084 — Cisco A9K line cards, NC57/NCS1K14, A9K bulk 
Migration 082 (13 models): A9K-4HG-FLEX-TR/SE, A9K-4T-B/E/L, A9K-4T16GE-SE/TR,
A9K-4X100GE-FC/SE/TR, A9K-MOD400-SE/TR, A9K-MOD80-SE/TR.
Sources: networktigers.com Shopify CDN + router-switch.com.

Migration 083 (15 models): NC57-18DD-SE/24DD/36H-SE/36H6D-S/48Q2D-S/48Q2D-SE-S/
MOD-S/MOD-SE-S + NC57-MPA-12L-S/1FH1D-S/2D4H-S + NCS1K14-2.4T-K9/L-K9/X-K9/TXL-K9.
Sources: Cisco TD CDN (numbered IDs) + datasheet-c78-742016 JCR renditions.

Migration 084 (62 models): A9K-8T/4-B/E/L (slash-format fix), A9K-16T/8-B,
1X/2X/8X/16X 100GE, 20HG-FLEX-FC/SE/TR, 24X10GE variants, 2T20GE-B/E/L,
36X10GE-SE/TR, 40GE-B/E/L/SE/TR, 48X10GE variants, RSP440/RSP880, SIP-700,
all MPA modules (11 non-FC + 3 dedicated FC + 6 FC reuse non-FC hardware image).
Sources: networktigers.com + cdn.shopify.com + router-switch.com + cloudappliances.co.uk.

Coverage after 084: 519 → ~581 (77.3% → ~86.6%).
 2026-04-21 21:20:47 +02:00
Rene Fichtmueller
772af1b6fb data: add image migrations 080-081 — Avaya, Advantech, Cisco N540/N540X/N560-IMA/ASR-9900-RP 
Migration 080 (3 images): Avaya ERS 4950GTS-PWR+ (planetrefurbished.com),
Advantech EKI-7720G-4FI and EKI-9516G-4GMXP (advdownload.advantech.com CDN).

Migration 081 (26 UPDATEs): Cisco N560 IMA modules (4 models, Cisco TD CDN
524xxx range), NCS 540 fixed-port (6Z/FH-AGG/FH-CSR/24Q8L2DD, TD CDN + manualslib),
NCS 540X (10 models, TD CDN 521-522xxx + eBay CDN + signellent.com),
ASR-9900-RP-SE/TR (brightstarsystems.com); upgrades ASR-9902/9903 and NCS1001/1002-K9
to official Cisco support CDN. Coverage: 466 → 491 (69.4% → 73.2%).
 2026-04-21 16:00:36 +02:00
Rene Fichtmueller
50f0e738c9 feat(data): image migrations 078-079 — Cisco NCS1K4/ASR9000v, WAGO, WatchGuard, ADTRAN, Phoenix Contact 
Migration 078: 21 Cisco models — ASR-9001-S, ASR-9000V-AC/DC-A, A9KV-V2-DC-A/DC-E
  (specific images), NCS1001/1002/1K-EDFA, all 13 NCS1K4 chassis variants.
  Sources: networktigers.com + router-switch.com

Migration 079: WAGO 852-1505, WatchGuard Firebox M5800, ADTRAN NetVanta 1560-48P,
  Phoenix Contact FL SWITCH 4808E-16FX-4GC (4 models).
  Sources: gilautomation.com Shopify, watchguard.com help-center, portal.adtran.com

Coverage: 443 → 466 images (66.0% → 69.4%)
 2026-04-21 15:01:39 +02:00
Rene Fichtmueller
356add2e49 feat(data): image migrations 075-077 — Cisco NCS 9300 SE1, QCT, QNAP, Sophos, Barracuda, Peplink, Westermo 
Migration 075: Cisco Nexus 9300 SE1 (7 models) — poster-image + support CDN
Migration 076: QuantaMesh (3), QNAP QSW-M5216-1T, Sophos CS210-48FP (5 models)
Migration 077: Barracuda F900, Peplink SD Switch 24-Port, Westermo Lynx 5612 (3 models)

Coverage: 428 → 443 images (63.8% → 66.0%)
 2026-04-21 14:49:29 +02:00
Rene Fichtmueller
ba5ab9c502 chore: redact internal IP from CHANGELOG before public push 2026-04-21 14:31:07 +02:00
Rene Fichtmueller
5d7a7a9876 data: add switch images migrations 073-074 (Cisco ASR9k/NCS540 + multi-vendor) 
Migration 073 — Cisco ASR 9000 + NCS 540 Series (18 entries):
  ASR-9001/9901/9902/9903 + FC fabric-card variants
  A9KV-V2-AC/DC-A/DC-E satellite shelf
  N540-28Z4C, N540-24Z8Q2C, N540-12Z20G, N540-ACC, N540X-16Z4G8Q2C
  Sources: networktigers.com Shopify CDN, tempestns.com WP CDN
  Coverage: 400 → 418 (62.3%)

Migration 074 — Extreme / Ruijie / Ruckus / ZTE / Edgecore (10 entries):
  Extreme SLX 9740-40C, X695-48Y-8C, 5520-48T
  Ruijie RG-S6920-4C, RG-S5760C-24SFP/8GT8XS-X
  Ruckus ICX 7150-48PF, ICX 7550-48ZP
  ZTE ZXR10 5960-56PM-H, ZXR10 9908
  Edgecore AS7712-32X
  Coverage: 418 → 428 (63.8%)

All URLs verified HTTP 200. A/D power variants share chassis images.
 2026-04-21 14:30:42 +02:00
Rene Fichtmueller
4e927acf55 data: switch image coverage 065-072 — Cisco/Juniper/Arista/NVIDIA/Huawei/Nokia/Dell/Extreme/HPE/Ubiquiti/Supermicro/Celestica/Asterfusion/FS.com/Edgecore 
Migrations 065-072: 72 verified image URLs across all 17 tier-1 vendors.
- 065: Cisco 8000/Catalyst/Nexus/NCS (14) — cisco.com/c/dam doc CDN
- 066: Juniper EX/MX/QFX (10) — juniper.net/content/dam image library
- 067: Arista remaining 7000-series (11) — arista.com QSG front-panel PNGs
- 068: NVIDIA Mellanox SN-series (5) — docscontent.nvidia.com dims4 CDN
- 069: Huawei CloudEngine/NE40E (5) + Nokia IXR-D3L/7750 SR-14s (2)
- 070: Dell PowerSwitch ON-series (5) + Extreme Networks 8720/X465 (2)
- 071: HPE Aruba CX 6300M/8100/8360 (3) + Ubiquiti USW (3) + Supermicro (2)
- 072: Celestica DS3000/4000/5000 (3) + Asterfusion CX-N (3) + FS.com (2) + Edgecore (2)

All URLs verified HTTP 200 (2026-04-21). 5 models skipped (no public image found):
Arista 7280R3A-48D5, 750-36Y; NVIDIA SN3750-SX; Nokia 7750 SR-1 (hotlink-protected), 7220 IXR-H4.
 2026-04-21 11:34:38 +02:00
Rene Fichtmueller
382de40d5d data: Avaya/NetApp/Keysight/A10/Evertz/RAD/Ekinops/DrayTek/Fujitsu/Broadcom/Calix/Citrix images (migration 064) 2026-04-21 10:18:51 +02:00
Rene Fichtmueller
3cd6d44ea5 data: Sophos XGS 6500 image + Zyxel XS3800-28 URL fix (migration 063) 2026-04-21 10:16:51 +02:00
Rene Fichtmueller
906619e602 docs: update CHANGELOG_PENDING with migrations 055-060 2026-04-21 09:57:06 +02:00
Rene Fichtmueller
e2061c6c4d data: direct image injection for Nokia, F5, Delta Networks, Siemens, TP-Link 
Migration 051: TP-Link TL-SG3452XP + TL-SX3016F via static.tp-link.com CDN
Migration 052: Nokia 6/6 — 7220 IXR-D3L/H4 (docs.nokia.com graphics),
  7250 IXR-10 + 7750 SR-1 (tempestns.com), SR-14s (telecomcauliffe.com),
  SR-1e (docs hardwareBanner — no standalone public image available)
Migration 053: F5 BIG-IP i5800/i10800 (wtit.com), i15800 (blueally CDN)
Migration 054: Delta Networks 4/4 (hardwarenation.com + manualslib),
  Siemens SCALANCE 4/4 — X-200/X-300/X-500 via images.sw.cdn.siemens.com

All 14 URLs verified HTTP 200 with correct image content-type (2026-04-21).
CHANGELOG_PENDING.md updated for all 4 migrations.
 2026-04-21 08:42:14 +02:00
Rene Fichtmueller
466cda285b data: image coverage improvements — QCT, Allied Telesis + CHANGELOG update 
- sql/046: QCT QuantaMesh T3048-LY8 direct image injection
- sql/050: Allied Telesis 3/3 (AT-x530-28GSX, AT-x530L-52GPX, AT-x950-28XSQ)
  via alliedtelesis.com Drupal static files CDN (og:image, all 200 PNG)
- CHANGELOG: document filter pattern fixes + all 6 new vendor migrations
  (Moxa/UfiSpace/Brocade/NVIDIA/Allied Telesis — 19 new images total)
 2026-04-21 08:11:57 +02:00
Rene Fichtmueller
307a5ea38a chore: update changelog for 2026-04-20 switch image + compat features 2026-04-20 22:53:24 +02:00
Rene Fichtmueller
62d97a783c feat: add claude-code LLM provider + update dashboard to fo-blog-v5 
- client.ts: add claude-code provider routing BLOG_LLM_PROVIDER=claude-code
  to claude-bridge (flat-rate, no API billing via Claude Code subscription)
- checkHealth() now pings /health on claude-bridge for real availability check
- Default OLLAMA_LLM_MODEL changed from qwen2.5:14b to fo-blog-v5
- Dashboard: add claude-code card (EMPFOHLEN), rename fo-blog-v3 → fo-blog-v5
- loadBlogLLMStatus() handles all 3 providers: claude-code/anthropic/ollama
- Grid expanded from 3 to 4 columns to accommodate new card
- ecosystem.config.js + .env on Erik: OLLAMA_LLM_MODEL=fo-blog-v5 confirmed
 2026-04-18 20:45:14 +02:00
Rene Fichtmueller
74b83de6e9 docs: changelog — tunnel DNS fix, image backfill, OSFP coverage 2026-04-18 13:23:57 +02:00
Rene Fichtmueller
1da4abc488 fix: FS.com price extraction — DOM-based prices + shipping-context exclusion 
- All 247 FS.com prices were €79 (shipping threshold, not product prices)
- Root cause: 'Gratis Versand ab 79 € (ohne MwSt.)' banner matched first
- Fix 1: DOM price extraction in page.evaluate with bad-parent skip list
- Fix 2: bodyText qualified patterns skip matches near shipping keywords
- Fix 3: waitForSelector for price DOM element before evaluate
- Fix 4: Deleted 247 invalid €79 observations from DB

Also included from previous session:
- db.ts: set has_image=true on image writes (fix 632 desync rows)
- spec-updater.ts: DR/FR/LR/ER/ZR → SMF, SR → MMF fiber type inference
 2026-04-18 13:10:35 +02:00
Rene Fichtmueller
6a33a17bca chore: changelog — Crawlee queue wipe, ATGBICS fix, Optcore skip 2026-04-18 05:42:37 +02:00
Rene Fichtmueller
1378a9bee8 chore: changelog — 10Gtek scraper fix (sfpcables.com, 49 prices) 2026-04-18 05:32:33 +02:00
Rene Fichtmueller
2a6ec90ecd fix: fs-com Phase 1+2 crawler.run() ENOENT guard — Crawlee catches and re-throws the post-run _isTaskReadyFunction ENOENT internally, which rejected crawler.run() and aborted Phase 2 before it could start. Wrap both crawler.run() calls in try/catch to swallow ENOENT from request_queues paths; all processing is already complete at this point. 2026-04-18 03:52:49 +02:00
Rene Fichtmueller
b3eff15fc4 chore: changelog — daemon stability, ATGBICS Playwright, health monitor accuracy 2026-04-18 03:25:02 +02:00
Rene Fichtmueller
0e4e6ff6b2 chore: changelog — FS.com ENOENT fix, PID lock, health monitor tiered alerts 2026-04-18 02:55:18 +02:00
Rene Fichtmueller
696127366c chore: changelog — Playwright headless shell fix, withIsolatedStorage race fix, FS.com launchd fix 2026-04-18 02:35:55 +02:00
Rene Fichtmueller
35a02057f4 chore: sync-to-erik now includes scraper rebuild + daemon restart; changelog 2026-04-18 01:37:12 +02:00
Rene Fichtmueller
1c8dec52c9 feat: Price Comparison dashboard + Eoptolink OEM scraper 
- Add public /api/price-comparison API (summary, top-50, per-SKU detail)
  — no auth required, 3 Express routes, DISTINCT ON latest-price logic
- Add '💲 Price Comparison' dashboard tab: stat cards, form-factor
  breakdown, top-50 SKU table (clickable rows → SKU detail), per-vendor
  price + stock + spread% lookup panel
- Add Eoptolink OEM catalog scraper (93 product-solution pages,
  part-number regex EOLO-*/EOLQ-* etc., no prices, seeds transceivers
  table as manufacturer entries)
- Register scrape:catalog:eoptolink in scheduler: schedule every 4h
  (40 */4 * * *), lazy-import worker, added to known-jobs array
 2026-04-18 01:02:08 +02:00
Rene Fichtmueller
63b4a65c28 chore: changelog entry — MCP Server v0.2.0 (finder + switch-docs + Ollama LLM tools) 2026-04-18 00:29:42 +02:00
Rene Fichtmueller
b88a6e28cf feat: /api/hype-cycle/analysis endpoint — DB-backed Bass-fitted results from hype_cycle_analysis table 2026-04-18 00:11:08 +02:00
Rene Fichtmueller
75cea9fe90 feat: Mouser Electronics API scraper for OEM reference prices (Juniper/Cisco/Arista PIDs) 2026-04-18 00:04:35 +02:00
Rene Fichtmueller
5c5841d4d1 chore: prune 242 irrelevant vendors from DB (348→106), add changelog entry 2026-04-17 23:58:13 +02:00
Rene Fichtmueller
60736bd5df chore: changelog entries 2026-04-17 stock dashboard + Cisco TMG + Juniper HCT 2026-04-17 23:33:49 +02:00
Rene Fichtmueller
5393f73c17 feat: stock quality schema + QSFPTEK/NADDOD v2 scrapers with real-time stock counts 
- Migration 028 (retroactive): document warehouse columns added to stock_observations
- Migration 037: composite indexes for DISTINCT ON (transceiver_id, source_vendor_id) queries
- Migration 038: add stock_confidence (1/2/3), price_currency, price_includes_tax,
  stock_vendor_ts to stock_observations + TRUNCATE test-run data

db.ts: upsertStockObservation now accepts stockConfidence, priceCurrency,
priceIncludesTax, stockVendorTs; delta detection includes quantity_available

fs-com.ts: passes stockConfidence=3 + priceCurrency=EUR + priceIncludesTax=false

qsfptek.ts v2: Phase 1 API listing + Phase 2 detail-page stock extraction
- Parses 'X in real-time stock, DATE' from product detail pages
- Writes stock_observations with confidence=2 + stockVendorTs
- Up to 500 detail pages/run at 2s rate limit

naddod.ts v2: complete rewrite from WooCommerce to Astro sitemap-based
- Discovers products via /sitemaps/products.xml (600+ products)
- URL format: /products/XXXXX.html
- Extracts 'In Stock: X' exact counts from SSR HTML
- Writes both price + stock observations (confidence 1 or 2)
 2026-04-17 22:54:40 +02:00
Rene Fichtmueller
5b35b2b8be feat(scraper+api): warehouse stock data pipeline — FS.com v2, SmartOptics v2, Stock API 
Scraper changes:
- fs-com.ts v2: Playwright stealth patches + www.fs.com/de/ URL fix (de.fs.com DNS NXDOMAIN).
  Extracts DE-Lager, Global-Lager, Nachlieferung, units_sold, compatible_brands, price_net.
  Mac-side runner (run-fs-scraper-mac.sh) via SSH tunnel for residential IP access.
  Fast-fail connectivity check on datacenter IPs that are blocked by Cloudflare.
- smartoptics.ts v2: WooCommerce REST API fallback + 8 catalog categories + relative URL fix.
  Was finding only 8 products, now discovers 18+ with multi-category crawl.

DB layer:
- db.ts: add upsertStockObservation() — writes 10 new stock_observations columns
  (warehouse_de_qty, warehouse_global_qty, backorder_qty, units_sold, compatible_brands,
  price_net, product_url, delivery dates) with dedup check.

API:
- routes/stock.ts: GET /api/stock, /api/stock/summary, /api/stock/:id
  Warehouse breakdowns per transceiver/vendor with top-sellers and vendor summary.
- routes/review.ts: equivalence review queue (approve/reject/bulk-approve).
- index.ts: register /api/stock and /api/review routes.

Dashboard:
- index.html: 🏭 Stock tab with stat cards (DE-Lager, Global-Lager, Nachlieferung totals),
  top-sellers table, vendor breakdown, recently-restocked events, part-number lookup.

SQL migrations:
- 034: blog-review-tag, 035: price-observations is_anomalous, 036: transceiver-equivalences.
 2026-04-17 10:45:59 +02:00
Rene Fichtmueller
662cd1f90b fix(scraper): FiberMall URL schema + price parser + Flexoptix EUR comma bug 
FiberMall:
- Correct /store-XXXXX-name.htm category URLs (was /c/xxx/ → HTTP 404)
- Parser: split on new_proList_mainListLi, price from data-price on
  currency_price span — fix 0.00 false-match from SKU variant items
- Also scrape SKU brand variant links from .sku_item divs
- Result: 3,410 prices now in DB (was 0)

Flexoptix:
- Fix extractPrice regex for EUR thousand-separator format
  (2,921.60 EUR was parsed as 2 EUR)
- Add OSFP224 / 1.6T search queries (4 new, form factor was missing)
- Fix O.138HG2.C.05 stale price 3009.60→2921.60 EUR

Schema: competitor_verified + competitor_verified_at columns
added via ALTER TABLE (were referenced in code but missing in DB)

CHANGELOG: added 6 entries for 2026-04-12
 2026-04-12 04:26:35 +02:00
Rene Fichtmueller
8905382a49 chore: add changelog entries 2026-04-05 through 2026-04-09 2026-04-11 18:50:17 +02:00
Rene Fichtmueller
ef30da18d1 chore: update changelog for 2026-04-11 session 2026-04-11 13:57:31 +02:00
Rene Fichtmueller
4f631fc61e feat(blog): Reduction Engine v1.0 + LaTeX/connector hard fails 
- Replace STEP8b_REDUCTION with 5-pass Reduction Engine:
  Pass 1: Repetition Kill (one concept, one home)
  Pass 2: Tech Prune (LaTeX hard delete, SKU removal, formula prose replacement)
  Pass 3: Flow Rebuild (close gaps after cuts, no new content)
  Pass 4: Weight Correction (title/content alignment throughout)
  Pass 5: Humanization (rhythm variation, hedge removal, punch ending)
  Target: 700-1000 words (600-1300 range, warnings outside)

- System prompt + STEP9 QA: add hard fails for
  LaTeX formulas (\[...\], \frac{}, \text{} etc) — destroys blog flow
  DR4 connector error (DR4=MPO-12, not LC duplex; FR4=LC duplex)
  Title/content mismatch (title topic must be the spine, not just the intro)

- Gold Standard 5: market alert / pricing article template
  (correct title alignment, no LaTeX, DR4=MPO-12, ending on topic)

- WRONG PATTERNS extended with 4 new entries covering above failures

- blog.ts: step log messages updated to 11-14/14; word count
  output shows % reduction and range warning (>1300 or <600)
 2026-04-04 08:57:21 +02:00
Rene Fichtmueller
3431ccbebc chore: changelog — blog engine v5 + linkedin post 2026-04-04 2026-04-04 08:30:54 +02:00
Rene Fichtmueller
be9209ffbd chore: changelog — proxy network geo/uptime fixes 2026-04-04 2026-04-04 08:15:58 +02:00
Rene Fichtmueller
9074b6ede0 docs: update changelog 2026-04-03/04 — scraper fixes, blog engine v4, proxy network, pg-boss fix 2026-04-04 07:58:29 +02:00
Rene Fichtmueller
1026787318 feat: add proxy network, image backfill, and scraper improvements 
- Add TIP Proxy Network (packages/proxy-agent): SOCKS5 proxy agent
  for residential IP bypass of CloudFront WAF blocks
- Add /api/proxy/* routes: node registration, heartbeat, load balancing
- Add image extraction to Flexoptix catalog scraper (GraphQL small_image)
- Add image extraction to Optcore scraper (Playwright gallery img)
- Fix Fluxlight price scraping (BigCommerce HTML structure: data-product-price-without-tax)
- Add SmartOptics scraper (8 DWDM/coherent products, og:image extraction)
- Fix findOrCreateScrapedTransceiver to update image_url for existing records
- Add image backfill script (backfill-images.ts): 178 Flexoptix images added
- Fix DB connection pool: max 5, idleTimeoutMillis 10s (was unlimited, caused >100 connections)
- Add proxy.ts utility for scraper proxy rotation
 2026-04-03 21:13:03 +02:00
Rene Fichtmueller
b7613538bf fix: remove internal IPs, ports, tokens from changelog entries 2026-04-02 08:16:48 +02:00
Rene Fichtmueller
b313522a2a chore: changelog entry for auth feature 2026-04-02 07:32:40 +02:00