feat(scraper): add 100+ OEM seed scrapers + tip-llm-guided inference layer
New OEM transceiver seed scrapers (94 cron-scheduled, 24/7): - Media/Broadcast: Evertz, Grass Valley, Haivision, Viasat - Asian Optical: FiberHome, Oplink, Accelink, Hisense Broadband - Optical Mfrs: Lumentum, II-VI/Coherent, Source Photonics, O-Net, InnoLight, AOI, Sumitomo Electric, NeoPhotonics - Industrial: GE Grid, Schweitzer, Moxa Industrial, Cisco IE, Phoenix Contact, Beckhoff, Omron, ABB, Siemens, Schneider, Rockwell, Belden - Enterprise/DC: Arista, Pica8, Pluribus, DriveNets, Cisco (Meraki/Catalyst/Nexus/ASR) - Cloud: AWS, Azure, Google Cloud, Meta - Storage: NetApp, Pure Storage, HPE Storage, IBM Storage, Dell Storage, Hitachi Vantara - 5G/RAN: Samsung Networks, Nokia AirScale, Ericsson RAN, Mavenir - Security: Check Point, Barracuda, Fortinet, Palo Alto - Telecom Optical: ADVA, PacketLight, FiberHome, Accelink, Hisense API: tip-llm-guided inference layer (strict schema + repair-retry + safe fallback) - POST /api/tip-llm/infer|research-plan|extract|finding|health - Hard JSON schema enforcement, create_finding=false on empty evidence - Confidence gate (>= 0.4), validation with consistency check Build: added incremental=true to scraper tsconfig (OOM prevention) Scheduler: 87 → 94 registered workers
This commit is contained in:
parent
4479429b29
commit
d7144731e0
@ -34,6 +34,7 @@ import { priceComparisonRouter } from "./routes/price-comparison";
|
|||||||
import { selflearningRouter } from "./routes/selflearning";
|
import { selflearningRouter } from "./routes/selflearning";
|
||||||
import { internalDemandRouter } from "./routes/internal-demand";
|
import { internalDemandRouter } from "./routes/internal-demand";
|
||||||
import { formFactorsRouter } from "./routes/form-factors";
|
import { formFactorsRouter } from "./routes/form-factors";
|
||||||
|
import { tipLlmRouter } from "./routes/tip-llm";
|
||||||
|
|
||||||
const app = express();
|
const app = express();
|
||||||
|
|
||||||
@ -99,6 +100,8 @@ app.use("/api/selflearning", selflearningRouter);
|
|||||||
app.use("/api/form-factors", formFactorsRouter);
|
app.use("/api/form-factors", formFactorsRouter);
|
||||||
// Internal-only — restricted to localhost / LAN by the router itself
|
// Internal-only — restricted to localhost / LAN by the router itself
|
||||||
app.use("/api/internal/demand", internalDemandRouter);
|
app.use("/api/internal/demand", internalDemandRouter);
|
||||||
|
// tip-llm-v1 guided inference
|
||||||
|
app.use("/api/tip-llm", tipLlmRouter);
|
||||||
|
|
||||||
// Dashboard (static HTML)
|
// Dashboard (static HTML)
|
||||||
app.use("/dashboard", express.static(join(__dirname, "..", "..", "dashboard")));
|
app.use("/dashboard", express.static(join(__dirname, "..", "..", "dashboard")));
|
||||||
|
|||||||
@ -5,14 +5,14 @@
|
|||||||
* Provider selection:
|
* Provider selection:
|
||||||
* BLOG_LLM_PROVIDER=claude-code → Claude via claude-bridge (flat-rate, recommended)
|
* BLOG_LLM_PROVIDER=claude-code → Claude via claude-bridge (flat-rate, recommended)
|
||||||
* BLOG_LLM_PROVIDER=anthropic → Claude Sonnet/Haiku via Anthropic API
|
* BLOG_LLM_PROVIDER=anthropic → Claude Sonnet/Haiku via Anthropic API
|
||||||
* BLOG_LLM_PROVIDER=ollama → fine-tuned fo-blog-v5 on local Ollama (default)
|
* BLOG_LLM_PROVIDER=ollama → local adapter bridge / Ollama-compatible endpoint (default)
|
||||||
*
|
*
|
||||||
* Claude-code is preferred: uses Claude Code subscription (flat-rate), no API costs.
|
* Claude-code is preferred: uses Claude Code subscription (flat-rate), no API costs.
|
||||||
* Ollama fo-blog-v5 is the fallback for offline/local usage.
|
* The default local blog model is the latest RunPod-trained FO_BlogLLM adapter.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
const OLLAMA_URL = process.env.OLLAMA_URL || "http://localhost:11434";
|
const OLLAMA_URL = process.env.OLLAMA_URL || "http://localhost:11434";
|
||||||
const LLM_MODEL = process.env.OLLAMA_LLM_MODEL || "fo-blog-v5";
|
const LLM_MODEL = process.env.OLLAMA_LLM_MODEL || "fo-blog-v7";
|
||||||
|
|
||||||
const ANTHROPIC_API_KEY = process.env.ANTHROPIC_API_KEY || "";
|
const ANTHROPIC_API_KEY = process.env.ANTHROPIC_API_KEY || "";
|
||||||
const ANTHROPIC_MODEL = process.env.ANTHROPIC_MODEL || "claude-sonnet-4-20250514";
|
const ANTHROPIC_MODEL = process.env.ANTHROPIC_MODEL || "claude-sonnet-4-20250514";
|
||||||
|
|||||||
@ -27,7 +27,7 @@
|
|||||||
* - Core principle: "Your content gets better the more you delete"
|
* - Core principle: "Your content gets better the more you delete"
|
||||||
*
|
*
|
||||||
* Dedicated FO_Blog_LLM:
|
* Dedicated FO_Blog_LLM:
|
||||||
* - Model: qwen2.5:14b on .213 (or override via FO_BLOG_MODEL env)
|
* - Model: fo-blog-v7 via local adapter bridge (or override via FO_BLOG_MODEL env)
|
||||||
* - System prompt loaded with accumulated feedback
|
* - System prompt loaded with accumulated feedback
|
||||||
* - Feedback loop: every blog gets rated, feedback trains next generation
|
* - Feedback loop: every blog gets rated, feedback trains next generation
|
||||||
*/
|
*/
|
||||||
|
|||||||
421
packages/api/src/llm/tip-llm-guided.ts
Normal file
421
packages/api/src/llm/tip-llm-guided.ts
Normal file
@ -0,0 +1,421 @@
|
|||||||
|
/**
|
||||||
|
* tip-llm-guided.ts — Structured Inference Engine for tip-llm-v1
|
||||||
|
*
|
||||||
|
* DESIGN PRINCIPLE: tip-llm-v1 must NEVER think freely.
|
||||||
|
* Every call goes through:
|
||||||
|
* 1. Hard system prompt that locks the output format
|
||||||
|
* 2. Schema validation of the response
|
||||||
|
* 3. Repair retry (max 2x) with diff showing what was wrong
|
||||||
|
* 4. Fallback to safe default (create_finding=false) on persistent failure
|
||||||
|
*
|
||||||
|
* Input: source text/HTML + context (vendor, product, URL)
|
||||||
|
* Output: validated TipLlmOutput JSON — always, never throws
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { generate } from "./client";
|
||||||
|
|
||||||
|
// ─── Model config ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
const TIP_LLM_MODEL = process.env.TIP_LLM_MODEL || "tip-llm-v1";
|
||||||
|
const TIP_OLLAMA_URL = process.env.TIP_OLLAMA_URL || process.env.OLLAMA_URL || "http://localhost:11434";
|
||||||
|
const TIP_LLM_TIMEOUT = parseInt(process.env.TIP_LLM_TIMEOUT_MS || "30000");
|
||||||
|
const TIP_LLM_MAX_RETRY = 2;
|
||||||
|
|
||||||
|
// ─── Schema ───────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
export type TipLlmTask =
|
||||||
|
| "research_plan" // produce a crawl/search plan for a vendor/product
|
||||||
|
| "extract" // extract structured facts from provided HTML/text
|
||||||
|
| "finding"; // produce a finding ticket from pre-gathered evidence
|
||||||
|
|
||||||
|
export interface TipLlmInput {
|
||||||
|
task: TipLlmTask;
|
||||||
|
/** Raw text or HTML snippet to analyse (max ~4000 chars recommended) */
|
||||||
|
text?: string;
|
||||||
|
/** Source URL the text was fetched from */
|
||||||
|
url?: string;
|
||||||
|
/** Vendor name context */
|
||||||
|
vendor?: string;
|
||||||
|
/** Product name / PID context */
|
||||||
|
product?: string;
|
||||||
|
/** Extra context string for the model */
|
||||||
|
context?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface EvidenceItem {
|
||||||
|
/** Verbatim quote or data point found in the source */
|
||||||
|
quote: string;
|
||||||
|
/** Source URL or description (must be from the provided input, not invented) */
|
||||||
|
source: string;
|
||||||
|
/** How confident this item is: 0.0–1.0 */
|
||||||
|
confidence: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface TipLlmOutput {
|
||||||
|
/** Task that was executed */
|
||||||
|
task: TipLlmTask;
|
||||||
|
/** For research_plan: search queries to execute */
|
||||||
|
search_patterns: string[];
|
||||||
|
/** For extract: field names + XPath/CSS selectors or regex patterns */
|
||||||
|
extraction_fields: Array<{
|
||||||
|
field: string;
|
||||||
|
selector_or_pattern: string;
|
||||||
|
example?: string;
|
||||||
|
}>;
|
||||||
|
/** Rules for what counts as relevant vs noise */
|
||||||
|
relevance_rules: string[];
|
||||||
|
/** Concrete evidence items from the provided source text — EMPTY if nothing found */
|
||||||
|
evidence: EvidenceItem[];
|
||||||
|
/** Aggregate confidence 0.0–1.0. Must be 0 if evidence is empty. */
|
||||||
|
confidence: number;
|
||||||
|
/** True ONLY if evidence is non-empty AND confidence >= 0.4 */
|
||||||
|
create_finding: boolean;
|
||||||
|
/** Human-readable summary — one sentence max */
|
||||||
|
summary: string;
|
||||||
|
/** Raw sources referenced (URLs or "provided_text") */
|
||||||
|
sources: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Validation ───────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
interface ValidationError {
|
||||||
|
field: string;
|
||||||
|
issue: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
function validateOutput(raw: unknown): { ok: true; data: TipLlmOutput } | { ok: false; errors: ValidationError[] } {
|
||||||
|
const errors: ValidationError[] = [];
|
||||||
|
|
||||||
|
if (typeof raw !== "object" || raw === null) {
|
||||||
|
return { ok: false, errors: [{ field: "root", issue: "Response is not a JSON object" }] };
|
||||||
|
}
|
||||||
|
|
||||||
|
const obj = raw as Record<string, unknown>;
|
||||||
|
|
||||||
|
// task
|
||||||
|
const validTasks: TipLlmTask[] = ["research_plan", "extract", "finding"];
|
||||||
|
if (!validTasks.includes(obj.task as TipLlmTask)) {
|
||||||
|
errors.push({ field: "task", issue: `Must be one of: ${validTasks.join(", ")}` });
|
||||||
|
}
|
||||||
|
|
||||||
|
// search_patterns
|
||||||
|
if (!Array.isArray(obj.search_patterns)) {
|
||||||
|
errors.push({ field: "search_patterns", issue: "Must be an array of strings" });
|
||||||
|
} else if (obj.search_patterns.some((x) => typeof x !== "string")) {
|
||||||
|
errors.push({ field: "search_patterns", issue: "All items must be strings" });
|
||||||
|
}
|
||||||
|
|
||||||
|
// extraction_fields
|
||||||
|
if (!Array.isArray(obj.extraction_fields)) {
|
||||||
|
errors.push({ field: "extraction_fields", issue: "Must be an array" });
|
||||||
|
} else {
|
||||||
|
for (const [i, ef] of obj.extraction_fields.entries()) {
|
||||||
|
if (typeof ef !== "object" || ef === null) {
|
||||||
|
errors.push({ field: `extraction_fields[${i}]`, issue: "Must be an object" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const e = ef as Record<string, unknown>;
|
||||||
|
if (typeof e.field !== "string") errors.push({ field: `extraction_fields[${i}].field`, issue: "Must be a string" });
|
||||||
|
if (typeof e.selector_or_pattern !== "string") errors.push({ field: `extraction_fields[${i}].selector_or_pattern`, issue: "Must be a string" });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// relevance_rules
|
||||||
|
if (!Array.isArray(obj.relevance_rules)) {
|
||||||
|
errors.push({ field: "relevance_rules", issue: "Must be an array of strings" });
|
||||||
|
}
|
||||||
|
|
||||||
|
// evidence
|
||||||
|
if (!Array.isArray(obj.evidence)) {
|
||||||
|
errors.push({ field: "evidence", issue: "Must be an array" });
|
||||||
|
} else {
|
||||||
|
for (const [i, ev] of obj.evidence.entries()) {
|
||||||
|
if (typeof ev !== "object" || ev === null) {
|
||||||
|
errors.push({ field: `evidence[${i}]`, issue: "Must be an object" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const e = ev as Record<string, unknown>;
|
||||||
|
if (typeof e.quote !== "string" || e.quote.trim().length === 0) {
|
||||||
|
errors.push({ field: `evidence[${i}].quote`, issue: "Must be a non-empty string" });
|
||||||
|
}
|
||||||
|
if (typeof e.source !== "string" || e.source.trim().length === 0) {
|
||||||
|
errors.push({ field: `evidence[${i}].source`, issue: "Must be a non-empty string" });
|
||||||
|
}
|
||||||
|
if (typeof e.confidence !== "number" || e.confidence < 0 || e.confidence > 1) {
|
||||||
|
errors.push({ field: `evidence[${i}].confidence`, issue: "Must be a number between 0 and 1" });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// confidence
|
||||||
|
if (typeof obj.confidence !== "number" || obj.confidence < 0 || obj.confidence > 1) {
|
||||||
|
errors.push({ field: "confidence", issue: "Must be a number between 0.0 and 1.0" });
|
||||||
|
}
|
||||||
|
|
||||||
|
// create_finding consistency check
|
||||||
|
if (typeof obj.create_finding !== "boolean") {
|
||||||
|
errors.push({ field: "create_finding", issue: "Must be a boolean" });
|
||||||
|
} else {
|
||||||
|
const evidence = Array.isArray(obj.evidence) ? obj.evidence : [];
|
||||||
|
const confidence = typeof obj.confidence === "number" ? obj.confidence : 0;
|
||||||
|
if (obj.create_finding === true && (evidence.length === 0 || confidence < 0.4)) {
|
||||||
|
errors.push({
|
||||||
|
field: "create_finding",
|
||||||
|
issue: "Cannot be true when evidence is empty or confidence < 0.4 — set to false",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// summary
|
||||||
|
if (typeof obj.summary !== "string") {
|
||||||
|
errors.push({ field: "summary", issue: "Must be a string" });
|
||||||
|
}
|
||||||
|
|
||||||
|
// sources
|
||||||
|
if (!Array.isArray(obj.sources)) {
|
||||||
|
errors.push({ field: "sources", issue: "Must be an array of strings" });
|
||||||
|
}
|
||||||
|
|
||||||
|
if (errors.length > 0) return { ok: false, errors };
|
||||||
|
|
||||||
|
return { ok: true, data: obj as unknown as TipLlmOutput };
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Safe fallback ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function safeDefault(input: TipLlmInput, reason: string): TipLlmOutput {
|
||||||
|
console.warn(`[tip-llm] Returning safe default — reason: ${reason}`);
|
||||||
|
return {
|
||||||
|
task: input.task,
|
||||||
|
search_patterns: [],
|
||||||
|
extraction_fields: [],
|
||||||
|
relevance_rules: [],
|
||||||
|
evidence: [],
|
||||||
|
confidence: 0,
|
||||||
|
create_finding: false,
|
||||||
|
summary: `No reliable output could be produced: ${reason}`,
|
||||||
|
sources: input.url ? [input.url] : ["provided_text"],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Prompts ──────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
const SYSTEM_PROMPT = `You are a structured data extraction engine for the Transceiver Intelligence Platform (TIP).
|
||||||
|
|
||||||
|
STRICT RULES — violating ANY of these results in your output being discarded:
|
||||||
|
1. You MUST respond with ONLY valid JSON. No markdown. No explanation. No code blocks.
|
||||||
|
2. The JSON MUST contain ALL of these fields: task, search_patterns, extraction_fields, relevance_rules, evidence, confidence, create_finding, summary, sources.
|
||||||
|
3. evidence MUST only contain information that is EXPLICITLY present in the provided source text. If the text does not contain the information, evidence MUST be an empty array [].
|
||||||
|
4. DO NOT invent, hallucinate, or assume any vendor details, lead times, pricing, specs, or availability without direct textual evidence.
|
||||||
|
5. create_finding MUST be false if evidence is empty or confidence < 0.4.
|
||||||
|
6. confidence MUST be 0.0 if evidence is empty.
|
||||||
|
7. Every evidence item MUST have a non-empty quote (verbatim from source), a source, and a confidence value between 0.0 and 1.0.
|
||||||
|
8. summary MUST be one sentence, maximum 150 characters.
|
||||||
|
|
||||||
|
JSON SCHEMA:
|
||||||
|
{
|
||||||
|
"task": "research_plan" | "extract" | "finding",
|
||||||
|
"search_patterns": ["string"],
|
||||||
|
"extraction_fields": [{"field": "string", "selector_or_pattern": "string", "example": "string (optional)"}],
|
||||||
|
"relevance_rules": ["string"],
|
||||||
|
"evidence": [{"quote": "string", "source": "string", "confidence": 0.0-1.0}],
|
||||||
|
"confidence": 0.0-1.0,
|
||||||
|
"create_finding": true | false,
|
||||||
|
"summary": "string (max 150 chars)",
|
||||||
|
"sources": ["string"]
|
||||||
|
}`;
|
||||||
|
|
||||||
|
function buildUserPrompt(input: TipLlmInput): string {
|
||||||
|
const parts: string[] = [];
|
||||||
|
|
||||||
|
parts.push(`TASK: ${input.task}`);
|
||||||
|
if (input.vendor) parts.push(`VENDOR: ${input.vendor}`);
|
||||||
|
if (input.product) parts.push(`PRODUCT: ${input.product}`);
|
||||||
|
if (input.url) parts.push(`URL: ${input.url}`);
|
||||||
|
if (input.context) parts.push(`CONTEXT: ${input.context}`);
|
||||||
|
|
||||||
|
if (input.text) {
|
||||||
|
// Truncate to 4000 chars to stay within tip-llm-v1 context window
|
||||||
|
const truncated = input.text.length > 4000
|
||||||
|
? input.text.slice(0, 4000) + "\n[... truncated]"
|
||||||
|
: input.text;
|
||||||
|
parts.push(`\nSOURCE TEXT:\n${truncated}`);
|
||||||
|
} else {
|
||||||
|
parts.push("\nSOURCE TEXT: (none provided — generate search plan only)");
|
||||||
|
}
|
||||||
|
|
||||||
|
parts.push("\nRespond with ONLY the JSON object. No other text.");
|
||||||
|
return parts.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildRepairPrompt(input: TipLlmInput, previousOutput: string, errors: ValidationError[]): string {
|
||||||
|
const errorList = errors.map((e) => `- ${e.field}: ${e.issue}`).join("\n");
|
||||||
|
return `Your previous response had schema errors. Fix them and return ONLY corrected JSON.
|
||||||
|
|
||||||
|
ERRORS:
|
||||||
|
${errorList}
|
||||||
|
|
||||||
|
PREVIOUS RESPONSE (first 500 chars):
|
||||||
|
${previousOutput.slice(0, 500)}
|
||||||
|
|
||||||
|
${buildUserPrompt(input)}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Raw Ollama call (bypasses blog LLM routing, hits tip-llm-v1 directly) ────
|
||||||
|
|
||||||
|
async function callTipLlm(
|
||||||
|
systemPrompt: string,
|
||||||
|
userPrompt: string,
|
||||||
|
attempt: number,
|
||||||
|
): Promise<string> {
|
||||||
|
// Override model env vars temporarily by calling Ollama API directly
|
||||||
|
// This ensures tip-llm-v1 is used, not fo-blog-v7
|
||||||
|
const resp = await fetch(`${TIP_OLLAMA_URL}/api/generate`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({
|
||||||
|
model: TIP_LLM_MODEL,
|
||||||
|
prompt: userPrompt,
|
||||||
|
system: systemPrompt,
|
||||||
|
stream: false,
|
||||||
|
options: {
|
||||||
|
temperature: attempt === 0 ? 0.1 : 0.05, // lower temp on retry
|
||||||
|
num_predict: 1500,
|
||||||
|
stop: ["\n\n\n"], // prevent runaway output
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
signal: AbortSignal.timeout(TIP_LLM_TIMEOUT),
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!resp.ok) {
|
||||||
|
const errText = await resp.text();
|
||||||
|
throw new Error(`tip-llm-v1 call failed: ${resp.status} ${errText.slice(0, 200)}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const data = await resp.json() as { response: string; model: string; total_duration: number };
|
||||||
|
const durationMs = Math.round(data.total_duration / 1_000_000);
|
||||||
|
console.log(`[tip-llm] ${data.model} attempt=${attempt} duration=${durationMs}ms chars=${data.response.length}`);
|
||||||
|
return data.response;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── JSON extraction ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
function extractJson(raw: string): unknown {
|
||||||
|
// Strip markdown code blocks if model wrapped output despite instructions
|
||||||
|
let text = raw.trim();
|
||||||
|
const codeBlock = text.match(/```(?:json)?\s*([\s\S]*?)```/);
|
||||||
|
if (codeBlock) text = codeBlock[1].trim();
|
||||||
|
|
||||||
|
// Find the first complete JSON object
|
||||||
|
const start = text.indexOf("{");
|
||||||
|
const end = text.lastIndexOf("}");
|
||||||
|
if (start === -1 || end === -1) throw new Error("No JSON object found in response");
|
||||||
|
|
||||||
|
return JSON.parse(text.slice(start, end + 1));
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Main entry point ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Run tip-llm-v1 with strict guided inference.
|
||||||
|
* Never throws — returns a safe default on persistent failure.
|
||||||
|
*/
|
||||||
|
export async function tipLlmInfer(input: TipLlmInput): Promise<TipLlmOutput> {
|
||||||
|
const startTime = Date.now();
|
||||||
|
let lastRaw = "";
|
||||||
|
let lastErrors: ValidationError[] = [];
|
||||||
|
|
||||||
|
for (let attempt = 0; attempt <= TIP_LLM_MAX_RETRY; attempt++) {
|
||||||
|
try {
|
||||||
|
const userPrompt = attempt === 0
|
||||||
|
? buildUserPrompt(input)
|
||||||
|
: buildRepairPrompt(input, lastRaw, lastErrors);
|
||||||
|
|
||||||
|
lastRaw = await callTipLlm(SYSTEM_PROMPT, userPrompt, attempt);
|
||||||
|
|
||||||
|
let parsed: unknown;
|
||||||
|
try {
|
||||||
|
parsed = extractJson(lastRaw);
|
||||||
|
} catch (parseErr) {
|
||||||
|
lastErrors = [{ field: "root", issue: `JSON parse error: ${(parseErr as Error).message}` }];
|
||||||
|
console.warn(`[tip-llm] attempt=${attempt} parse failed: ${(parseErr as Error).message}`);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = validateOutput(parsed);
|
||||||
|
if (result.ok) {
|
||||||
|
const totalMs = Date.now() - startTime;
|
||||||
|
console.log(`[tip-llm] SUCCESS attempt=${attempt} create_finding=${result.data.create_finding} evidence=${result.data.evidence.length} total=${totalMs}ms`);
|
||||||
|
return result.data;
|
||||||
|
}
|
||||||
|
|
||||||
|
lastErrors = result.errors;
|
||||||
|
console.warn(`[tip-llm] attempt=${attempt} validation failed (${result.errors.length} errors):`, result.errors);
|
||||||
|
|
||||||
|
} catch (err) {
|
||||||
|
const msg = (err as Error).message;
|
||||||
|
console.error(`[tip-llm] attempt=${attempt} call error: ${msg}`);
|
||||||
|
lastErrors = [{ field: "root", issue: msg }];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// All retries exhausted — return safe default
|
||||||
|
const errorSummary = lastErrors.map((e) => `${e.field}: ${e.issue}`).slice(0, 3).join("; ");
|
||||||
|
return safeDefault(input, `schema validation failed after ${TIP_LLM_MAX_RETRY + 1} attempts: ${errorSummary}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ─── Convenience wrappers ─────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
/** Generate a search/crawl plan for a vendor+product without any source text */
|
||||||
|
export async function tipLlmResearchPlan(
|
||||||
|
vendor: string,
|
||||||
|
product: string,
|
||||||
|
context?: string,
|
||||||
|
): Promise<TipLlmOutput> {
|
||||||
|
return tipLlmInfer({ task: "research_plan", vendor, product, context });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Extract structured facts from scraped HTML/text */
|
||||||
|
export async function tipLlmExtract(
|
||||||
|
text: string,
|
||||||
|
url: string,
|
||||||
|
vendor?: string,
|
||||||
|
product?: string,
|
||||||
|
): Promise<TipLlmOutput> {
|
||||||
|
return tipLlmInfer({ task: "extract", text, url, vendor, product });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Produce a finding from pre-gathered evidence text */
|
||||||
|
export async function tipLlmFinding(
|
||||||
|
text: string,
|
||||||
|
vendor: string,
|
||||||
|
product: string,
|
||||||
|
url?: string,
|
||||||
|
): Promise<TipLlmOutput> {
|
||||||
|
return tipLlmInfer({ task: "finding", text, url, vendor, product });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Health check — verifies tip-llm-v1 is loaded in Ollama */
|
||||||
|
export async function checkTipLlmHealth(): Promise<{
|
||||||
|
ok: boolean;
|
||||||
|
model: string;
|
||||||
|
error?: string;
|
||||||
|
}> {
|
||||||
|
try {
|
||||||
|
const resp = await fetch(`${TIP_OLLAMA_URL}/api/tags`, {
|
||||||
|
signal: AbortSignal.timeout(5000),
|
||||||
|
});
|
||||||
|
if (!resp.ok) {
|
||||||
|
return { ok: false, model: TIP_LLM_MODEL, error: `Ollama HTTP ${resp.status}` };
|
||||||
|
}
|
||||||
|
const data = await resp.json() as { models: Array<{ name: string }> };
|
||||||
|
const hasModel = data.models.some((m) => m.name.startsWith(TIP_LLM_MODEL.split(":")[0]));
|
||||||
|
return {
|
||||||
|
ok: hasModel,
|
||||||
|
model: TIP_LLM_MODEL,
|
||||||
|
error: hasModel ? undefined : `Model "${TIP_LLM_MODEL}" not found in Ollama`,
|
||||||
|
};
|
||||||
|
} catch (err) {
|
||||||
|
return { ok: false, model: TIP_LLM_MODEL, error: (err as Error).message };
|
||||||
|
}
|
||||||
|
}
|
||||||
130
packages/api/src/routes/tip-llm.ts
Normal file
130
packages/api/src/routes/tip-llm.ts
Normal file
@ -0,0 +1,130 @@
|
|||||||
|
/**
|
||||||
|
* /api/tip-llm — Guided Inference API for tip-llm-v1
|
||||||
|
*
|
||||||
|
* Endpoints:
|
||||||
|
* POST /api/tip-llm/infer — general inference (any task)
|
||||||
|
* POST /api/tip-llm/research-plan — generate search/crawl plan
|
||||||
|
* POST /api/tip-llm/extract — extract facts from HTML/text
|
||||||
|
* POST /api/tip-llm/finding — produce a finding from evidence
|
||||||
|
* GET /api/tip-llm/health — model health check
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { Router, type Request, type Response } from "express";
|
||||||
|
import {
|
||||||
|
tipLlmInfer,
|
||||||
|
tipLlmResearchPlan,
|
||||||
|
tipLlmExtract,
|
||||||
|
tipLlmFinding,
|
||||||
|
checkTipLlmHealth,
|
||||||
|
type TipLlmInput,
|
||||||
|
type TipLlmTask,
|
||||||
|
} from "../llm/tip-llm-guided";
|
||||||
|
|
||||||
|
export const tipLlmRouter = Router();
|
||||||
|
|
||||||
|
const VALID_TASKS: TipLlmTask[] = ["research_plan", "extract", "finding"];
|
||||||
|
|
||||||
|
// ── Health ─────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
tipLlmRouter.get("/health", async (_req: Request, res: Response) => {
|
||||||
|
const result = await checkTipLlmHealth();
|
||||||
|
res.status(result.ok ? 200 : 503).json(result);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── General infer ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
tipLlmRouter.post("/infer", async (req: Request, res: Response) => {
|
||||||
|
const { task, text, url, vendor, product, context } = req.body as {
|
||||||
|
task?: string;
|
||||||
|
text?: string;
|
||||||
|
url?: string;
|
||||||
|
vendor?: string;
|
||||||
|
product?: string;
|
||||||
|
context?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
if (!task || !VALID_TASKS.includes(task as TipLlmTask)) {
|
||||||
|
res.status(400).json({ error: `task is required and must be one of: ${VALID_TASKS.join(", ")}` });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (text && text.length > 8000) {
|
||||||
|
res.status(400).json({ error: "text must be <= 8000 characters" });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const input: TipLlmInput = {
|
||||||
|
task: task as TipLlmTask,
|
||||||
|
text,
|
||||||
|
url,
|
||||||
|
vendor,
|
||||||
|
product,
|
||||||
|
context,
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = await tipLlmInfer(input);
|
||||||
|
res.json({ success: true, data: result });
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Research plan ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
tipLlmRouter.post("/research-plan", async (req: Request, res: Response) => {
|
||||||
|
const { vendor, product, context } = req.body as {
|
||||||
|
vendor?: string;
|
||||||
|
product?: string;
|
||||||
|
context?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
if (!vendor || !product) {
|
||||||
|
res.status(400).json({ error: "vendor and product are required" });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = await tipLlmResearchPlan(vendor, product, context);
|
||||||
|
res.json({ success: true, data: result });
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Extract ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
tipLlmRouter.post("/extract", async (req: Request, res: Response) => {
|
||||||
|
const { text, url, vendor, product } = req.body as {
|
||||||
|
text?: string;
|
||||||
|
url?: string;
|
||||||
|
vendor?: string;
|
||||||
|
product?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
if (!text || !url) {
|
||||||
|
res.status(400).json({ error: "text and url are required" });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (text.length > 8000) {
|
||||||
|
res.status(400).json({ error: "text must be <= 8000 characters" });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = await tipLlmExtract(text, url, vendor, product);
|
||||||
|
res.json({ success: true, data: result });
|
||||||
|
});
|
||||||
|
|
||||||
|
// ── Finding ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
tipLlmRouter.post("/finding", async (req: Request, res: Response) => {
|
||||||
|
const { text, vendor, product, url } = req.body as {
|
||||||
|
text?: string;
|
||||||
|
vendor?: string;
|
||||||
|
product?: string;
|
||||||
|
url?: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
if (!text || !vendor || !product) {
|
||||||
|
res.status(400).json({ error: "text, vendor and product are required" });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (text.length > 8000) {
|
||||||
|
res.status(400).json({ error: "text must be <= 8000 characters" });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const result = await tipLlmFinding(text, vendor, product, url);
|
||||||
|
res.json({ success: true, data: result });
|
||||||
|
});
|
||||||
File diff suppressed because it is too large
Load Diff
118
packages/scraper/src/scrapers/abb-oem.ts
Normal file
118
packages/scraper/src/scrapers/abb-oem.ts
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
/**
|
||||||
|
* ABB OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for ABB industrial SFP/SFP+/SFP28/QSFP+ modules
|
||||||
|
* used in ABB industrial automation, power grid networking, and substation
|
||||||
|
* communication hardware including the FOX, MicroSCADA, and 800xA series.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - ABB Network Management SFP datasheets (new.abb.com/network-management)
|
||||||
|
* - ABB Industrial Automation product catalog
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/abb-oem.ts
|
||||||
|
* Cron: daily at 16:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AbbPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ABB_PIDS: AbbPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ABB-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "ABB industrial SFP 1G SX power grid/substation" },
|
||||||
|
{ pid: "ABB-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "ABB industrial SFP 1G LX power grid/substation" },
|
||||||
|
{ pid: "ABB-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "ABB industrial SFP 1G ZX 80km" },
|
||||||
|
{ pid: "ABB-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "ABB industrial SFP 1G copper RJ45" },
|
||||||
|
{ pid: "ABB-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "ABB SFP GE-T copper industrial automation" },
|
||||||
|
|
||||||
|
// ── 1G SFP BiDi ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ABB-SFP-1G-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1310", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", notes: "ABB SFP 1G BiDi TX1310 single fiber" },
|
||||||
|
{ pid: "ABB-SFP-1G-BIDI-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1550", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310nm", notes: "ABB SFP 1G BiDi TX1550 single fiber" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ABB-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "ABB industrial SFP+ 10G SR" },
|
||||||
|
{ pid: "ABB-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "ABB industrial SFP+ 10G LR" },
|
||||||
|
{ pid: "ABB-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "ABB industrial SFP+ 10G ER" },
|
||||||
|
|
||||||
|
// ── 1G CWDM SFP (factory/substation rings) ──────────────────────────────
|
||||||
|
{ pid: "ABB-SFP-CWDM-1470", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1470", fiberType: "SMF", connector: "LC", wavelengths: "1470nm", notes: "ABB SFP CWDM 1470nm 40km substation ring" },
|
||||||
|
{ pid: "ABB-SFP-CWDM-1530", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1530", fiberType: "SMF", connector: "LC", wavelengths: "1530nm", notes: "ABB SFP CWDM 1530nm 40km substation ring" },
|
||||||
|
{ pid: "ABB-SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", notes: "ABB SFP CWDM 1550nm 40km substation ring" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ABB-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "ABB QSFP+ 40G LR4 industrial automation" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ABB-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "ABB SFP28 25G SR industrial" },
|
||||||
|
{ pid: "ABB-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "ABB SFP28 25G LR industrial" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAbbOem(): Promise<void> {
|
||||||
|
console.log("=== ABB OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"ABB",
|
||||||
|
"oem",
|
||||||
|
"https://new.abb.com/network-management",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ABB_PIDS) {
|
||||||
|
const slug = `abb-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== ABB OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ABB_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAbbOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
131
packages/scraper/src/scrapers/accelink-oem.ts
Normal file
131
packages/scraper/src/scrapers/accelink-oem.ts
Normal file
@ -0,0 +1,131 @@
|
|||||||
|
/**
|
||||||
|
* Accelink Technologies OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Accelink-branded transceiver PIDs covering data-center switching,
|
||||||
|
* metro coherent, and high-speed 400G ZR/ZR+ optics from Accelink
|
||||||
|
* Technologies Co., Ltd. (Wuhan, China — a CETC subsidiary).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Accelink Optical Transceiver Product Catalog (accelink.com)
|
||||||
|
* - Accelink 400G ZR / ZR+ Data Sheet
|
||||||
|
* - Accelink CFP2-DCO Coherent Module Specification
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/accelink-oem.ts
|
||||||
|
* Cron: daily at 19:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AccelinkPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that belong to TELECOM category (CFP2-DCO, ZR/ZR+)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"ACL-SFP-10G-ZR",
|
||||||
|
"ACL-CFP2-DCO-100G",
|
||||||
|
"ACL-QSFP-DD-ZR-400G",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const ACCELINK_PIDS: AccelinkPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ACL-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Accelink 1G SFP SX 550m MMF" },
|
||||||
|
{ pid: "ACL-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Accelink 1G SFP LX 10km SMF" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ACL-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Accelink 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "ACL-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Accelink 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "ACL-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Accelink 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "ACL-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Accelink 10G SFP+ ZR 80km SMF" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ACL-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Accelink 25G SFP28 SR 100m MMF" },
|
||||||
|
{ pid: "ACL-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Accelink 25G SFP28 LR 10km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ACL-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Accelink 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "ACL-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "Accelink 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ACL-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Accelink 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "ACL-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Accelink 100G QSFP28 LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ACL-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Accelink 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "ACL-QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "Accelink 400G QSFP-DD SR8 100m MMF" },
|
||||||
|
|
||||||
|
// ── CFP2-DCO (Telecom) ───────────────────────────────────────────────────
|
||||||
|
{ pid: "ACL-CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1000000,reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF CFP2-DCO", notes: "Accelink 100G CFP2 DCO coherent C-band, 1000km span" },
|
||||||
|
|
||||||
|
// ── 400G ZR (Telecom) ────────────────────────────────────────────────────
|
||||||
|
{ pid: "ACL-QSFP-DD-ZR-400G", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1000000,reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF 400ZR", notes: "Accelink 400G QSFP-DD ZR coherent C-band, 1000km span" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAccelinkOem(): Promise<void> {
|
||||||
|
console.log("=== Accelink Technologies OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Accelink Technologies",
|
||||||
|
"oem",
|
||||||
|
"https://www.accelink.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ACCELINK_PIDS) {
|
||||||
|
const slug = `accelink-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Accelink Technologies OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ACCELINK_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAccelinkOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
131
packages/scraper/src/scrapers/adtran-oem.ts
Normal file
131
packages/scraper/src/scrapers/adtran-oem.ts
Normal file
@ -0,0 +1,131 @@
|
|||||||
|
/**
|
||||||
|
* ADTRAN OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds ADTRAN-branded transceiver PIDs for Total Access (TA),
|
||||||
|
* NetVanta, and Mosaic platforms (formerly ADVA Optical/Ensemble).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - ADTRAN Optics Catalog (adtran.com)
|
||||||
|
* - NetVanta Series Hardware Reference
|
||||||
|
* - Mosaic ONE Platform Hardware Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/adtran-oem.ts
|
||||||
|
* Cron: daily at 08:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AdtranPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ADTRAN_PIDS: AdtranPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "1442484G1", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "1442485G1", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "1442486G1", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH40", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "ADTRAN 1G SFP 40km" },
|
||||||
|
{ pid: "1442487G1", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "1200480G1", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "1700486F1", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "1700487F1", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "1700488F1", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "1700489F1", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "1700490F1", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "1700510F1", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "1700511F1", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "1700512F1", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "1700520F1", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "1700521F1", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "1700530F1", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "1700531F1", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "1700532F1", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "1700533F1", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD (Mosaic ONE) ────────────────────────────────────────────
|
||||||
|
{ pid: "1700550F1", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "1700551F1", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "1700552F1", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "1700553F1", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "1700491F1", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M",fiberType: "DAC", connector: "SFP+", notes: "ADTRAN 10G DAC 1m" },
|
||||||
|
{ pid: "1700492F1", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M",fiberType: "DAC", connector: "SFP+", notes: "ADTRAN 10G DAC 3m" },
|
||||||
|
{ pid: "1700534F1", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M",fiberType: "DAC", connector: "QSFP28", notes: "ADTRAN 100G DAC 1m" },
|
||||||
|
{ pid: "1700535F1", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M",fiberType: "DAC", connector: "QSFP28", notes: "ADTRAN 100G DAC 3m" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAdtranOem(): Promise<void> {
|
||||||
|
console.log("=== ADTRAN OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"ADTRAN",
|
||||||
|
"oem",
|
||||||
|
"https://www.adtran.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ADTRAN_PIDS) {
|
||||||
|
const slug = `adtran-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== ADTRAN OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ADTRAN_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAdtranOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
146
packages/scraper/src/scrapers/adtran-ta-oem.ts
Normal file
146
packages/scraper/src/scrapers/adtran-ta-oem.ts
Normal file
@ -0,0 +1,146 @@
|
|||||||
|
/**
|
||||||
|
* ADTRAN Total Access OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds ADTRAN-branded transceiver PIDs for Total Access (TA) platforms,
|
||||||
|
* OPTI-6100, SDX, and ADTRAN Optical Networking equipment. Covers
|
||||||
|
* enterprise access, PON/GPON OLT optics, and CWDM variants used in
|
||||||
|
* ADTRAN's broadband and carrier access product lines.
|
||||||
|
*
|
||||||
|
* Note: adtran-oem.ts covers general ADTRAN networking PIDs.
|
||||||
|
* This file focuses on Total Access (TA) platform optics, PON OLT SFPs,
|
||||||
|
* BiDi, and CWDM variants specific to ADTRAN's access/aggregation portfolio.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - ADTRAN Total Access 5000/6000 Series Compatible Optics
|
||||||
|
* - ADTRAN SDX Series Transceiver Compatibility Matrix
|
||||||
|
* - ADTRAN OPTI-6100 XGS Pluggable Optic Guide
|
||||||
|
* - ADTRAN PON OLT SFP+ Specification Sheet (adtran.com/products/optical)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/adtran-ta-oem.ts
|
||||||
|
* Cron: daily at 00:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AdtranTaPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ADTRAN_TA_PIDS: AdtranTaPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "1000BASE-T", fiberType: "Cu", connector: "RJ45" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom" },
|
||||||
|
{ pid: "SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "GE-T", fiberType: "Cu", connector: "RJ45" },
|
||||||
|
|
||||||
|
// ── PON OLT SFP (ADTRAN OPTI-6100 / Total Access PON) ───────────────────
|
||||||
|
{ pid: "SFP-GPON-OLT", formFactor: "SFP", speedGbps: 2.488, speed: "GPON", reachMeters: 20000, reachLabel: "GPON-OLT", fiberType: "SMF", connector: "SC", wavelengths: "1490nm TX / 1310nm RX", standard: "ITU-T G.984", category: "Telecom", notes: "ADTRAN Total Access GPON OLT SFP, ITU-T G.984.2 Class B+" },
|
||||||
|
{ pid: "SFP-XGS-PON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "XGS-PON-OLT", fiberType: "SMF", connector: "SC", wavelengths: "1577nm TX / 1270nm RX", standard: "ITU-T G.9807", category: "Telecom", notes: "ADTRAN SDX 10G XGS-PON OLT SFP+, ITU-T G.9807.1 Class N1" },
|
||||||
|
{ pid: "SFP-EPON-OLT", formFactor: "SFP", speedGbps: 1.25, speed: "EPON", reachMeters: 20000, reachLabel: "EPON-OLT", fiberType: "SMF", connector: "SC", wavelengths: "1490nm TX / 1310nm RX", standard: "IEEE 802.3ah", category: "Telecom", notes: "ADTRAN EPON OLT SFP for Total Access EPON/E-PON platforms" },
|
||||||
|
|
||||||
|
// ── BiDi SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-BIDI-1310-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-1310/1490", fiberType: "SMF", connector: "LC", wavelengths: "1310nm TX / 1490nm RX", standard: "1000BASE-BX", category: "Telecom", notes: "ADTRAN BiDi SFP 1310TX/1490RX for WDM-PON access networks" },
|
||||||
|
{ pid: "SFP-BIDI-1310-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "BiDi-1310/1550", fiberType: "SMF", connector: "LC", wavelengths: "1310nm TX / 1550nm RX", standard: "1000BASE-BX", category: "Telecom", notes: "ADTRAN BiDi SFP 1310TX/1550RX for extended reach access links" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1270-1330nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── CWDM SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "ADTRAN CWDM SFP 1550nm for WDM aggregation in TA/SDX platforms" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (PON, BiDi, CWDM, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"SFP-10G-ZR",
|
||||||
|
"SFP-GPON-OLT",
|
||||||
|
"SFP-XGS-PON-OLT",
|
||||||
|
"SFP-EPON-OLT",
|
||||||
|
"SFP-BIDI-1310-1490",
|
||||||
|
"SFP-BIDI-1310-1550",
|
||||||
|
"SFP-CWDM-1550",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeAdtranTaOem(): Promise<void> {
|
||||||
|
console.log("=== ADTRAN Total Access OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"ADTRAN",
|
||||||
|
"oem",
|
||||||
|
"https://www.adtran.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ADTRAN_TA_PIDS) {
|
||||||
|
const slug = `adtran-ta-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== ADTRAN Total Access OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ADTRAN_TA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAdtranTaOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
137
packages/scraper/src/scrapers/adva-oem.ts
Normal file
137
packages/scraper/src/scrapers/adva-oem.ts
Normal file
@ -0,0 +1,137 @@
|
|||||||
|
/**
|
||||||
|
* ADVA Optical Networking OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds ADVA-branded transceiver PIDs for FSP 150 (carrier Ethernet/ROADM),
|
||||||
|
* FSP 3000 (DWDM/OTN), and Ensemble Connector platforms.
|
||||||
|
* Note: ADVA Optical Networking is now part of ADTRAN but the ADVA brand
|
||||||
|
* and optical/telecom product lines remain distinct from adtran-oem.ts.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - ADVA FSP 150 Pluggable Optics Guide
|
||||||
|
* - ADVA FSP 3000 Transceiver Module Catalog
|
||||||
|
* - ADVA Ensemble Connector Hardware Reference
|
||||||
|
* - advaoptical.com product portfolio
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/adva-oem.ts
|
||||||
|
* Cron: daily at 20:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AdvaPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ADVA_PIDS: AdvaPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FSP-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "FSP-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "FSP-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FSP-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "FSP-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "FSP-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "FSP-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ (FSP 150 ROADM) ──────────────────────────────────────
|
||||||
|
{ pid: "FSP-SFP-10G-DW-TUNE", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "ADVA FSP 10G DWDM tunable SFP+" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FSP-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "FSP-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "FSP-QSFP28-100G-CWDM4",formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 Coherent (FSP 3000) ───────────────────────────────────────
|
||||||
|
{ pid: "FSP-CFP2-100G-DCO", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1500000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "ADVA FSP 100G CFP2 DCO coherent for FSP 3000" },
|
||||||
|
{ pid: "FSP-CFP2-100G-ACO", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 2000000, reachLabel: "ACO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "ADVA FSP 100G CFP2 ACO coherent" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FSP-QSFPDD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "FSP-QSFPDD-400G-LR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
{ pid: "FSP-QSFPDD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", category: "Telecom", notes: "ADVA FSP 400G ZR coherent QSFP-DD" },
|
||||||
|
{ pid: "FSP-QSFPDD-400G-ZRP", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 3000000, reachLabel: "ZR+", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "ADVA FSP 400G ZR+ coherent QSFP-DD" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FSP-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "FSP-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (coherent/DWDM); also respected via inline category field
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"FSP-SFP-10G-DW-TUNE",
|
||||||
|
"FSP-CFP2-100G-DCO",
|
||||||
|
"FSP-CFP2-100G-ACO",
|
||||||
|
"FSP-QSFPDD-400G-ZR",
|
||||||
|
"FSP-QSFPDD-400G-ZRP",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeAdvaOem(): Promise<void> {
|
||||||
|
console.log("=== ADVA Optical OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"ADVA Optical",
|
||||||
|
"oem",
|
||||||
|
"https://www.advaoptical.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ADVA_PIDS) {
|
||||||
|
const slug = `adva-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== ADVA Optical OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ADVA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAdvaOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
248
packages/scraper/src/scrapers/adva-optical-oem.ts
Normal file
248
packages/scraper/src/scrapers/adva-optical-oem.ts
Normal file
@ -0,0 +1,248 @@
|
|||||||
|
/**
|
||||||
|
* ADVA Optical Networking OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds ADVA-branded transceiver PIDs covering FSP 150 (carrier Ethernet),
|
||||||
|
* FSP 3000 (DWDM/OTN), and Ensemble Connector platforms.
|
||||||
|
* Note: ADVA Optical Networking merged with Adtran; the ADVA optical/telecom
|
||||||
|
* product lines remain distinct from adtran-oem.ts and adtran-ta-oem.ts.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - ADVA FSP 150 Pluggable Optics Guide (adva.com)
|
||||||
|
* - ADVA FSP 3000 Transceiver Module Catalog
|
||||||
|
* - ADVA Ensemble Connector Hardware Reference
|
||||||
|
* - adva.com product portfolio / transceiver selector
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/adva-optical-oem.ts
|
||||||
|
* Cron: daily at 22:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AdvaOpticalPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (coherent/DWDM/ZR/PON)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"ADVA-SFP-10G-ZR",
|
||||||
|
"ADVA-SFP-10G-DW-TUNE",
|
||||||
|
"ADVA-SFP-10G-CWDM-80",
|
||||||
|
"ADVA-QSFP28-100G-ZR4",
|
||||||
|
"ADVA-CFP2-100G-DCO",
|
||||||
|
"ADVA-QSFPDD-400G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const ADVA_OPTICAL_PIDS: AdvaOpticalPID[] = [
|
||||||
|
// ── 1G SFP (DataCenter) ─────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP-1G-SX",
|
||||||
|
formFactor: "SFP", speedGbps: 1, speed: "1G",
|
||||||
|
reachMeters: 550, reachLabel: "SX",
|
||||||
|
fiberType: "MMF", connector: "LC",
|
||||||
|
wavelengths: "850nm", standard: "1000BASE-SX",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP-1G-LX",
|
||||||
|
formFactor: "SFP", speedGbps: 1, speed: "1G",
|
||||||
|
reachMeters: 10000, reachLabel: "LX",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "1310nm", standard: "1000BASE-LX",
|
||||||
|
},
|
||||||
|
|
||||||
|
// ── 10G SFP+ DataCenter ─────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP-10G-SR",
|
||||||
|
formFactor: "SFP+", speedGbps: 10, speed: "10G",
|
||||||
|
reachMeters: 300, reachLabel: "SR",
|
||||||
|
fiberType: "MMF", connector: "LC",
|
||||||
|
wavelengths: "850nm", standard: "10GBASE-SR",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP-10G-LR",
|
||||||
|
formFactor: "SFP+", speedGbps: 10, speed: "10G",
|
||||||
|
reachMeters: 10000, reachLabel: "LR",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "1310nm", standard: "10GBASE-LR",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP-10G-ER",
|
||||||
|
formFactor: "SFP+", speedGbps: 10, speed: "10G",
|
||||||
|
reachMeters: 40000, reachLabel: "ER",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "1550nm", standard: "10GBASE-ER",
|
||||||
|
},
|
||||||
|
|
||||||
|
// ── 10G SFP+ Telecom ────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP-10G-ZR",
|
||||||
|
formFactor: "SFP+", speedGbps: 10, speed: "10G",
|
||||||
|
reachMeters: 80000, reachLabel: "ZR",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "1550nm", standard: "10GBASE-ZR",
|
||||||
|
category: "Telecom", notes: "ADVA FSP 10G ZR extended-reach SFP+",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP-10G-DW-TUNE",
|
||||||
|
formFactor: "SFP+", speedGbps: 10, speed: "10G",
|
||||||
|
reachMeters: 80000, reachLabel: "DWDM",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "C-band DWDM tunable",
|
||||||
|
category: "Telecom", notes: "ADVA FSP 3000 10G DWDM tunable SFP+ C-band",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP-10G-CWDM-80",
|
||||||
|
formFactor: "SFP+", speedGbps: 10, speed: "10G",
|
||||||
|
reachMeters: 80000, reachLabel: "CWDM",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "1530–1610nm CWDM",
|
||||||
|
category: "Telecom", notes: "ADVA FSP 10G CWDM SFP+ 1530-1610nm 80km",
|
||||||
|
},
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP28-25G-SR",
|
||||||
|
formFactor: "SFP28", speedGbps: 25, speed: "25G",
|
||||||
|
reachMeters: 100, reachLabel: "SR",
|
||||||
|
fiberType: "MMF", connector: "LC",
|
||||||
|
wavelengths: "850nm", standard: "25GBASE-SR",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "ADVA-SFP28-25G-LR",
|
||||||
|
formFactor: "SFP28", speedGbps: 25, speed: "25G",
|
||||||
|
reachMeters: 10000, reachLabel: "LR",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "1310nm", standard: "25GBASE-LR",
|
||||||
|
},
|
||||||
|
|
||||||
|
// ── 100G QSFP28 DataCenter ──────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-QSFP28-100G-SR4",
|
||||||
|
formFactor: "QSFP28", speedGbps: 100, speed: "100G",
|
||||||
|
reachMeters: 100, reachLabel: "SR4",
|
||||||
|
fiberType: "MMF", connector: "MPO",
|
||||||
|
wavelengths: "850nm", standard: "100GBASE-SR4",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "ADVA-QSFP28-100G-LR4",
|
||||||
|
formFactor: "QSFP28", speedGbps: 100, speed: "100G",
|
||||||
|
reachMeters: 10000, reachLabel: "LR4",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "1295–1310nm", standard: "100GBASE-LR4",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "ADVA-QSFP28-100G-ER4",
|
||||||
|
formFactor: "QSFP28", speedGbps: 100, speed: "100G",
|
||||||
|
reachMeters: 40000, reachLabel: "ER4",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "1295–1310nm", standard: "100GBASE-ER4",
|
||||||
|
},
|
||||||
|
|
||||||
|
// ── 100G QSFP28 Telecom (ZR4 coherent) ─────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-QSFP28-100G-ZR4",
|
||||||
|
formFactor: "QSFP28", speedGbps: 100, speed: "100G",
|
||||||
|
reachMeters: 80000, reachLabel: "ZR4",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "C-band DWDM",
|
||||||
|
category: "Telecom", notes: "ADVA FSP 3000 100G QSFP28 ZR4 coherent",
|
||||||
|
},
|
||||||
|
|
||||||
|
// ── 100G CFP2-DCO Telecom ───────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-CFP2-100G-DCO",
|
||||||
|
formFactor: "CFP2", speedGbps: 100, speed: "100G",
|
||||||
|
reachMeters: 1000000, reachLabel: "DCO",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "C-band",
|
||||||
|
category: "Telecom", notes: "ADVA FSP 3000 100G CFP2-DCO coherent 1000km metro/long-haul",
|
||||||
|
},
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD DataCenter ─────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-QSFPDD-400G-DR4",
|
||||||
|
formFactor: "QSFP-DD", speedGbps: 400, speed: "400G",
|
||||||
|
reachMeters: 500, reachLabel: "DR4",
|
||||||
|
fiberType: "SMF", connector: "MPO",
|
||||||
|
wavelengths: "1310nm", standard: "400GBASE-DR4",
|
||||||
|
},
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD Telecom (ZR coherent) ─────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "ADVA-QSFPDD-400G-ZR",
|
||||||
|
formFactor: "QSFP-DD", speedGbps: 400, speed: "400G",
|
||||||
|
reachMeters: 120000, reachLabel: "ZR",
|
||||||
|
fiberType: "SMF", connector: "LC",
|
||||||
|
wavelengths: "C-band", standard: "400ZR",
|
||||||
|
category: "Telecom", notes: "ADVA FSP 3000 400G QSFP-DD ZR coherent (OpenROADM/OpenZR+)",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAdvaOpticalOem(): Promise<void> {
|
||||||
|
console.log("=== ADVA Optical Networking OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"ADVA Optical Networking",
|
||||||
|
"oem",
|
||||||
|
"https://www.adva.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ADVA_OPTICAL_PIDS) {
|
||||||
|
const slug = `adva-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== ADVA Optical Networking OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total: ${ADVA_OPTICAL_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAdvaOpticalOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
117
packages/scraper/src/scrapers/advantech-oem.ts
Normal file
117
packages/scraper/src/scrapers/advantech-oem.ts
Normal file
@ -0,0 +1,117 @@
|
|||||||
|
/**
|
||||||
|
* Advantech OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Advantech industrial-grade SFP modules used in
|
||||||
|
* EKI and EKS series industrial Ethernet managed switches and routers.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Advantech SFP module datasheets (advantech.com)
|
||||||
|
* - Advantech EKI/EKS series hardware installation guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/advantech-oem.ts
|
||||||
|
* Cron: daily at 14:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AdvantechPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ADVANTECH_PIDS: AdvantechPID[] = [
|
||||||
|
// ── 100M SFP (legacy industrial) ────────────────────────────────────────
|
||||||
|
{ pid: "SFP-FE-MM-SC", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 2000, reachLabel: "FX", fiberType: "MMF", connector: "SC", wavelengths: "850nm", standard: "100BASE-FX", notes: "Advantech 100M MM SFP" },
|
||||||
|
{ pid: "SFP-FE-SM-SC", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 40000, reachLabel: "LFX", fiberType: "SMF", connector: "SC", wavelengths: "1310nm", notes: "Advantech 100M SM SFP 40km" },
|
||||||
|
|
||||||
|
// ── 1G SFP ───────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GE-SX-MM-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-GE-LX-SM-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-GE-LH-SM-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "Advantech 1G SM SFP LH 40km" },
|
||||||
|
{ pid: "SFP-GE-ZX-SM-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "SFP-GE-T-RJ45", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "SFP-GE-BX-U", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BX-U", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550", notes: "Advantech 1G BiDi SFP upstream" },
|
||||||
|
{ pid: "SFP-GE-BX-D", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BX-D", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310", notes: "Advantech 1G BiDi SFP downstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR-MM-LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR-SM-LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER-SM-LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR-SM-LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "SFP-10G-BX-U", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BX-U", fiberType: "SMF", connector: "LC", wavelengths: "TX1270/RX1330", notes: "Advantech 10G BiDi SFP+ upstream" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-SR-MM-LC", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "SFP-25G-LR-SM-LC", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── DAC ──────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-PLUS-DAC-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC1", fiberType: "DAC", connector: "LC" },
|
||||||
|
{ pid: "SFP-PLUS-DAC-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC3", fiberType: "DAC", connector: "LC" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAdvantechOem(): Promise<void> {
|
||||||
|
console.log("=== Advantech OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Advantech",
|
||||||
|
"oem",
|
||||||
|
"https://www.advantech.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ADVANTECH_PIDS) {
|
||||||
|
const slug = `advantech-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Advantech OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ADVANTECH_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAdvantechOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
123
packages/scraper/src/scrapers/ale-oem.ts
Normal file
123
packages/scraper/src/scrapers/ale-oem.ts
Normal file
@ -0,0 +1,123 @@
|
|||||||
|
/**
|
||||||
|
* Alcatel-Lucent Enterprise (ALE) OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds ALE-branded transceiver PIDs for OmniSwitch 6xxx/9xxx
|
||||||
|
* and OmniAccess Stellar platforms.
|
||||||
|
* Sources: ALE Transceiver Module Compatibility Guide (al-enterprise.com)
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ale-oem.ts
|
||||||
|
* Cron: daily at 10:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AlePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ALE_PIDS: AlePID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "iSFP-GIG-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "iSFP-GIG-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "iSFP-GIG-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "iSFP-GIG-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "iSFP-GIG-LH40", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH40", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
{ pid: "iSFP-GIG-BX-U", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BiDi-U", fiberType: "SMF", connector: "LC", wavelengths: "1310/1490nm", notes: "BiDi TX1310/RX1490" },
|
||||||
|
{ pid: "iSFP-GIG-BX-D", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BiDi-D", fiberType: "SMF", connector: "LC", wavelengths: "1490/1310nm", notes: "BiDi TX1490/RX1310" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "iSFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "iSFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "iSFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "iSFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "iSFP-10G-LRM", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM" },
|
||||||
|
{ pid: "iSFP-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "iSFP-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "iSFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "iSFP-25G-ER", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP28-100G-SR4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "QSFP28-100G-LR4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "QSFP28-100G-CWDM4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "QSFP28-100G-DR", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "iSFP-10G-DAC-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "iSFP-10G-DAC-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "QSFP-100G-DAC-1M", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "QSFP-100G-DAC-3M", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAleOem(): Promise<void> {
|
||||||
|
console.log("=== Alcatel-Lucent Enterprise (ALE) OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Alcatel-Lucent Enterprise",
|
||||||
|
"oem",
|
||||||
|
"https://www.al-enterprise.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ALE_PIDS) {
|
||||||
|
const slug = `ale-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== ALE OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ALE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAleOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
127
packages/scraper/src/scrapers/allied-telesis-oem.ts
Normal file
127
packages/scraper/src/scrapers/allied-telesis-oem.ts
Normal file
@ -0,0 +1,127 @@
|
|||||||
|
/**
|
||||||
|
* Allied Telesis OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Allied Telesis-branded SFP/SFP+/QSFP PIDs for AT-x530, AT-x550,
|
||||||
|
* AT-x950, and SwitchBlade series.
|
||||||
|
*
|
||||||
|
* Sources: Allied Telesis Transceiver Guide (alliedtelesis.com)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/allied-telesis-oem.ts
|
||||||
|
* Cron: daily at 09:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AlliedPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ALLIED_PIDS: AlliedPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AT-SPSX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "AT-SPLX10", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "AT-SPLX40", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH40", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
{ pid: "AT-SPZX80", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "AT-SPTX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "AT-SPBD10-13", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BiDi-1310", fiberType: "SMF", connector: "LC", wavelengths: "1310/1550nm", notes: "BiDi TX1310/RX1550" },
|
||||||
|
{ pid: "AT-SPBD10-15", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BiDi-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550/1310nm", notes: "BiDi TX1550/RX1310" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AT-SP10SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "AT-SP10LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "AT-SP10ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "AT-SP10ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "AT-SP10LRM", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM" },
|
||||||
|
{ pid: "AT-SP10T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
{ pid: "AT-SP10BD13", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1270/1330nm", notes: "10G BiDi single fiber" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AT-SP25SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "AT-SP25LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "AT-SP25ER", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AT-QSFPSR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "AT-QSFPLR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AT-QSFP28SR4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "AT-QSFP28LR4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "AT-QSFP28CWDM4",formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "AT-QSFP28DR", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AT-SP10TW1", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "AT-SP10TW3", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "AT-QSFPTW1", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP+" },
|
||||||
|
{ pid: "AT-QSFP28TW1", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "AT-QSFP28TW3", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAlliedTelesisOem(): Promise<void> {
|
||||||
|
console.log("=== Allied Telesis OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Allied Telesis",
|
||||||
|
"oem",
|
||||||
|
"https://www.alliedtelesis.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ALLIED_PIDS) {
|
||||||
|
const slug = `allied-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Allied Telesis OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ALLIED_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAlliedTelesisOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
116
packages/scraper/src/scrapers/antaira-oem.ts
Normal file
116
packages/scraper/src/scrapers/antaira-oem.ts
Normal file
@ -0,0 +1,116 @@
|
|||||||
|
/**
|
||||||
|
* Antaira Technologies OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Antaira industrial-grade SFP modules used in
|
||||||
|
* harsh-environment industrial Ethernet managed switches.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Antaira SFP/SFP+ module datasheets (antaira.com)
|
||||||
|
* - Antaira industrial switch hardware installation guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/antaira-oem.ts
|
||||||
|
* Cron: daily at 15:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AntairaPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ANTAIRA_PIDS: AntairaPID[] = [
|
||||||
|
// ── 100M SFP ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-FE-S15-S", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 15000, reachLabel: "LFX", fiberType: "SMF", connector: "SC", wavelengths: "1310nm", standard: "100BASE-FX", notes: "Antaira 100M SM SFP 15km SC" },
|
||||||
|
{ pid: "SFP-FE-M2-S", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 2000, reachLabel: "FX", fiberType: "MMF", connector: "SC", wavelengths: "850nm", standard: "100BASE-FX", notes: "Antaira 100M MM SFP SC" },
|
||||||
|
{ pid: "SFP-FE-S40-S", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 40000, reachLabel: "LFX", fiberType: "SMF", connector: "SC", wavelengths: "1310nm", notes: "Antaira 100M SM SFP 40km SC" },
|
||||||
|
{ pid: "SFP-FE-S80-S", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 80000, reachLabel: "LFX", fiberType: "SMF", connector: "SC", wavelengths: "1550nm", notes: "Antaira 100M SM SFP 80km SC" },
|
||||||
|
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GE-M550-S", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-GE-S10-S", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-GE-S20-S", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "Antaira 1G SM SFP 20km" },
|
||||||
|
{ pid: "SFP-GE-S40-S", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "Antaira 1G SM SFP 40km" },
|
||||||
|
{ pid: "SFP-GE-S80-S", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "SFP-GE-BX20U", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550", notes: "Antaira 1G BiDi SFP upstream" },
|
||||||
|
{ pid: "SFP-GE-BX20D", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310", notes: "Antaira 1G BiDi SFP downstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "SFP-10G-BXU", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1270/RX1330", notes: "Antaira 10G BiDi SFP+ upstream" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DACP-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC", fiberType: "DAC", connector: "LC" },
|
||||||
|
{ pid: "DACP-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC", fiberType: "DAC", connector: "LC" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAntairaOem(): Promise<void> {
|
||||||
|
console.log("=== Antaira Technologies OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Antaira Technologies",
|
||||||
|
"oem",
|
||||||
|
"https://www.antaira.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ANTAIRA_PIDS) {
|
||||||
|
const slug = `antaira-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Antaira Technologies OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ANTAIRA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAntairaOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
137
packages/scraper/src/scrapers/aoi-oem.ts
Normal file
137
packages/scraper/src/scrapers/aoi-oem.ts
Normal file
@ -0,0 +1,137 @@
|
|||||||
|
/**
|
||||||
|
* Applied Optoelectronics (AOI) OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds AOI-branded transceiver PIDs. Applied Optoelectronics, Inc.
|
||||||
|
* (Sugar Land, TX) is a US-listed manufacturer of fiber-optic networking
|
||||||
|
* products including transceivers for CATV, telecom, and data center markets.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - AOI Product Portfolio (ao-inc.com)
|
||||||
|
* - AOI Data Center Transceiver Specifications
|
||||||
|
* - AOI CATV / FTTH Optical Transceiver Application Notes
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/aoi-oem.ts
|
||||||
|
* Cron: daily at 21:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AoiPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"AOI-SFP-10G-ZR",
|
||||||
|
"AOI-SFP-GPON-OLT",
|
||||||
|
"AOI-SFP-XGS-PON-OLT",
|
||||||
|
"AOI-DWDM-SFP10G-C",
|
||||||
|
"AOI-CFP2-DCO-100G",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const AOI_PIDS: AoiPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AOI-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "AOI 1G SFP SX 550m MMF" },
|
||||||
|
{ pid: "AOI-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "AOI 1G SFP LX 10km SMF" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AOI-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "AOI 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "AOI-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "AOI 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "AOI-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "AOI 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "AOI-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "AOI 10G SFP+ ZR 80km SMF" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AOI-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "AOI 25G SFP28 SR 100m MMF" },
|
||||||
|
{ pid: "AOI-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "AOI 25G SFP28 LR 10km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AOI-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "AOI 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "AOI-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "AOI 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AOI-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "AOI 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "AOI-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "AOI 100G QSFP28 LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AOI-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "AOI 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "AOI-QSFP-DD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4", notes: "AOI 400G QSFP-DD FR4 2km SMF" },
|
||||||
|
|
||||||
|
// ── PON / Access (Telecom) ───────────────────────────────────────────────
|
||||||
|
{ pid: "AOI-SFP-GPON-OLT", formFactor: "SFP", speedGbps: 2.5, speed: "2.5G", reachMeters: 20000, reachLabel: "GPON", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "ITU-T G.984", notes: "AOI GPON OLT SFP TX1490/RX1310 20km" },
|
||||||
|
{ pid: "AOI-SFP-XGS-PON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 20000, reachLabel: "XGS-PON",fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "ITU-T G.9807", notes: "AOI XGS-PON OLT SFP+ TX1577/RX1270 20km" },
|
||||||
|
|
||||||
|
// ── DWDM (Telecom) ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "AOI-DWDM-SFP10G-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "ITU-T G.694.1", notes: "AOI 10G DWDM SFP+ C-band tunable 80km" },
|
||||||
|
|
||||||
|
// ── CFP2-DCO (Telecom coherent) ──────────────────────────────────────────
|
||||||
|
{ pid: "AOI-CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1000000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF CFP2-DCO", notes: "AOI 100G CFP2 DCO coherent C-band, 1000km" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAoiOem(): Promise<void> {
|
||||||
|
console.log("=== Applied Optoelectronics (AOI) OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Applied Optoelectronics",
|
||||||
|
"oem",
|
||||||
|
"https://www.ao-inc.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of AOI_PIDS) {
|
||||||
|
const slug = `aoi-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== AOI OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${AOI_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAoiOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
126
packages/scraper/src/scrapers/arista-7000-oem.ts
Normal file
126
packages/scraper/src/scrapers/arista-7000-oem.ts
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
/**
|
||||||
|
* Arista 7000 Series OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Arista 7000 Series-branded transceiver PIDs for data center switching
|
||||||
|
* platforms (7010, 7020, 7050, 7060, 7080, 7170, 7260, 7280, 7300, 7500, 7800).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Arista 7000 Series Hardware Architecture Guide
|
||||||
|
* - Arista EOS Transceiver Compatibility Matrix (arista.com)
|
||||||
|
* - Arista 7800R3 / 7500R3 400G Platform Datasheet
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/arista-7000-oem.ts
|
||||||
|
* Cron: daily at 14:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface Arista7kPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Arista 7000 Series transceiver catalog ──────────────────────────────────
|
||||||
|
// Source: Arista 7000 Series HCL and EOS Transceiver Guide
|
||||||
|
const ARISTA_7K_PIDS: Arista7kPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Arista 1G multimode SFP, 7010/7050/7060 management ports" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Arista 1G single-mode SFP 10km for 7000 management/OOB" },
|
||||||
|
{ pid: "SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Arista 1G copper SFP for 7000 Series out-of-band mgmt" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Arista 10G SR SFP+ for 7050/7060 ToR server downlinks" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Arista 10G LR SFP+ 10km for 7000 inter-DC links" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Arista 10G ER 40km for 7000 campus/metro aggregation" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Arista 10G ZR 80km dark-fiber WAN uplink for 7000" },
|
||||||
|
{ pid: "SFP-10G-LR-S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR-S", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Arista 10G LR-S variant, reduced power for 7050TX" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Arista 25G SR SFP28 for 7060X/7260X server leaf ports" },
|
||||||
|
{ pid: "SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Arista 25G LR SFP28 for 7060/7260 inter-rack spine uplinks" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Arista 40G SR4 QSFP+ for 7050Q/7260Q fabric uplinks" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", notes: "Arista 40G LR4 QSFP+ 10km for 7000 DCI spine uplinks" },
|
||||||
|
{ pid: "QSFP-40G-SRBD", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR-BiDi",fiberType: "MMF", connector: "LC", wavelengths: "TX832-860/RX882-910nm", standard: "40GBASE-BiDi", notes: "Arista 40G BiDi QSFP+ duplex LC OM3/OM4 for 7050/7060 rackscale" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Arista 100G SR4 QSFP28 for 7260/7280/7300 100G fabric" },
|
||||||
|
{ pid: "QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Arista 100G LR4 QSFP28 10km for 7000 DCI and spine" },
|
||||||
|
{ pid: "QSFP-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4", notes: "Arista 100G CWDM4 QSFP28 2km SMF for 7000 metro DCI" },
|
||||||
|
{ pid: "QSFP-100G-PSM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "PSM4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "100GBASE-PSM4", notes: "Arista 100G PSM4 QSFP28 500m parallel SMF for 7000 hyper-scale" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Arista 400G DR4 QSFP-DD for 7800R3/7500R3 leaf-spine fabric" },
|
||||||
|
{ pid: "QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "Arista 400G SR8 QSFP-DD for 7800R3 OM4 within-DC fabric" },
|
||||||
|
|
||||||
|
// ── 800G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-800G-DR8", formFactor: "QSFP-DD", speedGbps: 800, speed: "800G", reachMeters: 500, reachLabel: "DR8", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "800GBASE-DR8", notes: "Arista 800G DR8 QSFP-DD for next-gen 7800 AI/ML fabric spine" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeArista7000Oem(): Promise<void> {
|
||||||
|
console.log("=== Arista 7000 Series OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Arista 7000",
|
||||||
|
"oem",
|
||||||
|
"https://www.arista.com/en/products/7000-series",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ARISTA_7K_PIDS) {
|
||||||
|
const slug = `arista-7k-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Arista 7000 OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ARISTA_7K_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeArista7000Oem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
123
packages/scraper/src/scrapers/arris-oem.ts
Normal file
123
packages/scraper/src/scrapers/arris-oem.ts
Normal file
@ -0,0 +1,123 @@
|
|||||||
|
/**
|
||||||
|
* CommScope ARRIS OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds CommScope ARRIS-branded transceiver PIDs for CMTS, E6000,
|
||||||
|
* C3, and CAP-500 broadband platforms with SFP ports.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - CommScope ARRIS CMTS E6000 Hardware Installation Guide
|
||||||
|
* - CommScope CAP-500 Platform Transceiver Compatibility Matrix
|
||||||
|
* - CommScope C3 CCAP Platform Hardware Reference
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/arris-oem.ts
|
||||||
|
* Cron: daily at 19:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface ArrisPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ARRIS_PIDS: ArrisPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COM-SFP-1GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "COM-SFP-1GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "COM-SFP-1GE-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "COM-SFP-1GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "COM-SFP-1GE-BXU", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BXU", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", standard: "1000BASE-BX", notes: "ARRIS 1G BiDi SFP upstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COM-SFP-10GE-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "COM-SFP-10GE-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "COM-SFP-10GE-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "COM-SFP-10GE-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COM-SFP28-25GE-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "COM-SFP28-25GE-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COM-QSFP-40GE-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "COM-QSFP-40GE-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COM-QSFP28-100GE-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "COM-QSFP28-100GE-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "COM-QSFP28-100GE-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "COM-QSFP28-100GE-DR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COM-DAC-10GE-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "COM-DAC-10GE-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "COM-DAC-100GE-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "COM-DAC-100GE-3M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeArrisOem(): Promise<void> {
|
||||||
|
console.log("=== CommScope ARRIS OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"CommScope ARRIS",
|
||||||
|
"oem",
|
||||||
|
"https://www.commscope.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ARRIS_PIDS) {
|
||||||
|
const slug = `arris-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== CommScope ARRIS OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ARRIS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeArrisOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
121
packages/scraper/src/scrapers/aruba-cx-oem.ts
Normal file
121
packages/scraper/src/scrapers/aruba-cx-oem.ts
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
/**
|
||||||
|
* Aruba CX OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds HPE Aruba CX-branded transceiver PIDs for Aruba CX 6300, 6400,
|
||||||
|
* 8325, and 8400 series data center and campus switches.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - HPE Aruba CX Switch Series Data Sheet
|
||||||
|
* - HPE Aruba Networking Transceiver Selection Guide (a00045495enw)
|
||||||
|
* - HPE Aruba CX Switches Hardware Compatibility Matrix
|
||||||
|
* - HPE Networking Transceiver Module QuickSpecs
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/aruba-cx-oem.ts
|
||||||
|
* Cron: daily at 13:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface ArubaCxPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ARUBA_CX_PIDS: ArubaCxPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "J9142B", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "HPE Aruba 1G SX SFP LC 550m MMF transceiver" },
|
||||||
|
{ pid: "J9143B", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "HPE Aruba 1G LX SFP LC 10km SMF transceiver" },
|
||||||
|
{ pid: "ARUBA-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "HPE Aruba 1G T SFP RJ45 copper transceiver" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "J9150A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "HPE Aruba X2 10G SFP+ LC SR 300m MMF transceiver" },
|
||||||
|
{ pid: "J9151E", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "HPE Aruba X2 10G SFP+ LC LR 10km SMF transceiver" },
|
||||||
|
{ pid: "J9152E", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "HPE Aruba X2 10G SFP+ LC ER 40km SMF transceiver" },
|
||||||
|
{ pid: "J9153E", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "HPE Aruba X2 10G SFP+ LC ZR 80km SMF transceiver" },
|
||||||
|
{ pid: "J9153A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM", notes: "HPE Aruba X2 10G SFP+ LC LRM 220m MMF transceiver" },
|
||||||
|
{ pid: "ARUBA-SFP-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T", notes: "HPE Aruba 10G T SFP+ RJ45 copper transceiver 30m Cat6A" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "JL308A", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "HPE Aruba CX 25G SFP28 LC SR 100m MMF transceiver" },
|
||||||
|
{ pid: "JL309A", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "HPE Aruba CX 25G SFP28 LC LR 10km SMF transceiver" },
|
||||||
|
{ pid: "JL546A", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-ER", notes: "HPE Aruba CX 25G SFP28 LC ER 40km SMF transceiver" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "J9285B", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "HPE Aruba X142 40G QSFP+ MPO SR4 150m MMF transceiver" },
|
||||||
|
{ pid: "J9285D", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", notes: "HPE Aruba X142 40G QSFP+ LC LR4 10km SMF transceiver" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "JL072A", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "HPE Aruba 100G QSFP28 MPO SR4 100m MMF transceiver" },
|
||||||
|
{ pid: "JL073A", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "HPE Aruba 100G QSFP28 LC LR4 10km SMF transceiver" },
|
||||||
|
{ pid: "ARUBA-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4", notes: "HPE Aruba CX 100G QSFP28 CWDM4 2km SMF transceiver" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "R9B18A", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "HPE Aruba CX 8400 400G QSFP-DD MPO DR4 500m SMF transceiver" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeArubaCxOem(): Promise<void> {
|
||||||
|
console.log("=== Aruba CX OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Aruba CX",
|
||||||
|
"oem",
|
||||||
|
"https://www.arubanetworks.com/products/switches/",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ARUBA_CX_PIDS) {
|
||||||
|
const slug = `aruba-cx-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Aruba CX OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ARUBA_CX_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeArubaCxOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
330
packages/scraper/src/scrapers/aws-oem.ts
Normal file
330
packages/scraper/src/scrapers/aws-oem.ts
Normal file
@ -0,0 +1,330 @@
|
|||||||
|
/**
|
||||||
|
* AWS OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Amazon Web Services-branded transceiver PIDs used in Direct Connect
|
||||||
|
* dedicated connections and AWS Outposts rack/server hardware. AWS uses an
|
||||||
|
* AWS- prefix for SFP/QSFP optics validated on its custom network ASICs.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - AWS Direct Connect hardware requirements (aws.amazon.com/directconnect/)
|
||||||
|
* - AWS Outposts rack network hardware guide
|
||||||
|
* - SFF-8024 / IEEE 802.3 speed codes
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/aws-oem.ts
|
||||||
|
* Cron: daily at 07:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AwsPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that map to telecom / DWDM variants (ZR, ZR4)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"AWS-SFP-10G-ZR",
|
||||||
|
"AWS-QSFP28-100G-ZR4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const AWS_PIDS: AwsPID[] = [
|
||||||
|
// ── 1G SFP ───────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP-1G-SX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 550,
|
||||||
|
reachLabel: "SX",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "1000BASE-SX",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE SFP SX; AWS Outposts rack management port; OM2/OM3 MMF up to 550m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP-1G-LX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LX",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "1000BASE-LX",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE SFP LX; AWS Outposts rack management port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 1G Copper SFP ────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP-GE-T",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "T",
|
||||||
|
fiberType: "DAC",
|
||||||
|
connector: "RJ45",
|
||||||
|
standard: "1000BASE-T",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE copper SFP RJ45; AWS Outposts rack management copper option; Cat5e/6",
|
||||||
|
},
|
||||||
|
// ── 10G SFP+ ─────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP-10G-SR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ SR; AWS Outposts server uplink; OM3/OM4 MMF up to 300m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP-10G-LR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ LR; AWS Direct Connect 10G port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP-10G-ER",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 40000,
|
||||||
|
reachLabel: "ER",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1550nm",
|
||||||
|
standard: "10GBASE-ER",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ ER; AWS Direct Connect extended-reach; OS2 SMF up to 40km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP-10G-ZR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 80000,
|
||||||
|
reachLabel: "ZR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1550nm",
|
||||||
|
standard: "10GBASE-ZR",
|
||||||
|
category: "Telecom",
|
||||||
|
notes: "10GbE SFP+ ZR; AWS Direct Connect long-haul 80km DWDM link; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP28-25G-SR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "25GBASE-SR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "25GbE SFP28 SR; AWS Outposts server host port; OM4 MMF up to 100m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-SFP28-25G-LR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "25GBASE-LR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "25GbE SFP28 LR; AWS Outposts server host port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 40G QSFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AWS-QSFP-40G-SR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "40GBASE-SR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "40GbE QSFP+ SR4; AWS Outposts rack spine uplink; OM3/OM4 MPO-12 up to 150m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-QSFP-40G-LR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1270-1330nm",
|
||||||
|
standard: "40GBASE-LR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "40GbE QSFP+ LR4; AWS Direct Connect 40G port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AWS-QSFP28-100G-SR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "100GBASE-SR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "100GbE QSFP28 SR4; AWS Direct Connect 100G / Outposts spine; OM4 MPO-12 up to 100m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-QSFP28-100G-LR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-LR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "100GbE QSFP28 LR4; AWS Direct Connect 100G dedicated connection; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-QSFP28-100G-ZR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 80000,
|
||||||
|
reachLabel: "ZR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1271-1331nm",
|
||||||
|
standard: "100G-ZR4",
|
||||||
|
category: "Telecom",
|
||||||
|
notes: "100G QSFP28 ZR4; AWS Direct Connect DWDM long-haul 80km; coherent-lite; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 400G QSFP-DD ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AWS-QSFP-DD-400G-DR4",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 500,
|
||||||
|
reachLabel: "DR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "400GBASE-DR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "400GbE QSFP-DD DR4; AWS Outposts next-gen spine interconnect; OS2 SMF up to 500m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AWS-QSFP-DD-400G-SR8",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR8",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "400GBASE-SR8",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "400GbE QSFP-DD SR8; AWS Outposts next-gen spine interconnect; OM4/OM5 MPO-16 up to 100m",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAwsOem(): Promise<void> {
|
||||||
|
console.log("=== AWS OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Amazon Web Services",
|
||||||
|
"oem",
|
||||||
|
"https://aws.amazon.com/directconnect/",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of AWS_PIDS) {
|
||||||
|
const slug = `aws-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : (p.category ?? "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== AWS OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${AWS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAwsOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
331
packages/scraper/src/scrapers/azure-oem.ts
Normal file
331
packages/scraper/src/scrapers/azure-oem.ts
Normal file
@ -0,0 +1,331 @@
|
|||||||
|
/**
|
||||||
|
* Microsoft Azure OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Microsoft Azure-branded transceiver PIDs used in ExpressRoute
|
||||||
|
* dedicated ports and Azure Stack Hub/HCI hardware. Azure uses an AZ-
|
||||||
|
* prefix for SFP/QSFP optics validated on its custom Azurite/SONiC-based
|
||||||
|
* network ASICs and Azure Stack hardware.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Azure ExpressRoute prerequisites (azure.microsoft.com/products/expressroute/)
|
||||||
|
* - Azure Stack Hub network hardware guide
|
||||||
|
* - SFF-8024 / IEEE 802.3 speed codes
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/azure-oem.ts
|
||||||
|
* Cron: daily at 07:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AzurePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that map to telecom / DWDM variants (ZR, ER4)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"AZ-SFP-10G-ZR",
|
||||||
|
"AZ-QSFP28-100G-ER4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const AZURE_PIDS: AzurePID[] = [
|
||||||
|
// ── 1G SFP ───────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP-1G-SX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 550,
|
||||||
|
reachLabel: "SX",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "1000BASE-SX",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE SFP SX; Azure Stack Hub management port; OM2/OM3 MMF up to 550m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP-1G-LX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LX",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "1000BASE-LX",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE SFP LX; Azure Stack Hub management port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 1G Copper SFP ────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP-GE-T",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "T",
|
||||||
|
fiberType: "DAC",
|
||||||
|
connector: "RJ45",
|
||||||
|
standard: "1000BASE-T",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE copper SFP RJ45; Azure Stack Hub management copper option; Cat5e/6",
|
||||||
|
},
|
||||||
|
// ── 10G SFP+ ─────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP-10G-SR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ SR; Azure Stack Hub server uplink; OM3/OM4 MMF up to 300m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP-10G-LR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ LR; Azure ExpressRoute 10G port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP-10G-ER",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 40000,
|
||||||
|
reachLabel: "ER",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1550nm",
|
||||||
|
standard: "10GBASE-ER",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ ER; Azure ExpressRoute extended-reach; OS2 SMF up to 40km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP-10G-ZR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 80000,
|
||||||
|
reachLabel: "ZR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1550nm",
|
||||||
|
standard: "10GBASE-ZR",
|
||||||
|
category: "Telecom",
|
||||||
|
notes: "10GbE SFP+ ZR; Azure ExpressRoute long-haul DWDM 80km; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP28-25G-SR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "25GBASE-SR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "25GbE SFP28 SR; Azure Stack HCI server host port; OM4 MMF up to 100m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-SFP28-25G-LR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "25GBASE-LR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "25GbE SFP28 LR; Azure Stack HCI server host port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 40G QSFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AZ-QSFP-40G-SR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "40GBASE-SR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "40GbE QSFP+ SR4; Azure Stack Hub Top-of-Rack spine uplink; OM3/OM4 MPO-12 up to 150m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-QSFP-40G-LR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1270-1330nm",
|
||||||
|
standard: "40GBASE-LR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "40GbE QSFP+ LR4; Azure ExpressRoute 40G port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AZ-QSFP28-100G-SR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "100GBASE-SR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "100GbE QSFP28 SR4; Azure ExpressRoute 100G / Stack HCI spine; OM4 MPO-12 up to 100m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-QSFP28-100G-LR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-LR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "100GbE QSFP28 LR4; Azure ExpressRoute 100G dedicated connection; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-QSFP28-100G-ER4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 40000,
|
||||||
|
reachLabel: "ER4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-ER4",
|
||||||
|
category: "Telecom",
|
||||||
|
notes: "100G QSFP28 ER4; Azure ExpressRoute long-reach 40km inter-datacenter replication; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 400G QSFP-DD ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AZ-QSFP-DD-400G-DR4",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 500,
|
||||||
|
reachLabel: "DR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "400GBASE-DR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "400GbE QSFP-DD DR4; Azure next-gen datacenter spine interconnect; OS2 SMF up to 500m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AZ-QSFP-DD-400G-SR8",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR8",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "400GBASE-SR8",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "400GbE QSFP-DD SR8; Azure next-gen datacenter spine interconnect; OM4/OM5 MPO-16 up to 100m",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeAzureOem(): Promise<void> {
|
||||||
|
console.log("=== Microsoft Azure OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Microsoft Azure",
|
||||||
|
"oem",
|
||||||
|
"https://azure.microsoft.com/en-us/products/expressroute/",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of AZURE_PIDS) {
|
||||||
|
const slug = `azure-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : (p.category ?? "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Microsoft Azure OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${AZURE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeAzureOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
121
packages/scraper/src/scrapers/barracuda-oem.ts
Normal file
121
packages/scraper/src/scrapers/barracuda-oem.ts
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
/**
|
||||||
|
* Barracuda Networks OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Barracuda-branded transceiver PIDs for CloudGen Firewall
|
||||||
|
* F/S series and Email Security Gateway appliances.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Barracuda Networks Hardware Compatibility Guide (campus.barracuda.com)
|
||||||
|
* - CloudGen Firewall F-Series Hardware Reference
|
||||||
|
* - Barracuda CloudGen Firewall S-Series Datasheet
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/barracuda-oem.ts
|
||||||
|
* Cron: daily at 16:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface BarracudaPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const BARRACUDA_PIDS: BarracudaPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BNSF-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "BNSF-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "BNSF-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "BNSF-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BNSF-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "BNSF-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "BNSF-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "BNSF-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "BNSF-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BNSF-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "BNSF-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BNSF-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "BNSF-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BNSF-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "BNSF-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "BNSF-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BNSF-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "BNSF-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "BNSF-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeBarracudaOem(): Promise<void> {
|
||||||
|
console.log("=== Barracuda Networks OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Barracuda Networks",
|
||||||
|
"oem",
|
||||||
|
"https://www.barracuda.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of BARRACUDA_PIDS) {
|
||||||
|
const slug = `barracuda-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Barracuda Networks OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${BARRACUDA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeBarracudaOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
118
packages/scraper/src/scrapers/beckhoff-oem.ts
Normal file
118
packages/scraper/src/scrapers/beckhoff-oem.ts
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
/**
|
||||||
|
* Beckhoff OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Beckhoff SFP/SFP+/QSFP+ industrial modules
|
||||||
|
* used in EtherCAT/TwinCAT industrial networking infrastructure and
|
||||||
|
* CX, EK, and EL series automation hardware.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Beckhoff SFP module datasheets (beckhoff.com)
|
||||||
|
* - Beckhoff Industrial Ethernet documentation
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/beckhoff-oem.ts
|
||||||
|
* Cron: daily at 15:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface BeckhoffPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const BECKHOFF_PIDS: BeckhoffPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BHF-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Beckhoff industrial SFP 1G SX" },
|
||||||
|
{ pid: "BHF-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Beckhoff industrial SFP 1G LX" },
|
||||||
|
{ pid: "BHF-SFP-1G-LH", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "Beckhoff industrial SFP 1G LH 80km" },
|
||||||
|
{ pid: "BHF-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Beckhoff industrial SFP 1G copper RJ45" },
|
||||||
|
|
||||||
|
// ── 1G SFP BiDi ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BHF-SFP-GE-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1310", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", notes: "Beckhoff SFP 1G BiDi TX1310 single fiber" },
|
||||||
|
{ pid: "BHF-SFP-GE-BIDI-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1550", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310nm", notes: "Beckhoff SFP 1G BiDi TX1550 single fiber" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BHF-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Beckhoff industrial SFP+ 10G SR" },
|
||||||
|
{ pid: "BHF-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Beckhoff industrial SFP+ 10G LR" },
|
||||||
|
{ pid: "BHF-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Beckhoff industrial SFP+ 10G ER" },
|
||||||
|
|
||||||
|
// ── 1G CWDM SFP (factory/substation rings) ──────────────────────────────
|
||||||
|
{ pid: "BHF-SFP-CWDM-1470", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1470", fiberType: "SMF", connector: "LC", wavelengths: "1470nm", notes: "Beckhoff SFP CWDM 1470nm 40km" },
|
||||||
|
{ pid: "BHF-SFP-CWDM-1530", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1530", fiberType: "SMF", connector: "LC", wavelengths: "1530nm", notes: "Beckhoff SFP CWDM 1530nm 40km" },
|
||||||
|
{ pid: "BHF-SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", notes: "Beckhoff SFP CWDM 1550nm 40km" },
|
||||||
|
{ pid: "BHF-SFP-CWDM-1570", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1570", fiberType: "SMF", connector: "LC", wavelengths: "1570nm", notes: "Beckhoff SFP CWDM 1570nm 40km" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BHF-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Beckhoff QSFP+ 40G SR4 industrial" },
|
||||||
|
{ pid: "BHF-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "Beckhoff QSFP+ 40G LR4 industrial" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BHF-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Beckhoff SFP28 25G SR industrial" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeBeckhoffOem(): Promise<void> {
|
||||||
|
console.log("=== Beckhoff OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Beckhoff",
|
||||||
|
"oem",
|
||||||
|
"https://www.beckhoff.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of BECKHOFF_PIDS) {
|
||||||
|
const slug = `beckhoff-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Beckhoff OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${BECKHOFF_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeBeckhoffOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
121
packages/scraper/src/scrapers/belden-oem.ts
Normal file
121
packages/scraper/src/scrapers/belden-oem.ts
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
/**
|
||||||
|
* Belden OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Belden-branded SFP/SFP+/SFP28/QSFP+ modules
|
||||||
|
* used in Belden's industrial networking portfolio for manufacturing,
|
||||||
|
* energy, and transportation infrastructure.
|
||||||
|
*
|
||||||
|
* Note: hirschmann-oem.ts covers the Hirschmann sub-brand separately.
|
||||||
|
* This file covers Belden-branded transceivers only.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Belden SFP Modules product family (belden.com/optical-fiber/sfp-modules)
|
||||||
|
* - Belden Industrial Networking Solutions catalog
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/belden-oem.ts
|
||||||
|
* Cron: daily at 17:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface BeldenPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const BELDEN_PIDS: BeldenPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BLD-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Belden SFP 1G SX multimode industrial networking" },
|
||||||
|
{ pid: "BLD-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Belden SFP 1G LX single-mode 10km industrial" },
|
||||||
|
{ pid: "BLD-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "Belden SFP 1G ZX long-reach 80km" },
|
||||||
|
{ pid: "BLD-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Belden SFP 1G copper RJ45 industrial" },
|
||||||
|
|
||||||
|
// ── 1G SFP BiDi ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BLD-SFP-1G-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1310", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", notes: "Belden SFP 1G BiDi TX1310 single fiber 20km" },
|
||||||
|
{ pid: "BLD-SFP-1G-BIDI-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1550", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310nm", notes: "Belden SFP 1G BiDi TX1550 single fiber 20km" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BLD-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Belden SFP+ 10G SR multimode industrial" },
|
||||||
|
{ pid: "BLD-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Belden SFP+ 10G LR single-mode 10km" },
|
||||||
|
{ pid: "BLD-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Belden SFP+ 10G ER extended-reach 40km" },
|
||||||
|
{ pid: "BLD-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Belden SFP+ 10G ZR long-haul 80km" },
|
||||||
|
|
||||||
|
// ── 1G CWDM SFP (energy/transport rings) ────────────────────────────────
|
||||||
|
{ pid: "BLD-SFP-CWDM-1470", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1470", fiberType: "SMF", connector: "LC", wavelengths: "1470nm", notes: "Belden SFP CWDM 1470nm 40km energy/transport ring" },
|
||||||
|
{ pid: "BLD-SFP-CWDM-1530", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1530", fiberType: "SMF", connector: "LC", wavelengths: "1530nm", notes: "Belden SFP CWDM 1530nm 40km energy/transport ring" },
|
||||||
|
{ pid: "BLD-SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", notes: "Belden SFP CWDM 1550nm 40km energy/transport ring" },
|
||||||
|
{ pid: "BLD-SFP-CWDM-1610", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1610", fiberType: "SMF", connector: "LC", wavelengths: "1610nm", notes: "Belden SFP CWDM 1610nm 40km energy/transport ring" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BLD-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "Belden QSFP+ 40G LR4 industrial backbone" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "BLD-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Belden SFP28 25G LR industrial single-mode" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeBeldenOem(): Promise<void> {
|
||||||
|
console.log("=== Belden OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Belden",
|
||||||
|
"oem",
|
||||||
|
"https://www.belden.com/products/infrastructure/optical-fiber/sfp-modules",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of BELDEN_PIDS) {
|
||||||
|
const slug = `belden-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Belden OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${BELDEN_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeBeldenOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
127
packages/scraper/src/scrapers/blackbox-oem.ts
Normal file
127
packages/scraper/src/scrapers/blackbox-oem.ts
Normal file
@ -0,0 +1,127 @@
|
|||||||
|
/**
|
||||||
|
* Black Box Network Services OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Black Box-branded SFP/SFP+/SFP28/QSFP+/QSFP28/DAC transceiver PIDs
|
||||||
|
* for media converters, managed switches, and fiber infrastructure solutions.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Black Box Fiber Optic Transceivers product catalog (blackbox.com)
|
||||||
|
* - LFP/LSP/LEM/LQP/LPD series datasheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/blackbox-oem.ts
|
||||||
|
* Cron: daily at 16:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface BlackboxPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const BLACKBOX_PIDS: BlackboxPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LFP402", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Black Box 1G MM SFP 550m" },
|
||||||
|
{ pid: "LFP411", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "LFP421", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "Black Box 1G SM SFP 40km" },
|
||||||
|
{ pid: "LFP431", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "LFP451", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "LFP461", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550", notes: "Black Box 1G BiDi SFP upstream" },
|
||||||
|
{ pid: "LFP462", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310", notes: "Black Box 1G BiDi SFP downstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LSP401", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "LSP411", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "LSP421", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "LSP431", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "LSP441", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
{ pid: "LSP451", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1270/RX1330", notes: "Black Box 10G BiDi SFP+ upstream" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LEM401", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "LEM411", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "LEM421", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LQP401", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "LQP411", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LQP501", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "LQP511", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "LQP521", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "LQP531", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LPD401", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "LPD403", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "LPD501", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "LPD503", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeBlackboxOem(): Promise<void> {
|
||||||
|
console.log("=== Black Box Network Services OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Black Box",
|
||||||
|
"oem",
|
||||||
|
"https://www.blackbox.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of BLACKBOX_PIDS) {
|
||||||
|
const slug = `blackbox-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Black Box OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${BLACKBOX_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeBlackboxOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
129
packages/scraper/src/scrapers/broadcom-oem.ts
Normal file
129
packages/scraper/src/scrapers/broadcom-oem.ts
Normal file
@ -0,0 +1,129 @@
|
|||||||
|
/**
|
||||||
|
* Broadcom OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Broadcom-branded transceiver PIDs for Emulex HBAs (LPe-series),
|
||||||
|
* Brocade FC switches, and Ethernet adapters (NetXtreme / Liqid lineage).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Broadcom Emulex Gen 6/7 HBA Transceiver Guide (broadcom.com)
|
||||||
|
* - Brocade SAN Switch Optics Compatibility Matrix
|
||||||
|
* - Broadcom NetXtreme-E NIC Optics Reference
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/broadcom-oem.ts
|
||||||
|
* Cron: daily at 22:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface BroadcomPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const BROADCOM_PIDS: BroadcomPID[] = [
|
||||||
|
// ── FC HBA modules (Emulex LPe-series) ─────────────────────────────────
|
||||||
|
{ pid: "LPe16002-M6", formFactor: "SFP+", speedGbps: 16, speed: "16G", reachMeters: 100, reachLabel: "FC-SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", notes: "Emulex 16G FC dual-port HBA SFP+ module" },
|
||||||
|
{ pid: "LPe32002-M2", formFactor: "SFP28", speedGbps: 32, speed: "32G", reachMeters: 100, reachLabel: "FC-SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", notes: "Emulex 32G FC Gen 6 dual-port HBA SFP28 module" },
|
||||||
|
{ pid: "LPe35002-M2", formFactor: "SFP28", speedGbps: 32, speed: "32G", reachMeters: 100, reachLabel: "FC-SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", notes: "Emulex 32G FC Gen 7 dual-port HBA SFP28 module" },
|
||||||
|
|
||||||
|
// ── Fibre Channel SFP (Brocade / Broadcom SAN) ─────────────────────────
|
||||||
|
{ pid: "SFP-8G-FC-SW", formFactor: "SFP+", speedGbps: 8, speed: "8G", reachMeters: 150, reachLabel: "FC-SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "FC-PI-4", notes: "8G FC SW short-wave SFP" },
|
||||||
|
{ pid: "SFP-16G-FC-SW", formFactor: "SFP+", speedGbps: 16, speed: "16G", reachMeters: 100, reachLabel: "FC-SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "FC-PI-5", notes: "16G FC SW short-wave SFP" },
|
||||||
|
{ pid: "SFP-32G-FC-SW", formFactor: "SFP28", speedGbps: 32, speed: "32G", reachMeters: 100, reachLabel: "FC-SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "FC-PI-6", notes: "32G FC SW short-wave SFP28" },
|
||||||
|
{ pid: "SFP-64G-FC-SW", formFactor: "SFP56", speedGbps: 64, speed: "64G", reachMeters: 100, reachLabel: "FC-SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "FC-PI-7", notes: "64G FC SW short-wave SFP56" },
|
||||||
|
{ pid: "SFP-16G-FC-LW", formFactor: "SFP+", speedGbps: 16, speed: "16G", reachMeters: 10000, reachLabel: "FC-LW", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "FC-PI-5", notes: "16G FC LW long-wave SFP for inter-building SAN" },
|
||||||
|
{ pid: "SFP-32G-FC-LW", formFactor: "SFP28", speedGbps: 32, speed: "32G", reachMeters: 10000, reachLabel: "FC-LW", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "FC-PI-6", notes: "32G FC LW long-wave SFP28 for SAN extension" },
|
||||||
|
|
||||||
|
// ── 1G SFP (NetXtreme / Brocade access) ────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP+-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP+-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "10G ZR 80km for long-haul SAN extension" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD (NetXtreme-E P225P / BCM57508) ─────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeBroadcomOem(): Promise<void> {
|
||||||
|
console.log("=== Broadcom OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Broadcom",
|
||||||
|
"oem",
|
||||||
|
"https://www.broadcom.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of BROADCOM_PIDS) {
|
||||||
|
const slug = `broadcom-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
is_oem_seed = true,
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Broadcom OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${BROADCOM_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeBroadcomOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
138
packages/scraper/src/scrapers/brocade-oem.ts
Normal file
138
packages/scraper/src/scrapers/brocade-oem.ts
Normal file
@ -0,0 +1,138 @@
|
|||||||
|
/**
|
||||||
|
* Brocade OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Brocade-branded transceiver PIDs for Fibre Channel and IP networking.
|
||||||
|
* Brocade's networking division is now part of Broadcom/Avago, but transceivers
|
||||||
|
* are still sold under the Brocade brand (XBR- prefix) for FC SANs and Ethernet.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Broadcom Fibre Channel Networking product pages (broadcom.com/products/fibre-channel-networking)
|
||||||
|
* - Brocade SFP/QSFP Transceiver Hardware Reference
|
||||||
|
* - Brocade Switch Compatibility Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/brocade-oem.ts
|
||||||
|
* Cron: daily at 12:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface BrocadePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const BROCADE_PIDS: BrocadePID[] = [
|
||||||
|
// ── 1G SFP (Ethernet) ───────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-000099", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "XBR-000100", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "XBR-000102", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "XBR-000143", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 8G FC SFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-SFP8G0LR", formFactor: "SFP+", speedGbps: 8, speed: "8GFC", reachMeters: 10000, reachLabel: "LW", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "8GFC-LW" },
|
||||||
|
{ pid: "XBR-SFP8G0SR", formFactor: "SFP+", speedGbps: 8, speed: "8GFC", reachMeters: 500, reachLabel: "SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "8GFC-SW" },
|
||||||
|
|
||||||
|
// ── 16G FC SFP+ ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-SFP16GW", formFactor: "SFP+", speedGbps: 16, speed: "16GFC", reachMeters: 300, reachLabel: "SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "16GFC-SW" },
|
||||||
|
{ pid: "XBR-SFP16GLR", formFactor: "SFP+", speedGbps: 16, speed: "16GFC", reachMeters: 10000, reachLabel: "LW", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "16GFC-LW" },
|
||||||
|
{ pid: "XBR-SFP16GER", formFactor: "SFP+", speedGbps: 16, speed: "16GFC", reachMeters: 40000, reachLabel: "EW", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "16GFC-EW" },
|
||||||
|
|
||||||
|
// ── 32G FC SFP+ ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-SFP32GW", formFactor: "SFP+", speedGbps: 32, speed: "32GFC", reachMeters: 100, reachLabel: "SW", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "32GFC-SW" },
|
||||||
|
{ pid: "XBR-SFP32GLR", formFactor: "SFP+", speedGbps: 32, speed: "32GFC", reachMeters: 10000, reachLabel: "LW", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "32GFC-LW" },
|
||||||
|
{ pid: "XBR-SFP32GER", formFactor: "SFP+", speedGbps: 32, speed: "32GFC", reachMeters: 40000, reachLabel: "EW", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "32GFC-EW" },
|
||||||
|
|
||||||
|
// ── 10G Ethernet SFP+ ───────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-10G0SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "XBR-10G0LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "XBR-10G0ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-25G0SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "XBR-25G0LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-40G0SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "XBR-40G0LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-100G0SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "XBR-100G0LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "XBR-100G0CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── 128G FC QSFP28 ──────────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-QSFP28-128GW", formFactor: "QSFP28", speedGbps: 128, speed: "128GFC", reachMeters: 100, reachLabel: "SW", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "128GFC-SW" },
|
||||||
|
{ pid: "XBR-QSFP28-128GLR", formFactor: "QSFP28", speedGbps: 128, speed: "128GFC", reachMeters: 2000, reachLabel: "LW", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "128GFC-LW" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XBR-TWX-0101", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "XBR-TWX-0301", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "XBR-TWX-0107", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeBrocadeOem(): Promise<void> {
|
||||||
|
console.log("=== Brocade OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Brocade",
|
||||||
|
"oem",
|
||||||
|
"https://www.broadcom.com/products/fibre-channel-networking",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of BROCADE_PIDS) {
|
||||||
|
const slug = `brocade-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Brocade OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${BROCADE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeBrocadeOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
127
packages/scraper/src/scrapers/calix-access-oem.ts
Normal file
127
packages/scraper/src/scrapers/calix-access-oem.ts
Normal file
@ -0,0 +1,127 @@
|
|||||||
|
/**
|
||||||
|
* Calix Access OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Calix-branded transceiver PIDs for AXOS, GigaSPEED, and
|
||||||
|
* Intelligent Access EDGE platforms with a focus on PON/access optics
|
||||||
|
* (GPON, XGS-PON, EPON, BiDi) not covered in the base calix-oem.ts seed.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Calix Support Cloud Optics Compatibility Guide (calix.com)
|
||||||
|
* - Calix AXOS E9-2 / E7-2 / GigaSPEED Hardware Reference
|
||||||
|
* - ITU-T G.984/G.9807/G.987 PON specifications
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/calix-access-oem.ts
|
||||||
|
* Cron: daily at 22:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CalixAccessPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
isTelecom?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Calix Access OEM transceiver catalog ────────────────────────────────────
|
||||||
|
// Focus: PON OLT/ONU optics + access-edge SFP variants for AXOS platforms
|
||||||
|
const CALIX_ACCESS_PIDS: CalixAccessPID[] = [
|
||||||
|
// ── 1G SFP (access uplinks) ──────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "SFP-BIDI-1310-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1490nm", notes: "1G BiDi single-fiber upstream 1310nm / downstream 1490nm for AXOS access" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ (access uplinks) ────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-LR-S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR-S", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "10G LR industrial temp -40 to +85°C for outdoor AXOS pods" },
|
||||||
|
{ pid: "SFP-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-SR4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-100G-LR4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── PON OLT optics (Telecom) ─────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GPON-OLT", formFactor: "SFP", speedGbps: 2.5, speed: "2.5G", reachMeters: 20000, reachLabel: "GPON-OLT", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "ITU-T G.984", notes: "GPON OLT SFP Class B+; DS 1490nm 2.488Gbps / US 1310nm 1.244Gbps", isTelecom: true },
|
||||||
|
{ pid: "SFP-XGSPON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 20000, reachLabel: "XGSPON-OLT", fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "ITU-T G.9807", notes: "XGS-PON OLT SFP+ Class N1+; DS 1577nm 9.953Gbps / US 1270nm 9.953Gbps", isTelecom: true },
|
||||||
|
{ pid: "SFP-EPON-OLT", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "EPON-OLT", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "IEEE 802.3ah", notes: "EPON OLT SFP PX20+; DS 1490nm 1Gbps / US 1310nm 1Gbps", isTelecom: true },
|
||||||
|
{ pid: "SFP-GPON-ONU", formFactor: "SFP", speedGbps: 2.5, speed: "2.5G", reachMeters: 20000, reachLabel: "GPON-ONU", fiberType: "SMF", connector: "SC", wavelengths: "1310/1490nm", standard: "ITU-T G.984", notes: "GPON ONU SFP Class B+; US 1310nm 1.244Gbps / DS 1490nm 2.488Gbps", isTelecom: true },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCalixAccessOem(): Promise<void> {
|
||||||
|
console.log("=== Calix Access OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Calix",
|
||||||
|
"oem",
|
||||||
|
"https://www.calix.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CALIX_ACCESS_PIDS) {
|
||||||
|
const slug = `calix-access-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.isTelecom ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
is_oem_seed = true,
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Calix Access OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CALIX_ACCESS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCalixAccessOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
140
packages/scraper/src/scrapers/calix-gigapoint-oem.ts
Normal file
140
packages/scraper/src/scrapers/calix-gigapoint-oem.ts
Normal file
@ -0,0 +1,140 @@
|
|||||||
|
/**
|
||||||
|
* Calix GigaPoint OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Calix GigaPoint / Revenue EDGE-branded transceiver PIDs for
|
||||||
|
* Calix E-Series and AXOS OLT platforms supporting GPON, XGS-PON,
|
||||||
|
* and 10G-EPON broadband access deployments.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Calix Broadband Cloud Access Systems (calix.com)
|
||||||
|
* - Calix E7-2 / AXOS E9-2 Hardware Reference Guides
|
||||||
|
* - Calix GigaPoint Optical Module Data Sheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/calix-gigapoint-oem.ts
|
||||||
|
* Cron: daily at 09:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CalixGigapointPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
isTelecom?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
const TELECOM_PIDS: Set<string> = new Set([
|
||||||
|
"CGP-GPON-OLT-B",
|
||||||
|
"CGP-GPON-OLT-C",
|
||||||
|
"CGP-XGS-PON-OLT",
|
||||||
|
"CGP-XGS-PON-ONU",
|
||||||
|
"CGP-EPON-OLT",
|
||||||
|
"CGP-10G-EPON-OLT",
|
||||||
|
"CGP-SFP-CWDM-1490",
|
||||||
|
"CGP-SFP-CWDM-1550",
|
||||||
|
"CGP-SFP-BIDI-1310",
|
||||||
|
"CGP-SFP-BIDI-1490",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const CALIX_GIGAPOINT_PIDS: CalixGigapointPID[] = [
|
||||||
|
// ── GPON OLT SFP ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-GPON-OLT-B", formFactor: "SFP", speedGbps: 2.488, speed: "GPON", reachMeters: 20000, reachLabel: "GPON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "G.984", notes: "Calix GigaPoint GPON OLT SFP Class B+, E7-2/AXOS platform", isTelecom: true },
|
||||||
|
{ pid: "CGP-GPON-OLT-C", formFactor: "SFP", speedGbps: 2.488, speed: "GPON", reachMeters: 20000, reachLabel: "GPON-C+", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "G.984", notes: "Calix GigaPoint GPON OLT SFP Class C+, extended reach", isTelecom: true },
|
||||||
|
|
||||||
|
// ── XGS-PON OLT / ONU SFP+ ───────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-XGS-PON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "XGS-PON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "G.9807.1", notes: "Calix GigaPoint XGS-PON OLT SFP+ for E9-2", isTelecom: true },
|
||||||
|
{ pid: "CGP-XGS-PON-ONU", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "XGS-ONU-20K", fiberType: "SMF", connector: "SC", wavelengths: "1270/1577nm", standard: "G.9807.1", notes: "Calix GigaPoint XGS-PON ONU SFP+ 1270nm TX", isTelecom: true },
|
||||||
|
|
||||||
|
// ── EPON OLT SFP ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-EPON-OLT", formFactor: "SFP", speedGbps: 1, speed: "EPON", reachMeters: 20000, reachLabel: "EPON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "802.3ah", notes: "Calix GigaPoint EPON OLT SFP 1G/1G", isTelecom: true },
|
||||||
|
{ pid: "CGP-10G-EPON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "10G-EPON", reachMeters: 20000, reachLabel: "10G-EPON-20K",fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "802.3av", notes: "Calix GigaPoint 10G-EPON OLT SFP+", isTelecom: true },
|
||||||
|
|
||||||
|
// ── Standard 1G SFP ──────────────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "CGP-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
|
||||||
|
// ── Standard 10G SFP+ ────────────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "CGP-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
|
||||||
|
// ── CWDM SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-SFP-CWDM-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1490", fiberType: "SMF", connector: "LC", wavelengths: "1490nm", standard: "CWDM", notes: "Calix GigaPoint CWDM SFP 1490nm", isTelecom: true },
|
||||||
|
{ pid: "CGP-SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "CWDM", notes: "Calix GigaPoint CWDM SFP 1550nm", isTelecom: true },
|
||||||
|
|
||||||
|
// ── BiDi SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-SFP-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-1310", fiberType: "SMF", connector: "SC", wavelengths: "1310/1490nm", notes: "Calix GigaPoint 1G BiDi SFP 1310nm TX / 1490nm RX", isTelecom: true },
|
||||||
|
{ pid: "CGP-SFP-BIDI-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-1490", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", notes: "Calix GigaPoint 1G BiDi SFP 1490nm TX / 1310nm RX", isTelecom: true },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CGP-QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCalixGigapointOem(): Promise<void> {
|
||||||
|
console.log("=== Calix GigaPoint OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Calix GigaPoint",
|
||||||
|
"oem",
|
||||||
|
"https://www.calix.com/calix-platforms/calix-broadband-cloud-access-systems.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CALIX_GIGAPOINT_PIDS) {
|
||||||
|
const slug = `calix-gp-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Calix GigaPoint OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CALIX_GIGAPOINT_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCalixGigapointOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
119
packages/scraper/src/scrapers/calix-oem.ts
Normal file
119
packages/scraper/src/scrapers/calix-oem.ts
Normal file
@ -0,0 +1,119 @@
|
|||||||
|
/**
|
||||||
|
* Calix OEM Transceiver Catalog Seed
|
||||||
|
* Seeds Calix-branded transceiver PIDs for E-Series, GigaSPEED,
|
||||||
|
* AXOS, and Intelligent Access EDGE platforms.
|
||||||
|
* Sources: Calix Support Cloud / Hardware Compatibility Guide (calix.com)
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/calix-oem.ts
|
||||||
|
* Cron: daily at 08:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CalixPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Calix OEM transceiver catalog ───────────────────────────────────────────
|
||||||
|
// Source: Calix Support Cloud / Hardware Compatibility Guide (calix.com)
|
||||||
|
const CALIX_PIDS: CalixPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "100-02013", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "100-02014", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "100-02015", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "100-02016", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "100-02017", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH40", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "Calix 1G SM 40km" },
|
||||||
|
{ pid: "100-02018", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1550nm", notes: "Calix 1G BiDi single fiber" },
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "100-04025", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "100-04026", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "100-04027", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "100-04028", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "100-04029", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
{ pid: "100-04030", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1270/1330nm", notes: "Calix 10G BiDi SFP+" },
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "100-04050", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "100-04051", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "100-04052", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "100-04060", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "100-04061", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "100-04070", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "100-04071", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "100-04072", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "100-04073", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
{ pid: "100-04074", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "FR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-FR" },
|
||||||
|
// ── DAC cables ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "100-03901", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "100-03903", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "100-03911", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "100-03913", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCalixOem(): Promise<void> {
|
||||||
|
console.log("=== Calix OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const calixVendorId = await ensureVendor(
|
||||||
|
"Calix",
|
||||||
|
"oem",
|
||||||
|
"https://www.calix.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CALIX_PIDS) {
|
||||||
|
const slug = `calix-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, calixVendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Calix OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CALIX_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCalixOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
121
packages/scraper/src/scrapers/cambium-oem.ts
Normal file
121
packages/scraper/src/scrapers/cambium-oem.ts
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
/**
|
||||||
|
* Cambium Networks OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Cambium Networks branded optics used in
|
||||||
|
* cnMatrix EX series enterprise switches and PTP 820 platforms.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Cambium Networks cnMatrix transceiver compatibility guide (cambiumnetworks.com)
|
||||||
|
* - Cambium EX2052R/EX2082R hardware installation guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cambium-oem.ts
|
||||||
|
* Cron: daily at 11:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CambiumPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CAMBIUM_PIDS: CambiumPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "C000065L001A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "C000065L002A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "C000065L003A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "C000065L004A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "C000065L005A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1550nm", notes: "cnMatrix 1G BiDi SFP" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "C000065L010A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "C000065L011A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "C000065L012A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "C000065L013A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
{ pid: "C000065L014A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1270/1330nm" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "C000065L020A", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "C000065L021A", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "C000065L030A", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "C000065L031A", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "C000065L040A", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "C000065L041A", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "C000065L042A", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "C000065L050A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "C000065L051A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "C000065L060A", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCambiumOem(): Promise<void> {
|
||||||
|
console.log("=== Cambium Networks OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Cambium Networks",
|
||||||
|
"oem",
|
||||||
|
"https://www.cambiumnetworks.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CAMBIUM_PIDS) {
|
||||||
|
const slug = `cambium-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Cambium Networks OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CAMBIUM_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCambiumOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
126
packages/scraper/src/scrapers/casa-systems-oem.ts
Normal file
126
packages/scraper/src/scrapers/casa-systems-oem.ts
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
/**
|
||||||
|
* Casa Systems OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Casa Systems-branded transceiver PIDs for CMTS/CCAP platforms
|
||||||
|
* (APEX HFC, C100G, C40G) used in cable access and broadband networks.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Casa Systems Product Portfolio (casa-systems.com)
|
||||||
|
* - C100G CCAP Platform Hardware Guide
|
||||||
|
* - APEX HFC Platform Line Card Compatibility Matrix
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/casa-systems-oem.ts
|
||||||
|
* Cron: daily at 21:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CasaPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CASA_PIDS: CasaPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CASA-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "CASA-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "CASA-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Casa Systems 1G copper SFP" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CASA-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "CASA-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "CASA-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "CASA-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Casa Systems 10G ZR for CCAP uplink" },
|
||||||
|
{ pid: "CASA-SFP-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T", notes: "Casa Systems 10GBASE-T copper SFP+" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CASA-SFP-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "CASA-SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CASA-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "CASA-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CASA-QSFP-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "CASA-QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "CASA-QSFP-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "Casa Systems 100G ER4 for CCAP long-reach uplink" },
|
||||||
|
{ pid: "CASA-QSFP-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CASA-QSFPDD-400G-DR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Casa Systems 400G DR4 next-gen CCAP uplink" },
|
||||||
|
{ pid: "CASA-QSFPDD-400G-FR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CASA-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+", notes: "Casa Systems 10G DAC 1m" },
|
||||||
|
{ pid: "CASA-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+", notes: "Casa Systems 10G DAC 3m" },
|
||||||
|
{ pid: "CASA-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28", notes: "Casa Systems 100G DAC 1m" },
|
||||||
|
{ pid: "CASA-DAC-100G-3M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28", notes: "Casa Systems 100G DAC 3m" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCasaSystemsOem(): Promise<void> {
|
||||||
|
console.log("=== Casa Systems OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Casa Systems",
|
||||||
|
"oem",
|
||||||
|
"https://www.casa-systems.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CASA_PIDS) {
|
||||||
|
const slug = `casa-systems-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Casa Systems OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CASA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCasaSystemsOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
118
packages/scraper/src/scrapers/centec-oem.ts
Normal file
118
packages/scraper/src/scrapers/centec-oem.ts
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
/**
|
||||||
|
* Centec Networks OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Centec-branded transceiver PIDs for V580/V350/V330-series
|
||||||
|
* white-box Ethernet switches built on Centec Ethernet switching ASICs.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Centec Networks Optical Transceiver Data Sheet (centecnetworks.com)
|
||||||
|
* - Centec V580/V350 Hardware Compatibility Guide
|
||||||
|
* - Centec Open Networking Transceiver Module Specifications
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/centec-oem.ts
|
||||||
|
* Cron: daily at 11:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CentecPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CENTEC_PIDS: CentecPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CT-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Centec 1G SFP SX 550m MMF" },
|
||||||
|
{ pid: "CT-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Centec 1G SFP LX 10km SMF" },
|
||||||
|
{ pid: "CT-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "Centec 1G SFP ZX 80km SMF" },
|
||||||
|
{ pid: "CT-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Centec 1G SFP copper RJ45" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CT-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Centec 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "CT-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Centec 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "CT-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Centec 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "CT-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Centec 10G SFP+ ZR 80km SMF" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CT-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Centec 25G SFP28 SR 100m MMF" },
|
||||||
|
{ pid: "CT-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Centec 25G SFP28 LR 10km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CT-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Centec 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "CT-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "Centec 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CT-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Centec 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "CT-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Centec 100G QSFP28 LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CT-QSFP-DD-400G-DR4",formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Centec 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "CT-QSFP-DD-400G-SR8",formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "Centec 400G QSFP-DD SR8 100m MMF" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCentecOem(): Promise<void> {
|
||||||
|
console.log("=== Centec Networks OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Centec Networks",
|
||||||
|
"oem",
|
||||||
|
"https://www.centecnetworks.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CENTEC_PIDS) {
|
||||||
|
const slug = `centec-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Centec Networks OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CENTEC_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCentecOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
123
packages/scraper/src/scrapers/ceragon-oem.ts
Normal file
123
packages/scraper/src/scrapers/ceragon-oem.ts
Normal file
@ -0,0 +1,123 @@
|
|||||||
|
/**
|
||||||
|
* Ceragon Networks OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Ceragon-branded transceiver PIDs used in microwave and millimeter-wave
|
||||||
|
* wireless backhaul platforms (FibeAir IP-20, IP-50, IP-10). Ceragon is a
|
||||||
|
* Nasdaq-listed vendor (CRNT) and parent of Siklu.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Ceragon Networks Product Catalog (ceragon.com)
|
||||||
|
* - FibeAir IP-20/IP-50 hardware and interface guides
|
||||||
|
* - Ceragon SFP/QSFP optical compatibility matrices
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ceragon-oem.ts
|
||||||
|
* Cron: daily at 03:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CeragonPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CERAGON_PIDS: CeragonPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Ceragon 1G SFP SX for FibeAir IP-20 access nodes" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Ceragon 10G SFP+ SR; FibeAir IP-50 Ethernet uplink" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Ceragon 10G ZR for long-haul microwave site uplinks" },
|
||||||
|
|
||||||
|
// ── BiDi SFP ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GE-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BIDI", fiberType: "SMF", connector: "LC", wavelengths: "1310/1550nm", standard: "1000BASE-BX", notes: "BiDi SFP for single-fiber backhaul; Tx1310/Rx1550" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Ceragon 25G SFP28 for 5G fronthaul and aggregation" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "Ceragon 40G QSFP+ LR4 for FibeAir IP-50 hub uplink" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Ceragon 100G QSFP28 LR4 for high-capacity backhaul rings" },
|
||||||
|
|
||||||
|
// ── 1G Copper ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Copper GE SFP for management interface" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Ceragon 400G QSFP-DD for next-gen microwave core aggregation" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCeragonOem(): Promise<void> {
|
||||||
|
console.log("=== Ceragon Networks OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Ceragon Networks",
|
||||||
|
"oem",
|
||||||
|
"https://www.ceragon.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CERAGON_PIDS) {
|
||||||
|
const slug = `ceragon-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Ceragon OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CERAGON_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCeragonOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
133
packages/scraper/src/scrapers/checkpoint-oem.ts
Normal file
133
packages/scraper/src/scrapers/checkpoint-oem.ts
Normal file
@ -0,0 +1,133 @@
|
|||||||
|
/**
|
||||||
|
* Check Point OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Check Point-branded transceiver PIDs for Quantum Maestro,
|
||||||
|
* Quantum Spark, and 6000/7000/16000/26000 series security gateways.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Check Point Hardware Compatibility Guide (support.checkpoint.com)
|
||||||
|
* - Quantum Maestro Hyperscale Hardware Reference
|
||||||
|
* - Check Point 16000/26000 Appliance Hardware Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/checkpoint-oem.ts
|
||||||
|
* Cron: daily at 09:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CheckpointPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CHECKPOINT_PIDS: CheckpointPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CPAC-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "CPAC-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "CPAC-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "CPAC-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CPAC-SFP+-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "CPAC-SFP+-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "CPAC-SFP+-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "CPAC-SFP+-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "CPAC-SFP+-10G-LRM", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM" },
|
||||||
|
{ pid: "CPAC-SFP+-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CPAC-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "CPAC-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "CPAC-SFP28-25G-ER", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CPAC-QSFP+-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "CPAC-QSFP+-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CPAC-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "CPAC-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "CPAC-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "CPAC-QSFP28-100G-DR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
{ pid: "CPAC-QSFP28-100G-FR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "FR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-FR" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD (Quantum Maestro) ──────────────────────────────────────
|
||||||
|
{ pid: "CPAC-QSFPDD-400G-SR8", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "CPAC-QSFPDD-400G-DR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "CPAC-QSFPDD-400G-FR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "CPAC-QSFPDD-400G-LR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CPAC-CABLE-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "CPAC-CABLE-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "CPAC-CABLE-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "CPAC-CABLE-DAC-100G-3M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "CPAC-CABLE-DAC-400G-1M", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP-DD" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCheckpointOem(): Promise<void> {
|
||||||
|
console.log("=== Check Point OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Check Point",
|
||||||
|
"oem",
|
||||||
|
"https://www.checkpoint.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CHECKPOINT_PIDS) {
|
||||||
|
const slug = `checkpoint-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Check Point OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CHECKPOINT_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCheckpointOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
141
packages/scraper/src/scrapers/ciena-oem.ts
Normal file
141
packages/scraper/src/scrapers/ciena-oem.ts
Normal file
@ -0,0 +1,141 @@
|
|||||||
|
/**
|
||||||
|
* Ciena OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Ciena-branded transceiver PIDs for Waveserver, 6500/6550
|
||||||
|
* packet-optical platforms, and 3000/5000-series switches.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Ciena Optics Guide (ciena.com)
|
||||||
|
* - Ciena 6500 Packet-Optical Platform Hardware Guide
|
||||||
|
* - Ciena Waveserver AI hardware reference
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ciena-oem.ts
|
||||||
|
* Cron: daily at 06:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CienaPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CIENA_PIDS: CienaPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-A00G85", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "XCVR-A00G31", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "XCVR-A00G55", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "XCVR-A00T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-B10G85", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "XCVR-B10G31", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "XCVR-B10G55", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "XCVR-B10G55Z", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── DWDM SFP+ (Waveserver / 6500 platform) ──────────────────────────────
|
||||||
|
{ pid: "XCVR-B10U3131", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "10G DWDM tunable SFP+" },
|
||||||
|
{ pid: "XCVR-B10U31DT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM-T",fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "10G DWDM tunable extended temp" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-C25G85", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "XCVR-C25G31", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "XCVR-C25G31E", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-D40G85M4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "XCVR-D40G31L4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
{ pid: "XCVR-D40G55E4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-E100G85M4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "XCVR-E100G31L4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "XCVR-E100G31CW4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "XCVR-E100G31DR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
{ pid: "XCVR-E100G31FR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "FR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-FR" },
|
||||||
|
|
||||||
|
// ── 100G Coherent CFP2 (Waveserver 5 / 6500) ────────────────────────────
|
||||||
|
{ pid: "XCVR-E100GCFP2", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 2000000,reachLabel: "ZR+", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "100G coherent CFP2 long-haul" },
|
||||||
|
{ pid: "XCVR-E100GCFP2DT", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 2000000,reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "100G coherent CFP2 DWDM tunable" },
|
||||||
|
|
||||||
|
// ── 400G coherent QSFP-DD / OSFP (Waveserver Ai) ────────────────────────
|
||||||
|
{ pid: "XCVR-F400GQD-ZR", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", notes: "400G coherent QSFP-DD" },
|
||||||
|
{ pid: "XCVR-F400GQD-SR8", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "XCVR-F400GQD-DR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "XCVR-F400GQD-FR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "XCVR-F400GQD-LR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M",fiberType: "DAC", connector: "SFP+", notes: "10G DAC 1m" },
|
||||||
|
{ pid: "XCVR-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M",fiberType: "DAC", connector: "SFP+", notes: "10G DAC 3m" },
|
||||||
|
{ pid: "XCVR-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M",fiberType: "DAC", connector: "QSFP28", notes: "100G DAC 1m" },
|
||||||
|
{ pid: "XCVR-DAC-100G-3M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M",fiberType: "DAC", connector: "QSFP28", notes: "100G DAC 3m" },
|
||||||
|
{ pid: "XCVR-DAC-400G-1M", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 1, reachLabel: "DAC-1M",fiberType: "DAC", connector: "QSFP-DD",notes: "400G DAC 1m" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCienaOem(): Promise<void> {
|
||||||
|
console.log("=== Ciena OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Ciena",
|
||||||
|
"oem",
|
||||||
|
"https://www.ciena.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CIENA_PIDS) {
|
||||||
|
const slug = `ciena-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Ciena OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CIENA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCienaOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
150
packages/scraper/src/scrapers/ciena-waveserver-oem.ts
Normal file
150
packages/scraper/src/scrapers/ciena-waveserver-oem.ts
Normal file
@ -0,0 +1,150 @@
|
|||||||
|
/**
|
||||||
|
* Ciena WaveServer / WaveLogic OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Ciena-branded transceiver and coherent module PIDs for WaveServer,
|
||||||
|
* WaveLogic 5e/5n open line system platforms and packet-optical equipment.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Ciena WaveServer 5 Product Overview (ciena.com)
|
||||||
|
* - Ciena WaveLogic 5e/5n Module Datasheet
|
||||||
|
* - Ciena Client Optics Portfolio Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ciena-waveserver-oem.ts
|
||||||
|
* Cron: daily at 23:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CienaWaveserverPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CIENA_WAVESERVER_PIDS: CienaWaveserverPID[] = [
|
||||||
|
// ── 1G SFP Client ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 10G SONET/SDH ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "OC48-SFP", formFactor: "SFP", speedGbps: 2.5, speed: "OC48", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", category: "Telecom", notes: "Ciena OC-48/STM-16 2.488G SFP" },
|
||||||
|
{ pid: "OC192-SFP", formFactor: "SFP+", speedGbps: 10, speed: "OC192",reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", category: "Telecom", notes: "Ciena OC-192/STM-64 9.953G SFP+" },
|
||||||
|
|
||||||
|
// ── 10G OTN ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-OTU2-10G", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", category: "Telecom", notes: "Ciena OTU2 10.709G SFP+" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DWDM-SFP10G-C-xx.x", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band ITU DWDM", category: "Telecom", notes: "Ciena DWDM C-band 10G SFP+ (wavelength per order)" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-S10V", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Ciena WaveServer client SFP+ 10G" },
|
||||||
|
{ pid: "XCVR-S28V", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Ciena WaveServer client SFP28 25G" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-A00AQ4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Ciena WaveServer QSFP28 100G SR4" },
|
||||||
|
{ pid: "XCVR-A00BQ4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Ciena WaveServer QSFP28 100G LR4" },
|
||||||
|
{ pid: "QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 Coherent ───────────────────────────────────────────────────
|
||||||
|
{ pid: "CFP2-ACO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1000000, reachLabel: "ACO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Ciena 100G CFP2 Analogue Coherent Optics" },
|
||||||
|
{ pid: "CFP2-DCO-200G", formFactor: "CFP2", speedGbps: 200, speed: "200G", reachMeters: 1500000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Ciena 200G CFP2 Digital Coherent Optics" },
|
||||||
|
|
||||||
|
// ── 100G CFP OTN ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CFP-OTU4-100G", formFactor: "CFP", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", category: "Telecom", notes: "Ciena OTU4 100G CFP LR4" },
|
||||||
|
|
||||||
|
// ── WaveLogic 5 Coherent Line Modules ────────────────────────────────────
|
||||||
|
{ pid: "WL5e-SEQ", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 3000000, reachLabel: "WL5e", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Ciena WaveLogic 5e coherent 400G-800G QSFP-DD" },
|
||||||
|
{ pid: "WL5n-SEQ", formFactor: "QSFP-DD", speedGbps: 200, speed: "200G", reachMeters: 3000000, reachLabel: "WL5n", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Ciena WaveLogic 5 nano coherent QSFP-DD" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD Client ──────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-B00DPP", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Ciena WaveServer QSFP-DD 400G DR4" },
|
||||||
|
{ pid: "QSFP-DD-ZR-400G", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", standard: "400ZR", notes: "Ciena 400G ZR coherent QSFP-DD" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (coherent/OTN/SONET/DWDM)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"OC48-SFP",
|
||||||
|
"OC192-SFP",
|
||||||
|
"SFP-OTU2-10G",
|
||||||
|
"DWDM-SFP10G-C-xx.x",
|
||||||
|
"CFP2-ACO-100G",
|
||||||
|
"CFP2-DCO-200G",
|
||||||
|
"CFP-OTU4-100G",
|
||||||
|
"WL5e-SEQ",
|
||||||
|
"WL5n-SEQ",
|
||||||
|
"QSFP-DD-ZR-400G",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeCienaWaveserverOem(): Promise<void> {
|
||||||
|
console.log("=== Ciena WaveServer/WaveLogic OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Ciena",
|
||||||
|
"oem",
|
||||||
|
"https://www.ciena.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CIENA_WAVESERVER_PIDS) {
|
||||||
|
const slug = `ciena-ws-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Ciena WaveServer/WaveLogic OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CIENA_WAVESERVER_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCienaWaveserverOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
138
packages/scraper/src/scrapers/cimc-oem.ts
Normal file
138
packages/scraper/src/scrapers/cimc-oem.ts
Normal file
@ -0,0 +1,138 @@
|
|||||||
|
/**
|
||||||
|
* CIMC Semiconductors (legacy II-VI / Finisar integration-era, now Coherent Corp) OEM Seed
|
||||||
|
*
|
||||||
|
* Seeds CIMC-branded transceiver PIDs covering the legacy II-VI/Finisar product
|
||||||
|
* lines that Coherent Corp continues to manufacture and sell under the CIMC
|
||||||
|
* Semiconductors brand. Includes datacenter optics, telecom/OTN, CFP2-DCO,
|
||||||
|
* DWDM tunable, and CWDM variants from the II-VI/Finisar integration era.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Coherent Corp (formerly II-VI) product catalog (coherent.com)
|
||||||
|
* - CIMC Semiconductors transceiver datasheets
|
||||||
|
* - Legacy II-VI/Finisar product numbering conventions
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cimc-oem.ts
|
||||||
|
* Cron: daily at 05:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CimcPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that fall into Telecom/carrier category rather than DataCenter
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"CIMC-CFP2-DCO-200G",
|
||||||
|
"CIMC-QSFP-DD-ZR-400G",
|
||||||
|
"CIMC-SFP-OTN-10G",
|
||||||
|
"CIMC-SFP-CWDM-1550",
|
||||||
|
"CIMC-DWDM-SFP-C",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const CIMC_PIDS: CimcPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CIMC-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "CIMC 1G MM SFP — legacy Finisar FTLF8524P3BNV lineage" },
|
||||||
|
{ pid: "CIMC-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "CIMC 1G SM SFP — legacy Finisar FTLF1324P2BTL lineage" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CIMC-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "CIMC 10G SR SFP+ — legacy Finisar FTLX8571D3BCL lineage" },
|
||||||
|
{ pid: "CIMC-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "CIMC 10G LR SFP+ — legacy Finisar FTLX1475D3BCL lineage" },
|
||||||
|
{ pid: "CIMC-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "CIMC-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "CIMC 10G ZR 80km — ex-II-VI long-haul series" },
|
||||||
|
|
||||||
|
// ── 10G OTN / CWDM / DWDM (Telecom) ────────────────────────────────────
|
||||||
|
{ pid: "CIMC-SFP-OTN-10G", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "OTN", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OTU2", category: "Telecom", notes: "CIMC 10G OTN SFP+ for OTU2 transport — ex-II-VI Finisar OTN lineage" },
|
||||||
|
{ pid: "CIMC-SFP-CWDM-1550", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "CWDM", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "CWDM", category: "Telecom", notes: "CIMC 10G CWDM SFP+ 1550nm — legacy Finisar CWDM channel optic" },
|
||||||
|
{ pid: "CIMC-DWDM-SFP-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band ITU", standard: "DWDM ITU-T G.694.1",category: "Telecom", notes: "CIMC 10G DWDM tunable SFP+ C-band — ex-II-VI Finisar DWDM series" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CIMC-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "CIMC 25G SR SFP28 — ex-II-VI high-volume hyperscale series" },
|
||||||
|
{ pid: "CIMC-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CIMC-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "CIMC 100G SR4 QSFP28 — ex-II-VI Finisar FTLC9551REPM lineage" },
|
||||||
|
{ pid: "CIMC-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "CIMC-QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "CIMC 100G ER4 40km QSFP28 — ex-II-VI extended-reach datacenter" },
|
||||||
|
|
||||||
|
// ── 200G CFP2-DCO (Telecom) ─────────────────────────────────────────────
|
||||||
|
{ pid: "CIMC-CFP2-DCO-200G", formFactor: "CFP2", speedGbps: 200, speed: "200G", reachMeters: 1000000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF CFP2-DCO", category: "Telecom", notes: "CIMC CFP2-DCO 200G coherent — ex-II-VI Finisar DCO lineage, 1000km EDFA reach" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CIMC-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "CIMC 400G DR4 QSFP-DD — Coherent Corp next-gen datacenter optic" },
|
||||||
|
{ pid: "CIMC-QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "CIMC 400G SR8 QSFP-DD MPO for short-reach hyperscale fabric" },
|
||||||
|
|
||||||
|
// ── 400G ZR Coherent (Telecom) ──────────────────────────────────────────
|
||||||
|
{ pid: "CIMC-QSFP-DD-ZR-400G", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OpenZR+ / OIF 400ZR", category: "Telecom", notes: "CIMC 400G ZR QSFP-DD coherent — ex-II-VI Finisar 400ZR, 120km unamplified" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCimcOem(): Promise<void> {
|
||||||
|
console.log("=== CIMC Semiconductors (Coherent Corp / ex-II-VI Finisar) OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"CIMC Semiconductors",
|
||||||
|
"oem",
|
||||||
|
"https://www.coherent.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CIMC_PIDS) {
|
||||||
|
const slug = `cimc-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== CIMC Semiconductors OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CIMC_PIDS.length}`);
|
||||||
|
console.log(` Telecom PIDs: ${[...TELECOM_PIDS].length} (CFP2-DCO, ZR, OTN, CWDM, DWDM)\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCimcOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
130
packages/scraper/src/scrapers/cisco-asr-oem.ts
Normal file
130
packages/scraper/src/scrapers/cisco-asr-oem.ts
Normal file
@ -0,0 +1,130 @@
|
|||||||
|
/**
|
||||||
|
* Cisco ASR OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Cisco ASR-branded transceiver PIDs for ASR 1000, 9000, and 9900
|
||||||
|
* series service provider edge and core routers, including OC-n SONET,
|
||||||
|
* CFP/CFP2 coherent, and DWDM modules.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Cisco ASR 9000 Series Router Data Sheet
|
||||||
|
* - Cisco ASR 1000 Series Transceiver Module Selection Guide
|
||||||
|
* - Cisco 400G Coherent Pluggable Optics for ASR 9000 Data Sheet
|
||||||
|
* - Cisco DWDM SFP10G Transceiver Module (OID EDCS-959978)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cisco-asr-oem.ts
|
||||||
|
* Cron: daily at 13:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface AsrPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category: "DataCenter" | "Telecom";
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Telecom PIDs: OC-n SONET, CFP, CFP2-DCO coherent, DWDM, ZR variants.
|
||||||
|
const ASR_PIDS: AsrPID[] = [
|
||||||
|
// ── GE SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GE-S", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", category: "DataCenter", notes: "Cisco ASR GE multimode SFP" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", category: "DataCenter", notes: "Cisco ASR 10G multimode SFP+" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", category: "DataCenter", notes: "Cisco ASR 10G single-mode SFP+ 10km" },
|
||||||
|
{ pid: "SFP-10G-LR-S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", category: "DataCenter", notes: "Cisco ASR 10G single-mode SFP+ S-variant" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", category: "DataCenter", notes: "Cisco ASR 10G extended-reach SFP+ 40km" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom", notes: "Cisco ASR 10G ZR SFP+ 80km metro/SP" },
|
||||||
|
|
||||||
|
// ── SONET/SDH OC-n SFP ──────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "OC3", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Cisco ASR OC-3 short-reach SONET SFP" },
|
||||||
|
{ pid: "SFP-OC3-LR", formFactor: "SFP", speedGbps: 0.155, speed: "OC3", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Cisco ASR OC-3 long-reach SONET SFP" },
|
||||||
|
{ pid: "SFP-OC12-SR", formFactor: "SFP", speedGbps: 0.622, speed: "OC12", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Cisco ASR OC-12 short-reach SONET SFP" },
|
||||||
|
{ pid: "SFP-OC12-LR", formFactor: "SFP", speedGbps: 0.622, speed: "OC12", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Cisco ASR OC-12 long-reach SONET SFP" },
|
||||||
|
{ pid: "SFP-OC48-SR", formFactor: "SFP", speedGbps: 2.488, speed: "OC48", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-48/STM-16", category: "Telecom", notes: "Cisco ASR OC-48 short-reach SONET SFP" },
|
||||||
|
{ pid: "SFP-OC48-LR", formFactor: "SFP", speedGbps: 2.488, speed: "OC48", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-48/STM-16", category: "Telecom", notes: "Cisco ASR OC-48 long-reach SONET SFP" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-LR4-S", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", category: "DataCenter", notes: "Cisco ASR 40G QSFP+ LR4 10km" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 / CFP ───────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-LR4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", category: "DataCenter", notes: "Cisco ASR 100G QSFP28 LR4 10km" },
|
||||||
|
{ pid: "QSFP-100G-ZR-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "100GBASE-ZR", category: "Telecom", notes: "Cisco ASR 100G coherent ZR QSFP28 80km metro" },
|
||||||
|
{ pid: "CFP-100G-LR4", formFactor: "CFP", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", category: "Telecom", notes: "Cisco ASR 100G CFP LR4 10km SP router line card" },
|
||||||
|
{ pid: "CFP2-100G-LR4", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", category: "Telecom", notes: "Cisco ASR 100G CFP2 LR4 10km compact form factor" },
|
||||||
|
{ pid: "CFP2-DCO-200G", formFactor: "CFP2", speedGbps: 200, speed: "200G", reachMeters: 1000000, reachLabel: "DCO-longhaul", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", standard: "OTN/FlexE", category: "Telecom", notes: "Cisco ASR 200G CFP2 digital coherent optics long-haul DWDM" },
|
||||||
|
|
||||||
|
// ── DWDM SFP10G / 400G ZR ───────────────────────────────────────────────
|
||||||
|
{ pid: "DWDM-SFP10G-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM-C", fiberType: "SMF", connector: "LC", wavelengths: "C-band ITU 100GHz", standard: "10GBASE-ZR", category: "Telecom", notes: "Cisco ASR 9000 DWDM SFP10G C-band ITU tunable 80km" },
|
||||||
|
{ pid: "QSFP-DD-400G-ZR-S", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR-120km", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", standard: "400ZR OpenROADM", category: "Telecom", notes: "Cisco ASR 400G QSFP-DD coherent ZR 120km metro DWDM" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCiscoAsrOem(): Promise<void> {
|
||||||
|
console.log("=== Cisco ASR OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Cisco ASR",
|
||||||
|
"oem",
|
||||||
|
"https://www.cisco.com/c/en/us/products/routers/asr-series/index.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ASR_PIDS) {
|
||||||
|
const slug = `cisco-asr-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
category = EXCLUDED.category,
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const telecomCount = ASR_PIDS.filter(p => p.category === "Telecom").length;
|
||||||
|
const dcCount = ASR_PIDS.filter(p => p.category === "DataCenter").length;
|
||||||
|
|
||||||
|
console.log(`\n=== Cisco ASR OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ASR_PIDS.length} (Telecom: ${telecomCount}, DataCenter: ${dcCount})\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCiscoAsrOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
122
packages/scraper/src/scrapers/cisco-catalyst-oem.ts
Normal file
122
packages/scraper/src/scrapers/cisco-catalyst-oem.ts
Normal file
@ -0,0 +1,122 @@
|
|||||||
|
/**
|
||||||
|
* Cisco Catalyst OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Cisco Catalyst-branded transceiver PIDs (SFP= prefix) used in
|
||||||
|
* Catalyst 9000, 3000, 4000, and 6000 series access/distribution switches.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Cisco Catalyst Transceiver Module Selection Guide
|
||||||
|
* - Cisco SFP and SFP+ Transceiver Modules Data Sheet (EDCS-806701)
|
||||||
|
* - Cisco Catalyst 9000 Hardware Compatibility Matrix
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cisco-catalyst-oem.ts
|
||||||
|
* Cron: daily at 12:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CatalystPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CATALYST_PIDS: CatalystPID[] = [
|
||||||
|
// ── 1G GLC SFP ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GLC-SX-MMD", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Cisco GLC 1G multimode SFP w/ DOM" },
|
||||||
|
{ pid: "GLC-LH-SMD", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Cisco GLC 1G single-mode 10km w/ DOM" },
|
||||||
|
{ pid: "GLC-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Cisco GLC 1G copper SFP" },
|
||||||
|
{ pid: "GLC-BX-U", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BX-U", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1490nm", standard: "1000BASE-BX", notes: "Cisco 1G BiDi SFP upstream (1310nm TX)" },
|
||||||
|
{ pid: "GLC-BX-D", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BX-D", fiberType: "SMF", connector: "LC", wavelengths: "TX1490/RX1310nm", standard: "1000BASE-BX", notes: "Cisco 1G BiDi SFP downstream (1490nm TX)" },
|
||||||
|
{ pid: "GLC-ZX-SMD", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "Cisco GLC 1G extended-reach 80km w/ DOM" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Cisco 10G multimode SFP+" },
|
||||||
|
{ pid: "SFP-10G-SR-S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Cisco 10G multimode SFP+ S-variant (Catalyst)" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Cisco 10G single-mode SFP+ 10km" },
|
||||||
|
{ pid: "SFP-10G-LR-S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Cisco 10G single-mode SFP+ 10km S-variant" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Cisco 10G extended-reach SFP+ 40km" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Cisco 10G ZR SFP+ 80km" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-SR-S", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Cisco 25G multimode SFP28 Catalyst" },
|
||||||
|
{ pid: "SFP-25G-LR-S", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Cisco 25G single-mode SFP28 Catalyst 10km" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Cisco 40G multimode QSFP+" },
|
||||||
|
{ pid: "QSFP-40G-LR4-S", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", notes: "Cisco 40G single-mode QSFP+ Catalyst S-variant" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-SR4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Cisco 100G multimode QSFP28 Catalyst" },
|
||||||
|
{ pid: "QSFP-100G-LR4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Cisco 100G single-mode QSFP28 Catalyst 10km" },
|
||||||
|
{ pid: "QSFP-100G-CWDM4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4", notes: "Cisco 100G CWDM4 QSFP28 Catalyst 2km SMF" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-DR4-S", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Cisco 400G QSFP-DD DR4 Catalyst 500m SMF" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCiscoCatalystOem(): Promise<void> {
|
||||||
|
console.log("=== Cisco Catalyst OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Cisco Catalyst",
|
||||||
|
"oem",
|
||||||
|
"https://www.cisco.com/c/en/us/products/switches/catalyst-series-switches.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CATALYST_PIDS) {
|
||||||
|
const slug = `cisco-cat-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Cisco Catalyst OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CATALYST_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCiscoCatalystOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
119
packages/scraper/src/scrapers/cisco-ie-oem.ts
Normal file
119
packages/scraper/src/scrapers/cisco-ie-oem.ts
Normal file
@ -0,0 +1,119 @@
|
|||||||
|
/**
|
||||||
|
* Cisco Industrial Ethernet (IE) OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Cisco Industrial Ethernet SFP/SFP+/SFP28/QSFP+
|
||||||
|
* modules qualified for the IE 2000, IE 3000, IE 4000, and IE 5000 switch
|
||||||
|
* families used in industrial automation, utility, and ruggedized deployments.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Cisco Industrial Ethernet product catalog
|
||||||
|
* (cisco.com/c/en/us/products/switches/industrial-ethernet-switches)
|
||||||
|
* - Cisco IE-series compatibility matrices and SFP datasheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cisco-ie-oem.ts
|
||||||
|
* Cron: daily at 18:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CiscoIePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CISCO_IE_PIDS: CiscoIePID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IE-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Cisco IE SFP 1G SX IE2000/3000/4000/5000 series" },
|
||||||
|
{ pid: "IE-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Cisco IE SFP 1G LX IE2000/3000/4000/5000 series" },
|
||||||
|
{ pid: "IE-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Cisco IE SFP 1G copper RJ45 industrial" },
|
||||||
|
{ pid: "IE-SFP-GE-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "Cisco IE SFP GE ZX 80km long-haul industrial" },
|
||||||
|
|
||||||
|
// ── 1G SFP BiDi ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IE-SFP-GE-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1310", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1490nm", notes: "Cisco IE SFP GE BiDi TX1310 single fiber industrial" },
|
||||||
|
{ pid: "IE-SFP-GE-BIDI-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1490", fiberType: "SMF", connector: "LC", wavelengths: "TX1490/RX1310nm", notes: "Cisco IE SFP GE BiDi TX1490 single fiber industrial" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IE-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Cisco IE SFP+ 10G SR IE4000/IE5000 series" },
|
||||||
|
{ pid: "IE-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Cisco IE SFP+ 10G LR IE4000/IE5000 series" },
|
||||||
|
{ pid: "IE-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Cisco IE SFP+ 10G ER IE4000/IE5000 series" },
|
||||||
|
|
||||||
|
// ── 1G CWDM SFP (industrial/utility rings) ──────────────────────────────
|
||||||
|
{ pid: "IE-SFP-CWDM-1470", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1470", fiberType: "SMF", connector: "LC", wavelengths: "1470nm", notes: "Cisco IE SFP CWDM 1470nm 40km industrial ring" },
|
||||||
|
{ pid: "IE-SFP-CWDM-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1490", fiberType: "SMF", connector: "LC", wavelengths: "1490nm", notes: "Cisco IE SFP CWDM 1490nm 40km industrial ring" },
|
||||||
|
{ pid: "IE-SFP-CWDM-1510", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1510", fiberType: "SMF", connector: "LC", wavelengths: "1510nm", notes: "Cisco IE SFP CWDM 1510nm 40km industrial ring" },
|
||||||
|
{ pid: "IE-SFP-CWDM-1530", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1530", fiberType: "SMF", connector: "LC", wavelengths: "1530nm", notes: "Cisco IE SFP CWDM 1530nm 40km industrial ring" },
|
||||||
|
{ pid: "IE-SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", notes: "Cisco IE SFP CWDM 1550nm 40km industrial ring" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IE-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "Cisco IE QSFP+ 40G LR4 IE5000 core uplink" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IE-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Cisco IE SFP28 25G LR IE5000 next-gen uplink" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCiscoIeOem(): Promise<void> {
|
||||||
|
console.log("=== Cisco Industrial Ethernet (IE) OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Cisco Industrial",
|
||||||
|
"oem",
|
||||||
|
"https://www.cisco.com/c/en/us/products/switches/industrial-ethernet-switches/index.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CISCO_IE_PIDS) {
|
||||||
|
const slug = `cisco-ie-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Cisco Industrial Ethernet (IE) OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CISCO_IE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCiscoIeOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
120
packages/scraper/src/scrapers/cisco-meraki-oem.ts
Normal file
120
packages/scraper/src/scrapers/cisco-meraki-oem.ts
Normal file
@ -0,0 +1,120 @@
|
|||||||
|
/**
|
||||||
|
* Cisco Meraki OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Cisco Meraki-branded transceiver PIDs for MX security appliances,
|
||||||
|
* MS switches, and MR wireless platforms (cloud-managed networking).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Cisco Meraki Hardware Compatibility List (meraki.cisco.com)
|
||||||
|
* - Meraki MX/MS/MR Datasheet Transceiver Appendices
|
||||||
|
* - Cisco Meraki Transceiver Module Selection Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cisco-meraki-oem.ts
|
||||||
|
* Cron: daily at 12:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface MerakiPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const MERAKI_PIDS: MerakiPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "MA-SFP-1GB-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Meraki 1G multimode SFP, MX/MS" },
|
||||||
|
{ pid: "MA-SFP-1GB-LX10", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX10", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Meraki 1G single-mode SFP 10km" },
|
||||||
|
{ pid: "MA-SFP-1GB-TX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "TX", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Meraki 1G copper SFP" },
|
||||||
|
{ pid: "MA-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "GE-T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Meraki GE copper SFP alternate SKU" },
|
||||||
|
{ pid: "MA-SFP-1GB-SX-MR", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX-MR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Meraki MR-series 1G multimode SFP" },
|
||||||
|
{ pid: "MA-SFP-1GB-TELEWORKER", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-20km", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", standard: "1000BASE-BX", notes: "Meraki teleworker BiDi SFP 1310/1550nm" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "MA-SFP-10GB-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Meraki 10G multimode SFP+" },
|
||||||
|
{ pid: "MA-SFP-10GB-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Meraki 10G single-mode SFP+ 10km" },
|
||||||
|
{ pid: "MA-SFP-10GB-LRM", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM", notes: "Meraki 10G long-reach multimode SFP+" },
|
||||||
|
{ pid: "MA-SFP-10GB-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Meraki 10G extended-reach SFP+ 40km" },
|
||||||
|
{ pid: "MA-SFP-10GB-BIDI", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi-10km", fiberType: "SMF", connector: "LC", wavelengths: "TX1270/RX1330nm", standard: "10GBASE-BX", notes: "Meraki 10G BiDi SFP+ single fiber" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "MA-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Meraki 25G multimode SFP28" },
|
||||||
|
{ pid: "MA-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Meraki 25G single-mode SFP28 10km" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "MA-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Meraki 40G multimode QSFP+" },
|
||||||
|
{ pid: "MA-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", notes: "Meraki 40G single-mode QSFP+ 10km" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "MA-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Meraki 100G multimode QSFP28" },
|
||||||
|
{ pid: "MA-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Meraki 100G single-mode QSFP28 10km" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "MA-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Meraki 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCiscoMerakiOem(): Promise<void> {
|
||||||
|
console.log("=== Cisco Meraki OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Cisco Meraki",
|
||||||
|
"oem",
|
||||||
|
"https://meraki.cisco.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of MERAKI_PIDS) {
|
||||||
|
const slug = `meraki-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Cisco Meraki OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${MERAKI_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCiscoMerakiOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
125
packages/scraper/src/scrapers/cisco-nexus-oem.ts
Normal file
125
packages/scraper/src/scrapers/cisco-nexus-oem.ts
Normal file
@ -0,0 +1,125 @@
|
|||||||
|
/**
|
||||||
|
* Cisco Nexus OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Cisco Nexus-branded transceiver PIDs for Nexus 2000, 3000, 5000,
|
||||||
|
* 7000, and 9000 series data center switches (N-prefix PIDs).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Cisco Nexus Transceiver Module Selection Guide
|
||||||
|
* - Nexus 9000 Series Hardware Reference (N9K-C9336C-FX2, N9K-C93180YC-FX)
|
||||||
|
* - Cisco 400G Transceiver Modules for Nexus 9000 Data Sheet
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cisco-nexus-oem.ts
|
||||||
|
* Cron: daily at 12:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface NexusPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// All Nexus transceivers are DataCenter; no telecom OC-n PIDs in this portfolio.
|
||||||
|
const NEXUS_PIDS: NexusPID[] = [
|
||||||
|
// ── 1G GLC SFP ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GLC-SX-MMD", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Cisco 1G multimode SFP w/ DOM, Nexus compatible" },
|
||||||
|
{ pid: "GLC-LH-SMD", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Cisco 1G single-mode SFP 10km w/ DOM, Nexus" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Cisco 10G multimode SFP+ Nexus" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Cisco 10G single-mode SFP+ Nexus 10km" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Cisco 10G extended-reach SFP+ Nexus 40km" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-SR-S", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Cisco 25G multimode SFP28 Nexus" },
|
||||||
|
{ pid: "SFP-25G-LR-S", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Cisco 25G single-mode SFP28 Nexus 10km" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Cisco 40G multimode QSFP+ Nexus" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", notes: "Cisco 40G single-mode QSFP+ Nexus 10km" },
|
||||||
|
{ pid: "QSFP-40G-SR-BD", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR-BiDi", fiberType: "MMF", connector: "LC", wavelengths: "TX832-860/RX882-910nm", standard: "40GBASE-BiDi", notes: "Cisco 40G BiDi QSFP+ duplex LC on OM3/OM4 MMF" },
|
||||||
|
{ pid: "QSFP-40G-CSR4-S", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 400, reachLabel: "CSR4-400m", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-CSR4", notes: "Cisco 40G extended-reach AOC/CSR4 QSFP+ 400m MMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-SR4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Cisco 100G multimode QSFP28 Nexus" },
|
||||||
|
{ pid: "QSFP-100G-LR4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Cisco 100G single-mode QSFP28 Nexus 10km" },
|
||||||
|
{ pid: "QSFP-100G-CWDM4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4", notes: "Cisco 100G CWDM4 QSFP28 Nexus 2km SMF" },
|
||||||
|
{ pid: "QSFP-100G-PSM4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "PSM4-500m", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "100GBASE-PSM4", notes: "Cisco 100G PSM4 QSFP28 Nexus 500m parallel SMF" },
|
||||||
|
{ pid: "QSFP-100G-SM4-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "SM4-BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1270/RX1330nm", standard: "100GBASE-BiDi", notes: "Cisco 100G BiDi SM4 QSFP28 Nexus duplex SMF 2km" },
|
||||||
|
{ pid: "QSFP-100G-AOC1M-S", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "AOC-1m", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Cisco 100G active optical cable 1m Nexus" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-DR4-S", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Cisco 400G QSFP-DD DR4 Nexus 9000 500m SMF" },
|
||||||
|
{ pid: "QSFP-DD-400G-SR8-S", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "Cisco 400G QSFP-DD SR8 Nexus 9000 100m MMF" },
|
||||||
|
|
||||||
|
// ── 800G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-800G-DR8", formFactor: "QSFP-DD", speedGbps: 800, speed: "800G", reachMeters: 500, reachLabel: "DR8", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "800GBASE-DR8", notes: "Cisco 800G QSFP-DD DR8 Nexus 9000 next-gen DC" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCiscoNexusOem(): Promise<void> {
|
||||||
|
console.log("=== Cisco Nexus OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Cisco Nexus",
|
||||||
|
"oem",
|
||||||
|
"https://www.cisco.com/c/en/us/products/switches/nexus-series/index.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of NEXUS_PIDS) {
|
||||||
|
const slug = `cisco-nx-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Cisco Nexus OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${NEXUS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCiscoNexusOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
153
packages/scraper/src/scrapers/coherent-oem.ts
Normal file
153
packages/scraper/src/scrapers/coherent-oem.ts
Normal file
@ -0,0 +1,153 @@
|
|||||||
|
/**
|
||||||
|
* Coherent Corp (formerly II-VI / Finisar) OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Coherent Corp transceiver PIDs sold under OEM branding.
|
||||||
|
* Covers 1G–800G optical transceivers, coherent DWDM/ZR modules, and DAC cables
|
||||||
|
* across SFP, SFP+, SFP28, QSFP+, QSFP28, CFP2, QSFP-DD, and OSFP form factors.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Coherent Corp Product Selector (coherent.com/networking)
|
||||||
|
* - II-VI Incorporated Optical Transceivers Datasheet Archive
|
||||||
|
* - Finisar / II-VI FTLX / FTCD / FCBN series datasheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/coherent-oem.ts
|
||||||
|
* Cron: daily at 11:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CoherentPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const COHERENT_PIDS: CoherentPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTLF8521P2BNL", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "FTLF1318P3BTL", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "FTLF1324P3BTL", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "FTLF1318P3BCL", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX-10", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTLX8571D3BCL", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "FTLX1475D3BCL", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "FTLX1471D3BCL", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "FTLX1871D3BCL", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "FTLX1471D3BNL", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTLX3811M327-xx", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: undefined, notes: "II-VI 10G DWDM SFP+ tunable" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTCD1312E1PCM", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "FTCD1321E1PCM", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "FTCD1321E3PCM", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTL410QE2C", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "FTL4C1QE1C", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTCD2600P1BCM", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "FTCD4601P1BCM", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "FTCD4601PBCM", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "FTCD2601P2BCM", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
{ pid: "FTCD2621P2BCM", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "FR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-FR" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 coherent DWDM ─────────────────────────────────────────────
|
||||||
|
{ pid: "CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1500000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "II-VI 100G CFP2 DCO coherent" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTCD8601P1BCM", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "FTCD2628P2BCM", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "FTCD4628P2BCM", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "FTCD4629P2BCM", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 400G ZR coherent QSFP-DD ────────────────────────────────────────────
|
||||||
|
{ pid: "ZR400-QSFPDD", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", notes: "II-VI 400G ZR coherent QSFP-DD" },
|
||||||
|
|
||||||
|
// ── 800G OSFP ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "800G-OSFP-DR8", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 500, reachLabel: "DR8", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "800GBASE-DR8" },
|
||||||
|
{ pid: "800G-OSFP-FR8", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 2000, reachLabel: "FR8", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FCBN210QB1Cxx", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "FCBN225QB1Cxx", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP28" },
|
||||||
|
{ pid: "FCBN410QB1Cxx", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
/** PIDs that belong to the Telecom category instead of DataCenter */
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"FTLX3811M327-xx",
|
||||||
|
"CFP2-DCO-100G",
|
||||||
|
"ZR400-QSFPDD",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeCoherentOem(): Promise<void> {
|
||||||
|
console.log("=== Coherent Corp OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Coherent Corp",
|
||||||
|
"oem",
|
||||||
|
"https://www.coherent.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of COHERENT_PIDS) {
|
||||||
|
const slug = `coherent-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter";
|
||||||
|
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Coherent Corp OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${COHERENT_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCoherentOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
133
packages/scraper/src/scrapers/commscope-oem.ts
Normal file
133
packages/scraper/src/scrapers/commscope-oem.ts
Normal file
@ -0,0 +1,133 @@
|
|||||||
|
/**
|
||||||
|
* CommScope OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds CommScope-branded transceiver PIDs for RUCKUS ICX switching,
|
||||||
|
* former ARRIS broadband, and CommScope network infrastructure platforms.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - CommScope RUCKUS ICX Optics Compatibility Matrix (commscope.com)
|
||||||
|
* - CommScope Access Network Optics Portfolio
|
||||||
|
* - RUCKUS Networks SFP/QSFP Product Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/commscope-oem.ts
|
||||||
|
* Cron: daily at 23:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CommScopePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const COMMSCOPE_PIDS: CommScopePID[] = [
|
||||||
|
// ── 1G SFP ───────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "RJ45", fiberType: "Cu", connector: "RJ45" },
|
||||||
|
{ pid: "SFP-GE-BIDI", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BIDI", fiberType: "SMF", connector: "LC", wavelengths: "1310/1490nm", notes: "CommScope 1G BiDi SFP 1310Tx/1490Rx" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "SFP-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "RJ45", fiberType: "Cu", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "SFP28-25G-ER", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-ER" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
|
||||||
|
// ── PON / Access (Telecom) ────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GPON-OLT", formFactor: "SFP", speedGbps: 2.5, speed: "GPON", reachMeters: 20000, reachLabel: "OLT", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", category: "Telecom", notes: "CommScope GPON OLT SFP 2.488G Tx / 1.244G Rx" },
|
||||||
|
{ pid: "SFP-EPON-OLT", formFactor: "SFP", speedGbps: 1, speed: "EPON", reachMeters: 20000, reachLabel: "OLT", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", category: "Telecom", notes: "CommScope EPON OLT SFP 1G" },
|
||||||
|
{ pid: "SFP-XGS-PON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "OLT", fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", category: "Telecom", notes: "CommScope XGS-PON OLT SFP+ 10G symmetric" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (PON/access OLT modules)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"SFP-GPON-OLT",
|
||||||
|
"SFP-EPON-OLT",
|
||||||
|
"SFP-XGS-PON-OLT",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeCommScopeOem(): Promise<void> {
|
||||||
|
console.log("=== CommScope OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"CommScope",
|
||||||
|
"oem",
|
||||||
|
"https://www.commscope.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of COMMSCOPE_PIDS) {
|
||||||
|
const slug = `commscope-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== CommScope OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${COMMSCOPE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCommScopeOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
116
packages/scraper/src/scrapers/comnet-oem.ts
Normal file
116
packages/scraper/src/scrapers/comnet-oem.ts
Normal file
@ -0,0 +1,116 @@
|
|||||||
|
/**
|
||||||
|
* Comnet OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Comnet industrial-grade SFP modules used in
|
||||||
|
* media converters, managed switches, and NMS platforms for utilities
|
||||||
|
* and transportation infrastructure.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Comnet SFP module product catalog (comnet.net)
|
||||||
|
* - Comnet industrial switch hardware installation guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/comnet-oem.ts
|
||||||
|
* Cron: daily at 17:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface ComnetPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const COMNET_PIDS: ComnetPID[] = [
|
||||||
|
// ── 100M SFP ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1FESX-M", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 2000, reachLabel: "FX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "100BASE-FX" },
|
||||||
|
{ pid: "SFP-1FELX-S", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 40000, reachLabel: "LFX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100BASE-LFX" },
|
||||||
|
{ pid: "SFP-1FEW1BXU", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 20000, reachLabel: "BiDi-20K", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", notes: "Comnet 100M BiDi SFP upstream" },
|
||||||
|
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1GESX-M", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-1GELX-S10", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-1GELX-S20", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "Comnet 1G SM SFP 20km" },
|
||||||
|
{ pid: "SFP-1GELX-S40", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "Comnet 1G SM SFP 40km" },
|
||||||
|
{ pid: "SFP-1GEZX-S80", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "SFP-1GETP-M", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "SFP-1GEW1BXU", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-20K", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", notes: "Comnet 1G BiDi SFP upstream" },
|
||||||
|
{ pid: "SFP-1GEW1BXD", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-20K", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310nm", notes: "Comnet 1G BiDi SFP downstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10GESR-M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10GELR-S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10GEER-S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10GEZR-S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "SFP-10GETP-M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10GDAC1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1m", fiberType: "DAC", connector: "SFP+", notes: "Comnet SFP+ 10G DAC 1m" },
|
||||||
|
{ pid: "SFP-10GDAC3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3m", fiberType: "DAC", connector: "SFP+", notes: "Comnet SFP+ 10G DAC 3m" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeComnetOem(): Promise<void> {
|
||||||
|
console.log("=== Comnet OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Comnet",
|
||||||
|
"oem",
|
||||||
|
"https://www.comnet.net",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of COMNET_PIDS) {
|
||||||
|
const slug = `comnet-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Comnet OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${COMNET_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeComnetOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
147
packages/scraper/src/scrapers/comtrend-oem.ts
Normal file
147
packages/scraper/src/scrapers/comtrend-oem.ts
Normal file
@ -0,0 +1,147 @@
|
|||||||
|
/**
|
||||||
|
* Comtrend OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Comtrend-branded transceiver PIDs for DSL/GPON/XGS-PON/EPON
|
||||||
|
* CPE and OLT access platforms.
|
||||||
|
* Comtrend is a DSL and fibre access networking vendor supplying SFP
|
||||||
|
* modules for GPON, XGS-PON, EPON, and GE/10GE CPE and OLT equipment.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Comtrend GPON OLT/ONU SFP Module Data Sheets
|
||||||
|
* - Comtrend XGS-PON Pluggable Optics Guide
|
||||||
|
* - Comtrend EPON SFP Specification Documents
|
||||||
|
* - comtrend.com product catalogue
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/comtrend-oem.ts
|
||||||
|
* Cron: daily at 01:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface ComtrendPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const COMTREND_PIDS: ComtrendPID[] = [
|
||||||
|
// ── Standard 1G SFP ──────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "Cu", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom", notes: "Comtrend 10G ZR for extended backhaul" },
|
||||||
|
|
||||||
|
// ── GPON SFP (OLT-side, 1490nm TX / 1310nm RX) ───────────────────────────
|
||||||
|
{ pid: "SFP-GPON-OLT", formFactor: "SFP", speedGbps: 1.244, speed: "GPON", reachMeters: 20000, reachLabel: "OLT", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "ITU-T G.984", category: "Telecom", notes: "Comtrend GPON OLT SFP — 1490nm TX, 1310nm RX, class B+" },
|
||||||
|
// ── GPON SFP (ONU-side, 1310nm TX / 1490nm RX) ───────────────────────────
|
||||||
|
{ pid: "SFP-GPON-ONU", formFactor: "SFP", speedGbps: 1.244, speed: "GPON", reachMeters: 20000, reachLabel: "ONU", fiberType: "SMF", connector: "SC", wavelengths: "1310/1490nm", standard: "ITU-T G.984", category: "Telecom", notes: "Comtrend GPON ONU SFP — 1310nm TX, 1490nm RX" },
|
||||||
|
|
||||||
|
// ── XGS-PON SFP+ (OLT-side) ──────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-XGS-PON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "OLT", fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "ITU-T G.9807.1", category: "Telecom", notes: "Comtrend XGS-PON OLT SFP+ — 1577nm TX, 1270nm RX" },
|
||||||
|
// ── XGS-PON SFP+ (ONU-side) ──────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-XGS-PON-ONU", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "ONU", fiberType: "SMF", connector: "SC", wavelengths: "1270/1577nm", standard: "ITU-T G.9807.1", category: "Telecom", notes: "Comtrend XGS-PON ONU SFP+ — 1270nm TX, 1577nm RX" },
|
||||||
|
|
||||||
|
// ── EPON SFP (OLT-side) ──────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-EPON-OLT", formFactor: "SFP", speedGbps: 1.25, speed: "EPON", reachMeters: 20000, reachLabel: "OLT", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "IEEE 802.3ah", category: "Telecom", notes: "Comtrend EPON OLT SFP — 1490nm TX, 1310nm RX" },
|
||||||
|
// ── EPON SFP (ONU-side) ──────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-EPON-ONU", formFactor: "SFP", speedGbps: 1.25, speed: "EPON", reachMeters: 20000, reachLabel: "ONU", fiberType: "SMF", connector: "SC", wavelengths: "1310/1490nm", standard: "IEEE 802.3ah", category: "Telecom", notes: "Comtrend EPON ONU SFP — 1310nm TX, 1490nm RX" },
|
||||||
|
|
||||||
|
// ── BiDi SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-BIDI-1310-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1490nm", category: "Telecom", notes: "Comtrend BiDi SFP 1310nm TX / 1490nm RX" },
|
||||||
|
{ pid: "SFP-BIDI-1310-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1550nm", category: "Telecom", notes: "Comtrend BiDi SFP 1310nm TX / 1550nm RX" },
|
||||||
|
{ pid: "SFP-GE-BIDI-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "BiDi-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550/1310nm", category: "Telecom", notes: "Comtrend BiDi SFP 1550nm TX / 1310nm RX" },
|
||||||
|
|
||||||
|
// ── 10G BiDi SFP+ ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-BIDI-1270", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1270/1330nm", category: "Telecom", notes: "Comtrend 10G BiDi SFP+ 1270nm TX / 1330nm RX" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (PON, BiDi, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"SFP-10G-ZR",
|
||||||
|
"SFP-GPON-OLT",
|
||||||
|
"SFP-GPON-ONU",
|
||||||
|
"SFP-XGS-PON-OLT",
|
||||||
|
"SFP-XGS-PON-ONU",
|
||||||
|
"SFP-EPON-OLT",
|
||||||
|
"SFP-EPON-ONU",
|
||||||
|
"SFP-BIDI-1310-1490",
|
||||||
|
"SFP-BIDI-1310-1550",
|
||||||
|
"SFP-GE-BIDI-1550",
|
||||||
|
"SFP-10G-BIDI-1270",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeComtrendOem(): Promise<void> {
|
||||||
|
console.log("=== Comtrend OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Comtrend",
|
||||||
|
"oem",
|
||||||
|
"https://www.comtrend.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of COMTREND_PIDS) {
|
||||||
|
const slug = `comtrend-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Comtrend OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${COMTREND_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeComtrendOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
139
packages/scraper/src/scrapers/coriant-oem.ts
Normal file
139
packages/scraper/src/scrapers/coriant-oem.ts
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
/**
|
||||||
|
* Coriant OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Coriant-branded transceiver PIDs for WDM platforms:
|
||||||
|
* hiT 7300, hiT 7100 series, and Groove G30.
|
||||||
|
*
|
||||||
|
* Note: Coriant was previously Tellabs/Siemens/Nokia Siemens Networks.
|
||||||
|
* Coriant was acquired by Infinera in 2018. These PIDs cover legacy
|
||||||
|
* Coriant WDM platform optics and are distinct from infinera-oem.ts
|
||||||
|
* which covers modern Infinera HIL PIDs.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Coriant hiT 7300 Product Documentation
|
||||||
|
* - Coriant hiT 7100 System Description
|
||||||
|
* - Coriant Groove G30 Platform Guide
|
||||||
|
* - Infinera (Coriant) Legacy Product Portfolio (infinera.com/products/coriant/)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/coriant-oem.ts
|
||||||
|
* Cron: daily at 21:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CoriantPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CORIANT_PIDS: CoriantPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "COR-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "COR-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "COR-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "COR-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "COR-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ (hiT WDM) ─────────────────────────────────────────────
|
||||||
|
{ pid: "COR-SFP-10G-DW-TUNE", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "Coriant hiT 10G DWDM tunable SFP+" },
|
||||||
|
|
||||||
|
// ── 100G CFP Coherent (hiT 7300) ────────────────────────────────────────
|
||||||
|
{ pid: "COR-CFP-100G-OTU4", formFactor: "CFP", speedGbps: 100, speed: "100G", reachMeters: 1500000, reachLabel: "OTU4", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "OTU4", category: "Telecom", notes: "Coriant hiT 7300 100G CFP OTU4 coherent" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 Coherent (Groove G30) ─────────────────────────────────────
|
||||||
|
{ pid: "COR-CFP2-100G-DCO", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1500000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Coriant Groove G30 100G CFP2 DCO coherent" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "COR-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "COR-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-QSFPDD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "COR-QSFPDD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", category: "Telecom", notes: "Coriant/Infinera Groove 400G ZR coherent QSFP-DD" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "COR-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (coherent/DWDM)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"COR-SFP-10G-DW-TUNE",
|
||||||
|
"COR-CFP-100G-OTU4",
|
||||||
|
"COR-CFP2-100G-DCO",
|
||||||
|
"COR-QSFPDD-400G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeCoriantOem(): Promise<void> {
|
||||||
|
console.log("=== Coriant OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Coriant",
|
||||||
|
"oem",
|
||||||
|
"https://www.infinera.com/products/coriant/",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CORIANT_PIDS) {
|
||||||
|
const slug = `coriant-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Coriant OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CORIANT_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCoriantOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
123
packages/scraper/src/scrapers/corning-oem.ts
Normal file
123
packages/scraper/src/scrapers/corning-oem.ts
Normal file
@ -0,0 +1,123 @@
|
|||||||
|
/**
|
||||||
|
* Corning OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Corning-branded SFP/QSFP modules sold with Pretium EDGE infrastructure.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Corning Optical Communications Pretium EDGE Solutions (corning.com/optical-communications)
|
||||||
|
* - Corning Pretium EDGE Hardware & Transceiver Compatibility Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/corning-oem.ts
|
||||||
|
* Cron: daily at 02:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CorningPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CORNING_PIDS: CorningPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "COR-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "COR-SFP-1G-LX10", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX10", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX10" },
|
||||||
|
{ pid: "COR-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "Cu", connector: "RJ45", wavelengths: "N/A", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "COR-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "COR-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "COR-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "COR-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4" },
|
||||||
|
{ pid: "COR-QSFP-40G-ESR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 300, reachLabel: "ESR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-ESR4", notes: "Extended reach MMF 300m" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "COR-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "COR-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "COR-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "COR-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "COR-QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category
|
||||||
|
const TELECOM_PIDS = new Set<string>([]);
|
||||||
|
|
||||||
|
export async function scrapeCorningOem(): Promise<void> {
|
||||||
|
console.log("=== Corning OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Corning",
|
||||||
|
"oem",
|
||||||
|
"https://www.corning.com/optical-communications",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CORNING_PIDS) {
|
||||||
|
const slug = `corning-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Corning OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CORNING_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCorningOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
117
packages/scraper/src/scrapers/cradlepoint-oem.ts
Normal file
117
packages/scraper/src/scrapers/cradlepoint-oem.ts
Normal file
@ -0,0 +1,117 @@
|
|||||||
|
/**
|
||||||
|
* Cradlepoint OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Cradlepoint-branded transceiver PIDs for R1900, E3000, S700 series
|
||||||
|
* enterprise 5G/LTE branch routers with SFP ports.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Cradlepoint Hardware Compatibility Guide (cradlepoint.com)
|
||||||
|
* - R-series / E-series Router Hardware Reference
|
||||||
|
* - Cradlepoint S700 Series Datasheet
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cradlepoint-oem.ts
|
||||||
|
* Cron: daily at 18:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CradlepointPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CRADLEPOINT_PIDS: CradlepointPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CP-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "CP-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "CP-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "CP-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "CP-SFP-1G-BXU", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BXU", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", standard: "1000BASE-BX", notes: "Cradlepoint 1G BiDi SFP upstream" },
|
||||||
|
{ pid: "CP-SFP-1G-BXD", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BXD", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310nm", standard: "1000BASE-BX", notes: "Cradlepoint 1G BiDi SFP downstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CP-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "CP-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "CP-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "CP-SFP-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CP-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "CP-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CP-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "CP-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CP-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "CP-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "CP-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCradlepointOem(): Promise<void> {
|
||||||
|
console.log("=== Cradlepoint OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Cradlepoint",
|
||||||
|
"oem",
|
||||||
|
"https://www.cradlepoint.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CRADLEPOINT_PIDS) {
|
||||||
|
const slug = `cradlepoint-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Cradlepoint OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CRADLEPOINT_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCradlepointOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
120
packages/scraper/src/scrapers/cumulus-networks-oem.ts
Normal file
120
packages/scraper/src/scrapers/cumulus-networks-oem.ts
Normal file
@ -0,0 +1,120 @@
|
|||||||
|
/**
|
||||||
|
* Cumulus Networks / NVIDIA OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Cumulus-branded transceiver PIDs for Cumulus Linux NOS running
|
||||||
|
* on white-box and bare-metal switches (Open Compute, AS series, etc.).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Cumulus Networks Air Platform (air.cumulusnetworks.com)
|
||||||
|
* - Cumulus Linux Hardware Compatibility List (HCL)
|
||||||
|
* - NVIDIA Cumulus Linux Data Sheets and Platform Guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/cumulus-networks-oem.ts
|
||||||
|
* Cron: daily at 10:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface CumulusPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const CUMULUS_PIDS: CumulusPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CN-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "CN-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "CN-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CN-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "CN-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "CN-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CN-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "CN-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CN-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "CN-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CN-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "CN-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "CN-SFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "Extended reach for spine-to-spine links" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CN-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "CN-QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
|
||||||
|
// ── Copper SFP ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CN-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "GE copper for OOB management ports" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeCumulusNetworksOem(): Promise<void> {
|
||||||
|
console.log("=== Cumulus Networks / NVIDIA OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Cumulus Networks",
|
||||||
|
"oem",
|
||||||
|
"https://air.cumulusnetworks.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of CUMULUS_PIDS) {
|
||||||
|
const slug = `cumulus-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Cumulus Networks OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${CUMULUS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeCumulusNetworksOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
148
packages/scraper/src/scrapers/datang-oem.ts
Normal file
148
packages/scraper/src/scrapers/datang-oem.ts
Normal file
@ -0,0 +1,148 @@
|
|||||||
|
/**
|
||||||
|
* Datang Telecom / CICT OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Datang-branded transceiver PIDs used in TD-LTE and 5G NR base stations,
|
||||||
|
* OTN/DWDM transport systems, and carrier Ethernet platforms from Datang Telecom
|
||||||
|
* Technology (大唐电信) and its subsidiary CICT Mobile Communication Technology.
|
||||||
|
* Includes telecom-grade OTN, CWDM, DWDM, and CFP form factors in addition to
|
||||||
|
* standard DataCenter SFP/QSFP variants.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - CICT Mobile Product Catalog (cictmobile.com)
|
||||||
|
* - Datang Telecom TD-LTE/NR hardware guides
|
||||||
|
* - OTN/DWDM transceiver specification sheets
|
||||||
|
* - SFF-8024 and ITU-T G.698.2 DWDM channel plans
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/datang-oem.ts
|
||||||
|
* Cron: daily at 04:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface DatangPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const DATANG_PIDS: DatangPID[] = [
|
||||||
|
// ── 1G SFP (DataCenter) ─────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Datang 1G SFP SX for TD-LTE eNodeB uplink" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Copper GE SFP for management/S1 interface" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ (DataCenter) ───────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Datang 10G SFP+ SR for 5G NR gNB fronthaul" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
|
||||||
|
// ── 10G ZR SFP+ (Telecom — long-haul OTN backhaul) ─────────────────────
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom", notes: "Datang 10G ZR for OTN ring backhaul; carrier transport" },
|
||||||
|
|
||||||
|
// ── SDH / SONET OC-3 / OC-12 SFP (Telecom) ─────────────────────────────
|
||||||
|
{ pid: "SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "155M", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Datang OC-3/STM-1 SFP for SDH legacy circuit interconnect" },
|
||||||
|
{ pid: "SFP-OC12-SR", formFactor: "SFP", speedGbps: 0.622, speed: "622M", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Datang OC-12/STM-4 SFP; TD-LTE legacy SDH transport" },
|
||||||
|
|
||||||
|
// ── OTU2 10G SFP+ (Telecom) ─────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-OTU2-10G", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "G.709 OTU2", category: "Telecom", notes: "Datang OTU2 10G SFP+ for OTN framing on backhaul links" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 (DataCenter) ──────────────────────────────────────────────
|
||||||
|
{ pid: "SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Datang 25G SFP28 for 5G NR C-RAN fronthaul (eCPRI)" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ (DataCenter) ──────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 (DataCenter) ────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Datang 100G QSFP28 LR4 for 5G midhaul aggregation" },
|
||||||
|
|
||||||
|
// ── CWDM SFP (Telecom) ──────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "CWDM", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "G.694.2 CWDM", category: "Telecom", notes: "Datang CWDM SFP 1550nm for metro WDM rings; TD-LTE backhaul" },
|
||||||
|
|
||||||
|
// ── DWDM SFP (Telecom) ──────────────────────────────────────────────────
|
||||||
|
{ pid: "DWDM-SFP-C", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band ITU", standard: "G.694.1 DWDM", category: "Telecom", notes: "Datang DWDM SFP C-band (ITU grid) for dense WDM transport" },
|
||||||
|
|
||||||
|
// ── CFP 100G (Telecom — OTN/coherent line-side) ─────────────────────────
|
||||||
|
{ pid: "CFP-100G-LR4", formFactor: "CFP", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "OIF CFP", category: "Telecom", notes: "Datang 100G CFP LR4 for OTN/SDN line card; carrier transport" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD (DataCenter) ───────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-DR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Datang 400G QSFP-DD for 5G core DC interconnect" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs classified as Telecom (OTN, SONET/SDH, CWDM, DWDM, coherent CFP, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"SFP-10G-ZR",
|
||||||
|
"SFP-OC3-SR",
|
||||||
|
"SFP-OC12-SR",
|
||||||
|
"SFP-OTU2-10G",
|
||||||
|
"SFP-CWDM-1550",
|
||||||
|
"DWDM-SFP-C",
|
||||||
|
"CFP-100G-LR4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeDatangOem(): Promise<void> {
|
||||||
|
console.log("=== Datang Telecom / CICT OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Datang Telecom",
|
||||||
|
"oem",
|
||||||
|
"https://www.cictmobile.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of DATANG_PIDS) {
|
||||||
|
const slug = `datang-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Datang OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${DATANG_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeDatangOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
341
packages/scraper/src/scrapers/dell-storage-oem.ts
Normal file
341
packages/scraper/src/scrapers/dell-storage-oem.ts
Normal file
@ -0,0 +1,341 @@
|
|||||||
|
/**
|
||||||
|
* Dell Storage OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Dell Storage-branded transceiver PIDs for PowerStore, PowerMax,
|
||||||
|
* Unity XT, and SC Series (Compellent) SAN storage systems. Covers FC,
|
||||||
|
* iSCSI, and NVMe-oF host/target optical modules specific to Dell SAN storage.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Dell Technologies Compatibility Matrix (dell.com/support)
|
||||||
|
* - Dell PowerStore/PowerMax Hardware Installation Guides
|
||||||
|
* - Dell Unity XT Expansion Module optical compatibility tables
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/dell-storage-oem.ts
|
||||||
|
* Cron: daily at 06:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface DellStoragePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fibre Channel line-rate speeds (Gbps):
|
||||||
|
// 8G = 8.5 Gbps (8GFC)
|
||||||
|
// 16G = 14.025 Gbps (16GFC)
|
||||||
|
// 32G = 28.05 Gbps (32GFC)
|
||||||
|
// 64G = 56.1 Gbps (64GFC)
|
||||||
|
const DELL_STORAGE_PIDS: DellStoragePID[] = [
|
||||||
|
// ── 8G FC SFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-8G-FC-SW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC SFP+ SW; Dell PowerStore/Unity XT FC host port; OM3/OM4 MMF up to 150m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-8G-FC-LW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC SFP+ LW; Dell PowerStore/Unity XT FC host port long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 16G FC SFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-16G-FC-SW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 125,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ SW; Dell PowerStore 500T/1000T/3000T FC target port; OM4 MMF up to 125m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-16G-FC-LW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ LW; Dell PowerStore 500T/1000T/3000T FC target port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 32G FC SFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-32G-FC-SW",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 28.05,
|
||||||
|
speed: "32G FC",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "32GFC",
|
||||||
|
notes: "32G FC SFP28 SW; Dell PowerStore 5000T/9000T NVMe/FC port; OM4/OM5 MMF",
|
||||||
|
},
|
||||||
|
// ── 64G FC SFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP28-64G-FC-SW",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 56.1,
|
||||||
|
speed: "64G FC",
|
||||||
|
reachMeters: 70,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "64GFC",
|
||||||
|
notes: "64G FC SFP28 SW; Dell PowerMax 2500/8500 next-gen NVMe/FC; OM5 WBMMF up to 70m",
|
||||||
|
},
|
||||||
|
// ── 10GbE SFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-10G-SR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
notes: "10GbE SFP+ SR; Dell Unity XT/SC Series iSCSI and FCoE host port; OM3/OM4 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-10G-LR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
notes: "10GbE SFP+ LR; Dell Unity XT/SC Series iSCSI long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 25GbE SFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP28-25G-SR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "25GBASE-SR",
|
||||||
|
notes: "25GbE SFP28 SR; Dell PowerStore NVMe-oF iSCSI host port; OM4 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP28-25G-LR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "25GBASE-LR",
|
||||||
|
notes: "25GbE SFP28 LR; Dell PowerStore NVMe-oF iSCSI host port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 100GbE QSFP28 ────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-QSFP28-100G-SR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "100GBASE-SR4",
|
||||||
|
notes: "100GbE QSFP28 SR4; Dell PowerStore/PowerMax backend replication interconnect; OM4 MPO-12",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "DELL-QSFP28-100G-LR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-LR4",
|
||||||
|
notes: "100GbE QSFP28 LR4; Dell PowerStore/PowerMax replication long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 400GbE QSFP-DD ───────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-QSFP-DD-400G-DR4",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 500,
|
||||||
|
reachLabel: "DR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "400GBASE-DR4",
|
||||||
|
notes: "400GbE QSFP-DD DR4; Dell PowerMax next-gen NVMe-oF fabric; OS2 SMF up to 500m",
|
||||||
|
},
|
||||||
|
// ── 1GbE SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-1G-SX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 550,
|
||||||
|
reachLabel: "SX",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "1000BASE-SX",
|
||||||
|
notes: "1GbE SFP SX; Dell storage management and iSCSI port; OM2/OM3 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-1G-LX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LX",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "1000BASE-LX",
|
||||||
|
notes: "1GbE SFP LX; Dell storage management and iSCSI port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── Copper SFP ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-SFP-1G-T",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "T",
|
||||||
|
fiberType: "DAC",
|
||||||
|
connector: "RJ45",
|
||||||
|
standard: "1000BASE-T",
|
||||||
|
notes: "1GbE copper SFP RJ45; Dell storage management copper option; Cat5e/6",
|
||||||
|
},
|
||||||
|
// ── 40GbE QSFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "DELL-QSFP-40G-SR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "40GBASE-SR4",
|
||||||
|
notes: "40GbE QSFP+ SR4; Dell storage fabric switch uplink; OM3/OM4 MPO-12",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "DELL-QSFP-40G-LR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "40GBASE-LR4",
|
||||||
|
notes: "40GbE QSFP+ LR4; Dell storage fabric switch uplink long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeDellStorageOem(): Promise<void> {
|
||||||
|
console.log("=== Dell Storage OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Dell Storage",
|
||||||
|
"oem",
|
||||||
|
"https://www.dell.com/en-us/dt/storage/",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of DELL_STORAGE_PIDS) {
|
||||||
|
const slug = `dell-storage-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Dell Storage OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${DELL_STORAGE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeDellStorageOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
125
packages/scraper/src/scrapers/dlink-oem.ts
Normal file
125
packages/scraper/src/scrapers/dlink-oem.ts
Normal file
@ -0,0 +1,125 @@
|
|||||||
|
/**
|
||||||
|
* D-Link OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds D-Link-branded transceiver PIDs for DGS, DXS, and DBS
|
||||||
|
* series managed switches.
|
||||||
|
* Sources: D-Link Transceiver Module Datasheet (dlink.com)
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/dlink-oem.ts
|
||||||
|
* Cron: daily at 09:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface DlinkPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const DLINK_PIDS: DlinkPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DEM-310GT", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "DEM-311GT", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "DEM-314GT", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "DEM-315GT", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH40", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
{ pid: "DEM-302S-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Simplex" },
|
||||||
|
{ pid: "DEM-330T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BiDi-1310", fiberType: "SMF", connector: "LC", wavelengths: "1310/1550nm", notes: "WDM TX1310/RX1550" },
|
||||||
|
{ pid: "DEM-330R", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BiDi-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550/1310nm", notes: "WDM TX1550/RX1310" },
|
||||||
|
{ pid: "DEM-301T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DEM-431XT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "DEM-432XT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "DEM-433XT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "DEM-434XT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "DEM-435XT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM" },
|
||||||
|
{ pid: "DEM-CB100S-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T", notes: "Copper RJ45" },
|
||||||
|
{ pid: "DEM-436XT-BXU", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi-1270", fiberType: "SMF", connector: "LC", wavelengths: "1270/1330nm", notes: "BiDi TX1270" },
|
||||||
|
{ pid: "DEM-436XT-BXD", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi-1330", fiberType: "SMF", connector: "LC", wavelengths: "1330/1270nm", notes: "BiDi TX1330" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DEM-25GSR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "DEM-25GLR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DEM-Q40GSR", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "DEM-Q40MLR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DEM-Q28SR4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "DEM-Q28LR4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "DEM-Q28CWDM4", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DEM-CB100S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+", notes: "10G DAC 1m" },
|
||||||
|
{ pid: "DEM-CB300S", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "DEM-CB100Q", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP+" },
|
||||||
|
{ pid: "DEM-CB100C", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28", notes: "100G DAC 1m" },
|
||||||
|
{ pid: "DEM-CB300C", formFactor: "QSFP28",speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeDlinkOem(): Promise<void> {
|
||||||
|
console.log("=== D-Link OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"D-Link",
|
||||||
|
"oem",
|
||||||
|
"https://www.dlink.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of DLINK_PIDS) {
|
||||||
|
const slug = `dlink-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== D-Link OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${DLINK_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeDlinkOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
128
packages/scraper/src/scrapers/drivenets-oem.ts
Normal file
128
packages/scraper/src/scrapers/drivenets-oem.ts
Normal file
@ -0,0 +1,128 @@
|
|||||||
|
/**
|
||||||
|
* DriveNets OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds DriveNets-branded transceiver PIDs for the Network Cloud (DNOS)
|
||||||
|
* distributed routing NOS running on white-box hardware (Comcast/Lumen
|
||||||
|
* Network Cloud deployments, DNOS-on-COTS).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - DriveNets Network Cloud Hardware Guide (drivenets.com)
|
||||||
|
* - DriveNets DNOS Transceiver Compatibility Matrix
|
||||||
|
* - DriveNets 400G/800G Data Center Interconnect Deployment Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/drivenets-oem.ts
|
||||||
|
* Cron: daily at 15:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface DrivenetsPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── DriveNets OEM transceiver catalog ───────────────────────────────────────
|
||||||
|
// Source: DriveNets Network Cloud HCL and DNOS platform guides
|
||||||
|
const DRIVENETS_PIDS: DrivenetsPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DN-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "DriveNets 1G MM SFP for DNOS white-box management ports" },
|
||||||
|
{ pid: "DN-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "DriveNets 1G SM SFP 10km for DNOS Network Cloud mgmt" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DN-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "DriveNets 10G SR SFP+ for DNOS distributed router fabric" },
|
||||||
|
{ pid: "DN-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "DriveNets 10G LR SFP+ 10km for DNOS inter-cluster links" },
|
||||||
|
{ pid: "DN-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "DriveNets 10G ER 40km for DNOS metro DCI aggregation" },
|
||||||
|
{ pid: "DN-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "DriveNets 10G ZR 80km dark-fiber for DNOS edge PoP" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DN-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "DriveNets 25G SR SFP28 for DNOS white-box server downlinks" },
|
||||||
|
{ pid: "DN-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "DriveNets 25G LR SFP28 for DNOS Network Cloud fabric uplinks" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DN-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "DriveNets 100G SR4 QSFP28 for DNOS white-box 100G chassis" },
|
||||||
|
{ pid: "DN-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "DriveNets 100G LR4 QSFP28 for DNOS inter-PoP DCI" },
|
||||||
|
{ pid: "DN-QSFP28-100G-ZR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 80000, reachLabel: "ZR4", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "100G ZR4", category: "Telecom", notes: "DriveNets 100G ZR4 coherent QSFP28 for DNOS long-haul DCI" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DN-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "DriveNets 400G DR4 QSFP-DD for DNOS Network Cloud fabric" },
|
||||||
|
{ pid: "DN-QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "DriveNets 400G SR8 QSFP-DD for DNOS OM4 cluster fabric" },
|
||||||
|
{ pid: "DN-QSFP-DD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", category: "Telecom", notes: "DriveNets 400G ZR coherent QSFP-DD for DNOS open-line DCI" },
|
||||||
|
|
||||||
|
// ── 800G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DN-QSFP-DD-800G-DR8", formFactor: "QSFP-DD", speedGbps: 800, speed: "800G", reachMeters: 500, reachLabel: "DR8", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "800GBASE-DR8", notes: "DriveNets 800G DR8 QSFP-DD for next-gen DNOS hyper-scale fabric" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (coherent ZR/ZR4)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"DN-QSFP28-100G-ZR4",
|
||||||
|
"DN-QSFP-DD-400G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeDrivenetsOem(): Promise<void> {
|
||||||
|
console.log("=== DriveNets OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"DriveNets",
|
||||||
|
"oem",
|
||||||
|
"https://drivenets.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of DRIVENETS_PIDS) {
|
||||||
|
const slug = `drivenets-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== DriveNets OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${DRIVENETS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeDrivenetsOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
129
packages/scraper/src/scrapers/dzs-oem.ts
Normal file
129
packages/scraper/src/scrapers/dzs-oem.ts
Normal file
@ -0,0 +1,129 @@
|
|||||||
|
/**
|
||||||
|
* DZS OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds DZS-branded transceiver PIDs for FiberXtend, E-series, MXK,
|
||||||
|
* and MALC fiber access / broadband platforms (formerly Zhone Technologies).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - DZS Product Catalog (dzsi.com)
|
||||||
|
* - DZS MXK / MALC Hardware Reference Guides
|
||||||
|
* - DZS FiberXtend Platform Data Sheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/dzs-oem.ts
|
||||||
|
* Cron: daily at 13:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface DzsPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const DZS_PIDS: DzsPID[] = [
|
||||||
|
// ── 1G SFP — GPON / EPON OLT ────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-GPON-CLASS-B+", formFactor: "SFP", speedGbps: 1, speed: "GPON", reachMeters: 20000, reachLabel: "GPON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "G.984", notes: "DZS GPON OLT SFP Class B+" },
|
||||||
|
{ pid: "SFP-GPON-CLASS-C+", formFactor: "SFP", speedGbps: 1, speed: "GPON", reachMeters: 20000, reachLabel: "GPON-C+", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", notes: "DZS GPON OLT SFP Class C+" },
|
||||||
|
{ pid: "SFP-EPON-1G", formFactor: "SFP", speedGbps: 1, speed: "EPON", reachMeters: 20000, reachLabel: "EPON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "802.3ah", notes: "DZS EPON OLT SFP 1G/1G" },
|
||||||
|
|
||||||
|
// ── 1G SFP — Standard ────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ — XGS-PON OLT ───────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-XGS-PON", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "XGS-PON", fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "G.9807.1", notes: "DZS XGS-PON OLT SFP+" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ — Standard ──────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "SFP-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── DAC ──────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeDzsOem(): Promise<void> {
|
||||||
|
console.log("=== DZS OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"DZS",
|
||||||
|
"oem",
|
||||||
|
"https://www.dzsi.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of DZS_PIDS) {
|
||||||
|
const slug = `dzs-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== DZS OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${DZS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeDzsOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
139
packages/scraper/src/scrapers/edgewater-networks-oem.ts
Normal file
139
packages/scraper/src/scrapers/edgewater-networks-oem.ts
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
/**
|
||||||
|
* Edgewater Networks OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Edgewater Networks (Ribbon EdgeMarc) branded transceiver PIDs for
|
||||||
|
* session border controller (SBC) and enterprise edge platforms.
|
||||||
|
* Edgewater Networks / Ribbon EdgeMarc builds session border controllers
|
||||||
|
* and enterprise edge devices (EdgeMarc 6000/7000 series) used by UCaaS
|
||||||
|
* providers and enterprises for SIP trunking and unified communications.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Ribbon EdgeMarc Pluggable Module Compatibility Guide
|
||||||
|
* - Edgewater Networks SFP/QSFP Interface Data Sheets
|
||||||
|
* - ribboncommunications.com EdgeMarc product catalogue
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/edgewater-networks-oem.ts
|
||||||
|
* Cron: daily at 02:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface EdgewaterPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const EDGEWATER_PIDS: EdgewaterPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "Cu", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "Cu", connector: "RJ45", standard: "1000BASE-T", notes: "Edgewater GE copper SFP for EdgeMarc management ports" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom", notes: "Edgewater 10G ZR for extended SBC uplinks" },
|
||||||
|
|
||||||
|
// ── BiDi SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-BIDI-1310-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1550nm", category: "Telecom", notes: "Edgewater BiDi SFP 1310nm TX / 1550nm RX single-fibre" },
|
||||||
|
|
||||||
|
// ── SONET/SDH SFP (OC-3 / STM-1) ─────────────────────────────────────────
|
||||||
|
{ pid: "SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "OC3", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Edgewater OC-3 short-reach SFP for TDM-over-IP gateway" },
|
||||||
|
{ pid: "SFP-OC3-LR", formFactor: "SFP", speedGbps: 0.155, speed: "OC3", reachMeters: 15000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Edgewater OC-3 long-reach SFP for TDM-over-IP gateway" },
|
||||||
|
|
||||||
|
// ── SONET/SDH SFP (OC-12 / STM-4) ────────────────────────────────────────
|
||||||
|
{ pid: "SFP-OC12-SR", formFactor: "SFP", speedGbps: 0.622, speed: "OC12", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Edgewater OC-12 short-reach SFP" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1270-1330nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── DWDM SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DWDM-SFP-C", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "Edgewater ITU-T DWDM C-band SFP for WDM aggregation" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (SONET/SDH, DWDM, BiDi, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"SFP-10G-ZR",
|
||||||
|
"SFP-BIDI-1310-1550",
|
||||||
|
"SFP-OC3-SR",
|
||||||
|
"SFP-OC3-LR",
|
||||||
|
"SFP-OC12-SR",
|
||||||
|
"DWDM-SFP-C",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeEdgewaterNetworksOem(): Promise<void> {
|
||||||
|
console.log("=== Edgewater Networks OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Edgewater Networks",
|
||||||
|
"oem",
|
||||||
|
"https://www.ribboncommunications.com/products/enterprise-products/edgewater-networks",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of EDGEWATER_PIDS) {
|
||||||
|
const slug = `edgewater-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Edgewater Networks OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${EDGEWATER_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeEdgewaterNetworksOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
132
packages/scraper/src/scrapers/ekinops-oem.ts
Normal file
132
packages/scraper/src/scrapers/ekinops-oem.ts
Normal file
@ -0,0 +1,132 @@
|
|||||||
|
/**
|
||||||
|
* Ekinops OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Ekinops-branded transceiver PIDs for optical transport and access
|
||||||
|
* equipment platforms: 300 series, 360 series, and OneAccess platforms.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Ekinops 300 Series Line Card Datasheet
|
||||||
|
* - Ekinops 360 Series Transceiver Module Guide
|
||||||
|
* - Ekinops OneAccess Platform Product Portfolio (ekinops.com)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ekinops-oem.ts
|
||||||
|
* Cron: daily at 20:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface EkinopsPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const EKINOPS_PIDS: EkinopsPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EK-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "EK-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "EK-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EK-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "EK-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "EK-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "EK-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "EK-SFP-10G-DW-C-TUNE", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "Ekinops 10G DWDM tunable SFP+" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EK-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "EK-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "EK-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 Coherent ───────────────────────────────────────────────────
|
||||||
|
{ pid: "EK-CFP2-100G-DCO", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1500000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Ekinops 100G CFP2 DCO coherent" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EK-QSFPDD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "EK-QSFPDD-400G-LR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
{ pid: "EK-QSFPDD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", category: "Telecom", notes: "Ekinops 400G ZR coherent QSFP-DD" },
|
||||||
|
{ pid: "EK-QSFPDD-400G-ZRP", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 3000000, reachLabel: "ZR+", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Ekinops 400G ZR+ coherent QSFP-DD" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EK-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "EK-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (coherent/DWDM)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"EK-SFP-10G-DW-C-TUNE",
|
||||||
|
"EK-CFP2-100G-DCO",
|
||||||
|
"EK-QSFPDD-400G-ZR",
|
||||||
|
"EK-QSFPDD-400G-ZRP",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeEkinopsOem(): Promise<void> {
|
||||||
|
console.log("=== Ekinops OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Ekinops",
|
||||||
|
"oem",
|
||||||
|
"https://www.ekinops.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of EKINOPS_PIDS) {
|
||||||
|
const slug = `ekinops-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Ekinops OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${EKINOPS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeEkinopsOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
342
packages/scraper/src/scrapers/emulex-oem.ts
Normal file
342
packages/scraper/src/scrapers/emulex-oem.ts
Normal file
@ -0,0 +1,342 @@
|
|||||||
|
/**
|
||||||
|
* Emulex / Broadcom OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Emulex-branded transceiver PIDs for Fibre Channel HBAs and
|
||||||
|
* FCoE/Ethernet CNAs. Emulex is now owned by Broadcom but continues
|
||||||
|
* shipping adapters under the Emulex brand in FC storage markets.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Broadcom FC Networking products (broadcom.com/products/fibre-channel-networking)
|
||||||
|
* - Emulex LightPulse HBA optical transceiver guides
|
||||||
|
* - SFF-8024 Fibre Channel speed codes
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/emulex-oem.ts
|
||||||
|
* Cron: daily at 05:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface EmulexPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fibre Channel line-rate speeds (Gbps):
|
||||||
|
// 4G = 4.25 Gbps (2x FC)
|
||||||
|
// 8G = 8.5 Gbps (4x FC)
|
||||||
|
// 16G = 14.025 Gbps (8x FC)
|
||||||
|
// 32G = 28.05 Gbps (16x FC)
|
||||||
|
// 64G = 56.1 Gbps (32x FC)
|
||||||
|
const EMULEX_PIDS: EmulexPID[] = [
|
||||||
|
// ── 4G FC SFP ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-4G-FC-SW",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 4.25,
|
||||||
|
speed: "4G FC",
|
||||||
|
reachMeters: 380,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "4GFC",
|
||||||
|
notes: "4G FC short-wave; OM2/OM3 MMF; Emulex LightPulse LP1150/LP11002 HBAs",
|
||||||
|
},
|
||||||
|
// ── 8G FC SFP ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-8G-FC-SW",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC short-wave; OM3/OM4 MMF; Emulex LightPulse LPe12000 HBAs",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-8G-FC-LW",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC long-wave; OS2 SMF; extended-distance SAN trunks",
|
||||||
|
},
|
||||||
|
// ── 16G FC SFP ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-16G-FC-SW",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 125,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC short-wave; OM4 MMF; Emulex LightPulse LPe16000 HBAs",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-16G-FC-LW",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC long-wave; OS2 SMF; inter-building SAN connectivity",
|
||||||
|
},
|
||||||
|
// ── 32G FC SFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-32G-FC-SW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 28.05,
|
||||||
|
speed: "32G FC",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "32GFC",
|
||||||
|
notes: "32G FC short-wave; OM4/OM5 MMF; Emulex LightPulse LPe32000 HBAs",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-32G-FC-LW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 28.05,
|
||||||
|
speed: "32G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "32GFC",
|
||||||
|
notes: "32G FC long-wave; OS2 SMF; campus SAN interconnect",
|
||||||
|
},
|
||||||
|
// ── 64G FC SFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP28-64G-FC-SW",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 56.1,
|
||||||
|
speed: "64G FC",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "64GFC",
|
||||||
|
notes: "64G FC short-wave; OM4/OM5 MMF; Broadcom/Emulex LPe36000 NVMe/FC HBAs",
|
||||||
|
},
|
||||||
|
// ── FCoE SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-FCoE-10G-SR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
notes: "10GbE SFP+ SR for Emulex OneConnect OCe10100 FCoE CNA",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-FCoE-10G-LR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
notes: "10GbE SFP+ LR for Emulex OneConnect FCoE CNA",
|
||||||
|
},
|
||||||
|
// ── 10G Ethernet SFP+ ────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-10G-SR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
notes: "10GbE SFP+ SR for Emulex OneConnect 10GbE NIC",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-10G-LR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
notes: "10GbE SFP+ LR for Emulex OneConnect 10GbE NIC",
|
||||||
|
},
|
||||||
|
// ── 25G Ethernet SFP28 ───────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-25G-SR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "25GBASE-SR",
|
||||||
|
notes: "25GbE SFP28 SR for Broadcom/Emulex Ethernet CNA",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-25G-LR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "25GBASE-LR",
|
||||||
|
notes: "25GbE SFP28 LR for Broadcom/Emulex Ethernet CNA",
|
||||||
|
},
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-QSFP28-100G-SR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "100GBASE-SR4",
|
||||||
|
notes: "100GbE QSFP28 SR4; Broadcom/Emulex 100G CNA modules",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "EML-QSFP28-100G-LR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-LR4",
|
||||||
|
notes: "100GbE QSFP28 LR4; Broadcom/Emulex 100G CNA modules",
|
||||||
|
},
|
||||||
|
// ── 1G Ethernet SFP ──────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-1G-SX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 550,
|
||||||
|
reachLabel: "SX",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "1000BASE-SX",
|
||||||
|
notes: "1GbE SFP SX for Emulex management and iSCSI ports",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "EML-SFP-1G-LX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LX",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "1000BASE-LX",
|
||||||
|
notes: "1GbE SFP LX for Emulex management and iSCSI ports",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeEmulexOem(): Promise<void> {
|
||||||
|
console.log("=== Emulex / Broadcom OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Emulex",
|
||||||
|
"oem",
|
||||||
|
"https://www.broadcom.com/products/fibre-channel-networking/",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of EMULEX_PIDS) {
|
||||||
|
const slug = `emulex-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Emulex OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${EMULEX_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeEmulexOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
138
packages/scraper/src/scrapers/ericsson-oem.ts
Normal file
138
packages/scraper/src/scrapers/ericsson-oem.ts
Normal file
@ -0,0 +1,138 @@
|
|||||||
|
/**
|
||||||
|
* Ericsson OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Ericsson-branded transceiver PIDs for MINI-LINK, Router 6000,
|
||||||
|
* and SAOS-based platforms (formerly Corecess/Zhone).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Ericsson Transceiver Module Product Guide (ericsson.com)
|
||||||
|
* - Ericsson Router 6000 Hardware Guide
|
||||||
|
* - Ericsson SAOS Platform Hardware Reference
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ericsson-oem.ts
|
||||||
|
* Cron: daily at 06:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface EricssonPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ERICSSON_PIDS: EricssonPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ROF137 R1A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Ericsson 1G SX SFP" },
|
||||||
|
{ pid: "ROF137 R2A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Ericsson 1G LX SFP" },
|
||||||
|
{ pid: "ROF137 R3A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH40", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
{ pid: "ROF137 R4A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "ROF137 R5A", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ROF155 R1A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Ericsson 10G SR SFP+" },
|
||||||
|
{ pid: "ROF155 R2A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Ericsson 10G LR SFP+" },
|
||||||
|
{ pid: "ROF155 R3A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "ROF155 R4A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "ROF155 R5A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM" },
|
||||||
|
{ pid: "ROF155 R6A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "ROF155 D1A", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "10G DWDM tunable SFP+ for Router 6000" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ROF166 R1A", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "ROF166 R2A", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "ROF166 R3A", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ROF177 R1A", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "ROF177 R2A", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
{ pid: "ROF177 R3A", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "PLR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", notes: "PSM4 parallel single-mode" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ROF188 R1A", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "ROF188 R2A", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "ROF188 R3A", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "ROF188 R4A", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
{ pid: "ROF188 R5A", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "FR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-FR" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ROF199 R1A", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "ROF199 R2A", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "ROF199 R3A", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "ROF199 R4A", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ROF155 D10-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M",fiberType: "DAC", connector: "SFP+", notes: "10G DAC 1m" },
|
||||||
|
{ pid: "ROF155 D10-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M",fiberType: "DAC", connector: "SFP+", notes: "10G DAC 3m" },
|
||||||
|
{ pid: "ROF188 D100-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M",fiberType: "DAC", connector: "QSFP28", notes: "100G DAC 1m" },
|
||||||
|
{ pid: "ROF188 D100-3M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M",fiberType: "DAC", connector: "QSFP28", notes: "100G DAC 3m" },
|
||||||
|
{ pid: "ROF199 D400-1M", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 1, reachLabel: "DAC-1M",fiberType: "DAC", connector: "QSFP-DD",notes: "400G DAC 1m" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeEricssonOem(): Promise<void> {
|
||||||
|
console.log("=== Ericsson OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Ericsson",
|
||||||
|
"oem",
|
||||||
|
"https://www.ericsson.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ERICSSON_PIDS) {
|
||||||
|
const slug = `ericsson-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Ericsson OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ERICSSON_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeEricssonOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
135
packages/scraper/src/scrapers/ericsson-ran-oem.ts
Normal file
135
packages/scraper/src/scrapers/ericsson-ran-oem.ts
Normal file
@ -0,0 +1,135 @@
|
|||||||
|
/**
|
||||||
|
* Ericsson Radio OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Ericsson Radio System-branded transceiver PIDs used in CPRI and
|
||||||
|
* eCPRI fronthaul deployments for Ericsson RRUS (Remote Radio Unit System),
|
||||||
|
* RADIO (AIR series), and baseband (BBU6630/BBU6648/vDU) units.
|
||||||
|
*
|
||||||
|
* CPRI option / speed mapping:
|
||||||
|
* Option 1/2: 614.4 Mbps | Option 3: 1228.8 Mbps
|
||||||
|
* Option 4: 2457.6 Mbps | Option 5: 4915.2 Mbps
|
||||||
|
* Option 6: 9830.4 Mbps | Option 7: 10137.6 Mbps
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Ericsson Radio System product portfolio (ericsson.com)
|
||||||
|
* - CPRI Specification v7.0
|
||||||
|
* - eCPRI Specification v2.0
|
||||||
|
* - Ericsson 5G NR fronthaul transport guidelines
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ericsson-ran-oem.ts
|
||||||
|
* Cron: daily at 09:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface EricssonRanPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ERICSSON_RAN_PIDS: EricssonRanPID[] = [
|
||||||
|
// ── CPRI Option 1/2 — 614.4 Mbps ────────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP-CPRI-614M", formFactor: "SFP", speedGbps: 0.6144, speed: "0.6G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Ericsson RRUS CPRI option 1 (614.4 Mbps) / option 2 SFP — narrow-band LTE remote radio" },
|
||||||
|
|
||||||
|
// ── CPRI Option 3 — 1228.8 Mbps ─────────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP-CPRI-1228M", formFactor: "SFP", speedGbps: 1.2288, speed: "1.2G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Ericsson RRUS CPRI option 3 (1228.8 Mbps) SFP — 2x10 MHz LTE carrier fronthaul" },
|
||||||
|
|
||||||
|
// ── CPRI Option 4 — 2457.6 Mbps ─────────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP-CPRI-2457M", formFactor: "SFP", speedGbps: 2.4576, speed: "2.5G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OTN", notes: "Ericsson RRUS CPRI option 4 (2457.6 Mbps) SFP — 4x10 MHz LTE carrier fronthaul" },
|
||||||
|
|
||||||
|
// ── CPRI Option 5 — 4915.2 Mbps ─────────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP-CPRI-4915M", formFactor: "SFP", speedGbps: 4.9152, speed: "4.9G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OTN", notes: "Ericsson RRUS CPRI option 5 (4915.2 Mbps) SFP — 4x20 MHz LTE carrier, typical RRU 2T2R" },
|
||||||
|
|
||||||
|
// ── CPRI Option 6 — 9830.4 Mbps ─────────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP-CPRI-9830M", formFactor: "SFP+", speedGbps: 9.83, speed: "9.8G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Ericsson RRUS CPRI option 6 (9830.4 Mbps) SFP+ — 4T4R LTE or 2T2R 4-carrier fronthaul" },
|
||||||
|
|
||||||
|
// ── CPRI Option 7 — 10137.6 Mbps ────────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP-CPRI-10G-SR", formFactor: "SFP+", speedGbps: 10.137, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Ericsson RRUS CPRI option 7 (10137.6 Mbps) SFP+ SR — high-capacity LTE/5G NR fronthaul" },
|
||||||
|
{ pid: "ER-SFP-CPRI-10G-LR", formFactor: "SFP+", speedGbps: 10.137, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Ericsson RRUS CPRI option 7 SFP+ LR — long-reach fronthaul up to 10 km" },
|
||||||
|
|
||||||
|
// ── eCPRI — 25G SFP28 ───────────────────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP28-ECPRI-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Ericsson AIR eCPRI 25G SFP28 SR for 5G NR O-RAN split 7-2x fronthaul" },
|
||||||
|
{ pid: "ER-SFP28-ECPRI-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Ericsson AIR eCPRI 25G SFP28 LR for long-reach 5G NR fronthaul" },
|
||||||
|
|
||||||
|
// ── eCPRI — 100G QSFP28 ─────────────────────────────────────────────────
|
||||||
|
{ pid: "ER-QSFP28-ECPRI-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Ericsson AIR eCPRI 100G QSFP28 SR4 for massive MIMO (64T64R) aggregated fronthaul" },
|
||||||
|
|
||||||
|
// ── Standard data-plane SFP+ ────────────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Ericsson Radio System standard 10G SFP+ SR for BBU uplink / transport" },
|
||||||
|
{ pid: "ER-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G / 100G data-plane ───────────────────────────────────────────────
|
||||||
|
{ pid: "ER-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
{ pid: "ER-QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Ericsson Radio 100G QSFP28 LR4 for midhaul/backhaul aggregation" },
|
||||||
|
|
||||||
|
// ── 1G SFP management / cell site ───────────────────────────────────────
|
||||||
|
{ pid: "ER-SFP-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "ER-SFP-GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeEricssonRanOem(): Promise<void> {
|
||||||
|
console.log("=== Ericsson Radio OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Ericsson Radio",
|
||||||
|
"oem",
|
||||||
|
"https://www.ericsson.com/en/portfolio/networks/ericsson-radio-system",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ERICSSON_RAN_PIDS) {
|
||||||
|
const slug = `ericsson-r-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Ericsson Radio OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ERICSSON_RAN_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeEricssonRanOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
155
packages/scraper/src/scrapers/ericsson-transport-oem.ts
Normal file
155
packages/scraper/src/scrapers/ericsson-transport-oem.ts
Normal file
@ -0,0 +1,155 @@
|
|||||||
|
/**
|
||||||
|
* Ericsson Transport OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Ericsson-branded transceiver PIDs for MINI-LINK transport and
|
||||||
|
* IP/optical networking platforms: MINI-LINK 6352, 6372, TN, E, and
|
||||||
|
* Ericsson DWDM/coherent optics for ASON/WSON networks.
|
||||||
|
*
|
||||||
|
* Note: ericsson-oem.ts covers general Ericsson router/switch PIDs.
|
||||||
|
* This file focuses on transport, telecom, and MINI-LINK specific optics
|
||||||
|
* including SONET/SDH, OTN, CFP2, and tunable DWDM variants.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Ericsson MINI-LINK Product Portfolio (ericsson.com/en/portfolio/networks/transport)
|
||||||
|
* - Ericsson Optical Networking Hardware Overview
|
||||||
|
* - Ericsson MINI-LINK 6352/6372 Transceiver Compatibility Guide
|
||||||
|
* - Ericsson OTN/SDH Platform Optics Specification
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ericsson-transport-oem.ts
|
||||||
|
* Cron: daily at 00:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface EricssonTransportPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ERICSSON_TRANSPORT_PIDS: EricssonTransportPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "1000BASE-T", fiberType: "Cu", connector: "RJ45" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom" },
|
||||||
|
|
||||||
|
// ── SDH/SONET SFP (MINI-LINK transport) ──────────────────────────────────
|
||||||
|
{ pid: "SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "OC-3/STM-1", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Ericsson MINI-LINK OC-3 / STM-1 short reach" },
|
||||||
|
{ pid: "SFP-OC3-LR", formFactor: "SFP", speedGbps: 0.155, speed: "OC-3/STM-1", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Ericsson MINI-LINK OC-3 / STM-1 long reach" },
|
||||||
|
{ pid: "SFP-STM1-SR", formFactor: "SFP", speedGbps: 0.155, speed: "STM-1", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "STM-1", category: "Telecom", notes: "Ericsson SDH STM-1 short reach SFP" },
|
||||||
|
{ pid: "SFP-OC12-SR", formFactor: "SFP", speedGbps: 0.622, speed: "OC-12/STM-4", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Ericsson MINI-LINK OC-12 / STM-4 short reach" },
|
||||||
|
{ pid: "SFP-OC12-LR", formFactor: "SFP", speedGbps: 0.622, speed: "OC-12/STM-4", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Ericsson MINI-LINK OC-12 / STM-4 long reach" },
|
||||||
|
{ pid: "SFP-STM4-LR", formFactor: "SFP", speedGbps: 0.622, speed: "STM-4", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "STM-4", category: "Telecom", notes: "Ericsson SDH STM-4 long reach SFP" },
|
||||||
|
{ pid: "SFP-OC48-SR", formFactor: "SFP", speedGbps: 2.488, speed: "OC-48/STM-16", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-48/STM-16", category: "Telecom", notes: "Ericsson MINI-LINK OC-48 / STM-16 short reach" },
|
||||||
|
{ pid: "SFP-STM16-LR", formFactor: "SFP", speedGbps: 2.488, speed: "STM-16", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "STM-16", category: "Telecom", notes: "Ericsson SDH STM-16 long reach SFP" },
|
||||||
|
|
||||||
|
// ── OTN SFP+ ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-OTU2-10G", formFactor: "SFP+", speedGbps: 10, speed: "OTU2", reachMeters: 40000, reachLabel: "OTU2", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OTU2", category: "Telecom", notes: "Ericsson 10G OTN OTU2 SFP+ for MINI-LINK TN" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1270-1330nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 Coherent ───────────────────────────────────────────────────
|
||||||
|
{ pid: "CFP2-100G-LR4", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", category: "Telecom", notes: "Ericsson CFP2 100G LR4 for optical transport" },
|
||||||
|
|
||||||
|
// ── DWDM SFP+ tunable ───────────────────────────────────────────────────
|
||||||
|
{ pid: "DWDM-SFP-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "Ericsson C-band DWDM SFP+ fixed-wavelength for WDM transport" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD coherent ZR ─────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", category: "Telecom", notes: "Ericsson 400G ZR coherent QSFP-DD for DWDM long haul" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (SDH/SONET, OTN, DWDM, ZR, CFP2)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"SFP-10G-ZR",
|
||||||
|
"SFP-OC3-SR", "SFP-OC3-LR",
|
||||||
|
"SFP-STM1-SR",
|
||||||
|
"SFP-OC12-SR", "SFP-OC12-LR",
|
||||||
|
"SFP-STM4-LR",
|
||||||
|
"SFP-OC48-SR",
|
||||||
|
"SFP-STM16-LR",
|
||||||
|
"SFP-OTU2-10G",
|
||||||
|
"CFP2-100G-LR4",
|
||||||
|
"DWDM-SFP-C",
|
||||||
|
"QSFP-DD-400G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeEricssonTransportOem(): Promise<void> {
|
||||||
|
console.log("=== Ericsson Transport OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Ericsson",
|
||||||
|
"oem",
|
||||||
|
"https://www.ericsson.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ERICSSON_TRANSPORT_PIDS) {
|
||||||
|
const slug = `ericsson-t-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Ericsson Transport OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ERICSSON_TRANSPORT_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeEricssonTransportOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
116
packages/scraper/src/scrapers/etherwan-oem.ts
Normal file
116
packages/scraper/src/scrapers/etherwan-oem.ts
Normal file
@ -0,0 +1,116 @@
|
|||||||
|
/**
|
||||||
|
* EtherWAN OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for EtherWAN industrial-grade SFP modules used in
|
||||||
|
* EX/FX series managed switches for harsh environment deployments.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - EtherWAN SFP module product pages (etherwan.com)
|
||||||
|
* - EtherWAN EX series hardware datasheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/etherwan-oem.ts
|
||||||
|
* Cron: daily at 16:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface EtherwanPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ETHERWAN_PIDS: EtherwanPID[] = [
|
||||||
|
// ── 100M SFP ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EX44018", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 2000, reachLabel: "FX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "100BASE-FX" },
|
||||||
|
{ pid: "EX44019", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 15000, reachLabel: "LFX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100BASE-LFX" },
|
||||||
|
{ pid: "EX44020", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 40000, reachLabel: "LFX-40K", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "EtherWAN 100M SM SFP 40km" },
|
||||||
|
{ pid: "EX44021", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 20000, reachLabel: "BiDi-20K",fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550", notes: "EtherWAN 100M BiDi SFP" },
|
||||||
|
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EX45000", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "EX45001", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "EX45002", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "LH20", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "EtherWAN 1G SM SFP 20km" },
|
||||||
|
{ pid: "EX45003", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH40", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "EtherWAN 1G SM SFP 40km" },
|
||||||
|
{ pid: "EX45004", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "EX45005", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "EX45006", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-U", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550", notes: "EtherWAN 1G BiDi SFP upstream" },
|
||||||
|
{ pid: "EX45007", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-D", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310", notes: "EtherWAN 1G BiDi SFP downstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EX46000", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "EX46001", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "EX46002", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "EX46003", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "EX46004", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi-U", fiberType: "SMF", connector: "LC", wavelengths: "TX1270/RX1330", notes: "EtherWAN 10G BiDi SFP+ upstream" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EX46010", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "EX46011", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeEtherwanOem(): Promise<void> {
|
||||||
|
console.log("=== EtherWAN OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"EtherWAN Systems",
|
||||||
|
"oem",
|
||||||
|
"https://www.etherwan.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ETHERWAN_PIDS) {
|
||||||
|
const slug = `etherwan-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== EtherWAN OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ETHERWAN_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeEtherwanOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
139
packages/scraper/src/scrapers/evertz-oem.ts
Normal file
139
packages/scraper/src/scrapers/evertz-oem.ts
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
/**
|
||||||
|
* Evertz Microsystems OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Evertz-branded transceiver PIDs for broadcast infrastructure:
|
||||||
|
* EXE modular chassis, 570IPG/7700 series SDI-over-IP, and MAGNUM SDN
|
||||||
|
* orchestration platforms.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Evertz Product Catalog (evertz.com)
|
||||||
|
* - Evertz EXE-1000/EXE-2000 Hardware Reference
|
||||||
|
* - Evertz 570IPG-X19-25G Platform Transceiver Guide
|
||||||
|
* - Evertz MAGNUM SDN Controller Compatibility Notes
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/evertz-oem.ts
|
||||||
|
* Cron: daily at 18:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface EvertzPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const EVERTZ_PIDS: EvertzPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EVZ-SFP-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Evertz 1G SX SFP for EXE chassis and 7700 series" },
|
||||||
|
{ pid: "EVZ-SFP-GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Evertz 1G LX SFP for EXE chassis management" },
|
||||||
|
{ pid: "EVZ-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Evertz 1G copper SFP for out-of-band management" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EVZ-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "EVZ-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "EVZ-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "EVZ-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom", notes: "Evertz 10G ZR SFP+ for broadcast contribution over dark fiber" },
|
||||||
|
|
||||||
|
// ── SONET/SDH SFP (SDI-over-fiber / broadcast contribution) ─────────────
|
||||||
|
{ pid: "EVZ-SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "155M", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Evertz OC-3 SFP for SDI-over-fiber broadcast ring" },
|
||||||
|
{ pid: "EVZ-SFP-OC12-SR", formFactor: "SFP", speedGbps: 0.622, speed: "622M", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Evertz OC-12 SFP for broadcast contribution networks" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EVZ-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Evertz 25G SR for 570IPG-X19-25G SDI-over-IP blade" },
|
||||||
|
{ pid: "EVZ-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Evertz 25G LR for long-reach SDI-over-IP links" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EVZ-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "EVZ-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EVZ-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "EVZ-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 (broadcast 100G contribution) ─────────────────────────────
|
||||||
|
{ pid: "EVZ-CFP2-100G-LR4", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", category: "Telecom", notes: "Evertz CFP2 100G LR4 for high-density broadcast contribution" },
|
||||||
|
|
||||||
|
// ── DWDM SFP (broadcast DWDM contribution ring) ──────────────────────────
|
||||||
|
{ pid: "EVZ-DWDM-SFP-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Evertz 10G DWDM SFP+ tunable C-band for broadcast contribution DWDM ring" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EVZ-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Evertz 400G DR4 for next-gen MAGNUM SDN fabric interconnect" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (SONET/SDH, DWDM, ZR, CFP2)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"EVZ-SFP-10G-ZR",
|
||||||
|
"EVZ-SFP-OC3-SR",
|
||||||
|
"EVZ-SFP-OC12-SR",
|
||||||
|
"EVZ-CFP2-100G-LR4",
|
||||||
|
"EVZ-DWDM-SFP-C",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeEvertzOem(): Promise<void> {
|
||||||
|
console.log("=== Evertz Microsystems OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Evertz Microsystems",
|
||||||
|
"oem",
|
||||||
|
"https://www.evertz.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of EVERTZ_PIDS) {
|
||||||
|
const slug = `evertz-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Evertz OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${EVERTZ_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeEvertzOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
135
packages/scraper/src/scrapers/exfo-network-oem.ts
Normal file
135
packages/scraper/src/scrapers/exfo-network-oem.ts
Normal file
@ -0,0 +1,135 @@
|
|||||||
|
/**
|
||||||
|
* EXFO Network OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds EXFO-branded transceiver PIDs for EXFO network test and
|
||||||
|
* service assurance platforms (FTB, EX, and OTDR series).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - EXFO Network Test Instruments (exfo.com)
|
||||||
|
* - EXFO FTBx Platform Module Data Sheets
|
||||||
|
* - EXFO Service Assurance Platform Hardware Guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/exfo-network-oem.ts
|
||||||
|
* Cron: daily at 10:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface ExfoNetworkPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs requiring TELECOM/ITU-T classification (CFP2, OC-3/12/48, ZR, ER4)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"EXFO-CFP2-100G-LR4",
|
||||||
|
"EXFO-SFP-OC3-SR",
|
||||||
|
"EXFO-SFP-OC12-SR",
|
||||||
|
"EXFO-SFP-OC48-SR",
|
||||||
|
"EXFO-SFP-10G-ZR",
|
||||||
|
"EXFO-QSFP28-100G-ER4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const EXFO_NETWORK_PIDS: ExfoNetworkPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EXFO-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "EXFO-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EXFO-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "EXFO-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "EXFO-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "EXFO-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "DWDM extended reach, ITU-T G.694.1" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EXFO-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "EXFO-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EXFO-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "EXFO-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EXFO-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "EXFO-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "EXFO-QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "Extended reach, ITU-T G.694 WDM" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "EXFO-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 (Telecom) ──────────────────────────────────────────────────
|
||||||
|
{ pid: "EXFO-CFP2-100G-LR4", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "CFP2 MSA, OTU4, ITU-T G.709" },
|
||||||
|
|
||||||
|
// ── SONET/SDH SFP (Telecom) ──────────────────────────────────────────────
|
||||||
|
{ pid: "EXFO-SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "OC-3", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", notes: "155 Mbps SONET OC-3 / SDH STM-1" },
|
||||||
|
{ pid: "EXFO-SFP-OC12-SR", formFactor: "SFP", speedGbps: 0.622, speed: "OC-12", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", notes: "622 Mbps SONET OC-12 / SDH STM-4" },
|
||||||
|
{ pid: "EXFO-SFP-OC48-SR", formFactor: "SFP", speedGbps: 2.488, speed: "OC-48", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-48/STM-16", notes: "2.488 Gbps SONET OC-48 / SDH STM-16" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeExfoNetworkOem(): Promise<void> {
|
||||||
|
console.log("=== EXFO Network OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"EXFO Network",
|
||||||
|
"oem",
|
||||||
|
"https://www.exfo.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of EXFO_NETWORK_PIDS) {
|
||||||
|
const slug = `exfo-net-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== EXFO Network OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${EXFO_NETWORK_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeExfoNetworkOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
144
packages/scraper/src/scrapers/exfo-oem.ts
Normal file
144
packages/scraper/src/scrapers/exfo-oem.ts
Normal file
@ -0,0 +1,144 @@
|
|||||||
|
/**
|
||||||
|
* EXFO OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds EXFO-branded transceiver PIDs used with optical test equipment
|
||||||
|
* and network deployment platforms (FTB series, T-BERD/MTS).
|
||||||
|
* Covers DataCenter and Telecom (coherent/DWDM) form factors.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - EXFO FTB-1 Pro Modular Test Platform Hardware Guide
|
||||||
|
* - EXFO T-BERD/MTS Platform Transceiver Compatibility List
|
||||||
|
* - EXFO Optical Transceiver Portfolio (exfo.com)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/exfo-oem.ts
|
||||||
|
* Cron: daily at 14:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface ExfoPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const EXFO_PIDS: ExfoPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-SFP-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "FTB-SFP-GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "FTB-SFP-GE-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "FTB-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "FTB-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "FTB-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "FTB-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-SFP-10G-DW-TUNE", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "EXFO 10G DWDM tunable SFP+" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "FTB-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "FTB-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "FTB-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "FTB-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "FTB-QSFP28-100G-DR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 Coherent ───────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-CFP2-100G-DCO", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1500000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "EXFO 100G CFP2 DCO coherent test transceiver" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-QSFPDD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "FTB-QSFPDD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "FTB-QSFPDD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "FTB-QSFPDD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", category: "Telecom", notes: "EXFO 400G ZR coherent QSFP-DD" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FTB-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "FTB-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "FTB-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "FTB-DAC-400G-1M", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP-DD" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (coherent/DWDM)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"FTB-SFP-10G-DW-TUNE",
|
||||||
|
"FTB-CFP2-100G-DCO",
|
||||||
|
"FTB-QSFPDD-400G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeExfoOem(): Promise<void> {
|
||||||
|
console.log("=== EXFO OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"EXFO",
|
||||||
|
"oem",
|
||||||
|
"https://www.exfo.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of EXFO_PIDS) {
|
||||||
|
const slug = `exfo-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== EXFO OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${EXFO_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeExfoOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
122
packages/scraper/src/scrapers/extreme-campus-oem.ts
Normal file
122
packages/scraper/src/scrapers/extreme-campus-oem.ts
Normal file
@ -0,0 +1,122 @@
|
|||||||
|
/**
|
||||||
|
* Extreme Campus OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Extreme Networks campus-branded transceiver PIDs for EXOS and
|
||||||
|
* VOSS platforms including X440-G2, X450-G2, X460-G2, X620, X690,
|
||||||
|
* X870, and VSP series switches.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Extreme Networks Optics Compatibility Matrix
|
||||||
|
* - Extreme Networks Transceiver Module Data Sheet
|
||||||
|
* - Extreme Networks Summit Switch Optics Selection Guide
|
||||||
|
* - ExtremeXOS Hardware Compatibility Table (EXOS HCT)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/extreme-campus-oem.ts
|
||||||
|
* Cron: daily at 14:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface ExtremeCampusPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const EXTREME_CAMPUS_PIDS: ExtremeCampusPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "10301", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Extreme Networks 1GbE SX SFP 550m MMF (PN 10301)" },
|
||||||
|
{ pid: "10302", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Extreme Networks 1GbE LX SFP 10km SMF (PN 10302)" },
|
||||||
|
{ pid: "10303", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Extreme Networks 1GbE copper SFP RJ45 (PN 10303)" },
|
||||||
|
{ pid: "10303T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Extreme Networks 1GbE copper SFP RJ45 alt SKU (PN 10303T)" },
|
||||||
|
{ pid: "10304", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 70000, reachLabel: "ZX-70km", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "Extreme Networks 1GbE ZX SFP 70km SMF (PN 10304)" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "10305", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Extreme Networks 10GbE SR SFP+ 300m MMF (PN 10305)" },
|
||||||
|
{ pid: "10306", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Extreme Networks 10GbE LR SFP+ 10km SMF (PN 10306)" },
|
||||||
|
{ pid: "10307", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Extreme Networks 10GbE ER SFP+ 40km SMF (PN 10307)" },
|
||||||
|
{ pid: "10308", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Extreme Networks 10GbE ZR SFP+ 80km SMF (PN 10308)" },
|
||||||
|
{ pid: "10309", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM", notes: "Extreme Networks 10GbE LRM SFP+ 220m MMF (PN 10309)" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "10321", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Extreme Networks 40GbE SR4 QSFP+ 150m MMF (PN 10321)" },
|
||||||
|
{ pid: "10322", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", notes: "Extreme Networks 40GbE LR4 QSFP+ 10km SMF (PN 10322)" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "10323", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Extreme Networks 25GbE SR SFP28 100m MMF (PN 10323)" },
|
||||||
|
{ pid: "10324", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Extreme Networks 25GbE LR SFP28 10km SMF (PN 10324)" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "10331", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Extreme Networks 100GbE SR4 QSFP28 100m MMF (PN 10331)" },
|
||||||
|
{ pid: "10332", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Extreme Networks 100GbE LR4 QSFP28 10km SMF (PN 10332)" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "10340", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Extreme Networks 400GbE DR4 QSFP-DD 500m SMF (PN 10340)" },
|
||||||
|
{ pid: "10341", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "Extreme Networks 400GbE SR8 QSFP-DD 100m MMF (PN 10341)" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeExtremeCampusOem(): Promise<void> {
|
||||||
|
console.log("=== Extreme Campus OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Extreme Campus",
|
||||||
|
"oem",
|
||||||
|
"https://www.extremenetworks.com/products/switches/",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of EXTREME_CAMPUS_PIDS) {
|
||||||
|
const slug = `extreme-campus-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Extreme Campus OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${EXTREME_CAMPUS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeExtremeCampusOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
120
packages/scraper/src/scrapers/f5-oem.ts
Normal file
120
packages/scraper/src/scrapers/f5-oem.ts
Normal file
@ -0,0 +1,120 @@
|
|||||||
|
/**
|
||||||
|
* F5 Networks OEM Transceiver Catalog Seed
|
||||||
|
* Seeds F5-branded transceiver PIDs for BIG-IP i-series, rSeries, and VELOS chassis.
|
||||||
|
* Sources: F5 Hardware and Software Compatibility Matrix (support.f5.com)
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/f5-oem.ts
|
||||||
|
* Cron: daily at 07:30
|
||||||
|
*/
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface F5PID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const F5_PIDS: F5PID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "OPT-0010-00", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "OPT-0011-00", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "OPT-0016-00", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "OPT-0019-00", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "OPT-0030-00", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "OPT-0031-00", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "OPT-0032-00", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "OPT-0033-00", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "OPT-0039-00", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "OPT-0050-00", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "OPT-0051-00", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "OPT-0040-00", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "OPT-0041-00", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "OPT-0060-00", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "OPT-0061-00", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "OPT-0062-00", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "OPT-0063-00", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD (VELOS chassis) ────────────────────────────────────────
|
||||||
|
{ pid: "OPT-0080-00", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "OPT-0081-00", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "OPT-0082-00", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "CBL-0010-01", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "CBL-0010-03", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "CBL-0060-01", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "CBL-0060-03", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeF5Oem(): Promise<void> {
|
||||||
|
console.log("=== F5 Networks OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"F5 Networks",
|
||||||
|
"oem",
|
||||||
|
"https://www.f5.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of F5_PIDS) {
|
||||||
|
const slug = `f5-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== F5 Networks OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${F5_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeF5Oem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
144
packages/scraper/src/scrapers/fiberhome-oem.ts
Normal file
144
packages/scraper/src/scrapers/fiberhome-oem.ts
Normal file
@ -0,0 +1,144 @@
|
|||||||
|
/**
|
||||||
|
* FiberHome Technologies OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds FiberHome-branded transceiver PIDs covering PON access, metro
|
||||||
|
* transport, DWDM/CWDM, and data-center switching platforms from
|
||||||
|
* FiberHome Technologies Group (Wuhan, China).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - FiberHome Optical Transceiver Product Overview (fiberhome.com)
|
||||||
|
* - FiberHome AN6000 / TN55 / TN22 Compatibility Guide
|
||||||
|
* - FiberHome PON OLT Transceiver Specification Sheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/fiberhome-oem.ts
|
||||||
|
* Cron: daily at 19:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface FiberhomePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
isTelecom?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that belong to TELECOM category (PON, BiDi, CWDM, DWDM, CFP2-DCO, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"FH-SFP-GPON-OLT",
|
||||||
|
"FH-SFP-XGS-PON-OLT",
|
||||||
|
"FH-SFP-GE-BIDI-1310",
|
||||||
|
"FH-SFP-CWDM-1550",
|
||||||
|
"FH-DWDM-SFP-C",
|
||||||
|
"FH-CFP2-DCO-100G",
|
||||||
|
"FH-SFP-10G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const FIBERHOME_PIDS: FiberhomePID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-SFP-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "FiberHome 1G SFP SX 550m MMF" },
|
||||||
|
{ pid: "FH-SFP-GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "FiberHome 1G SFP LX 10km SMF" },
|
||||||
|
{ pid: "FH-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "FiberHome 1G SFP copper RJ45" },
|
||||||
|
{ pid: "FH-SFP-GE-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1490nm", standard: "1000BASE-BX10", notes: "FiberHome 1G SFP BiDi TX1310/RX1490 20km SMF" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "FiberHome 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "FH-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "FiberHome 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "FH-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "FiberHome 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "FH-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "FiberHome 10G SFP+ ZR 80km SMF" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "FiberHome 25G SFP28 SR 100m MMF" },
|
||||||
|
{ pid: "FH-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "FiberHome 25G SFP28 LR 10km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "FiberHome 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "FH-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "FiberHome 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "FiberHome 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "FH-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "FiberHome 100G QSFP28 LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "FiberHome 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
|
||||||
|
// ── PON / Access (Telecom) ───────────────────────────────────────────────
|
||||||
|
{ pid: "FH-SFP-GPON-OLT", formFactor: "SFP", speedGbps: 2.5, speed: "2.5G", reachMeters: 20000, reachLabel: "GPON", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "ITU-T G.984", notes: "FiberHome GPON OLT SFP TX1490/RX1310 20km" },
|
||||||
|
{ pid: "FH-SFP-XGS-PON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 20000, reachLabel: "XGS-PON",fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "ITU-T G.9807", notes: "FiberHome XGS-PON OLT SFP+ TX1577/RX1270 20km" },
|
||||||
|
|
||||||
|
// ── CWDM (Telecom) ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-SFP-CWDM-1550", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "CWDM", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "ITU-T G.694.2", notes: "FiberHome 10G CWDM SFP+ 1550nm 80km SMF" },
|
||||||
|
|
||||||
|
// ── DWDM (Telecom) ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-DWDM-SFP-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "ITU-T G.694.1", notes: "FiberHome 10G DWDM SFP+ C-band tunable 80km" },
|
||||||
|
|
||||||
|
// ── CFP2-DCO (Telecom) ───────────────────────────────────────────────────
|
||||||
|
{ pid: "FH-CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1000000,reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF CFP2-DCO", notes: "FiberHome 100G CFP2 DCO coherent C-band, 1000km span" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeFiberhomeOem(): Promise<void> {
|
||||||
|
console.log("=== FiberHome Technologies OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"FiberHome Technologies",
|
||||||
|
"oem",
|
||||||
|
"https://www.fiberhome.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of FIBERHOME_PIDS) {
|
||||||
|
const slug = `fiberhome-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== FiberHome Technologies OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${FIBERHOME_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeFiberhomeOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
121
packages/scraper/src/scrapers/fortinet-oem.ts
Normal file
121
packages/scraper/src/scrapers/fortinet-oem.ts
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
/**
|
||||||
|
* Fortinet OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Fortinet-branded transceiver PIDs for FortiGate firewalls and
|
||||||
|
* FortiSwitch series. All PIDs use the real-world FG-TRAN- naming convention
|
||||||
|
* matching Fortinet's hardware compatibility lists.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Fortinet Transceiver Module Data Sheet (fortinet.com)
|
||||||
|
* - FortiGate 7000/6000/3000 Series Chassis Hardware Guide
|
||||||
|
* - FortiSwitch 200/400/500/600 Series Hardware Reference
|
||||||
|
* - Fortinet Compatibility Matrix (HCL)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/fortinet-oem.ts
|
||||||
|
* Cron: daily at 22:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface FortinetPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const FORTINET_PIDS: FortinetPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FG-TRAN-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "FG-TRAN-GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "FG-TRAN-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Copper RJ45 1G SFP for FortiGate/FortiSwitch" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FG-TRAN-SFP+SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "FG-TRAN-SFP+LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "FG-TRAN-SFP+ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "FG-TRAN-SFP+BIDI", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1490nm", notes: "10G BiDi single-fiber, 1310Tx/1490Rx, 20km" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FG-TRAN-SFP28-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "FG-TRAN-SFP28-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FG-TRAN-QSFP+SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "FG-TRAN-QSFP+LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FG-TRAN-QSFP28-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "FG-TRAN-QSFP28-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FG-TRAN-DAC10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "FG-TRAN-DAC10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "FG-TRAN-DAC40G-1M", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP+" },
|
||||||
|
{ pid: "FG-TRAN-DAC100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeFortinetOem(): Promise<void> {
|
||||||
|
console.log("=== Fortinet OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Fortinet",
|
||||||
|
"oem",
|
||||||
|
"https://www.fortinet.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of FORTINET_PIDS) {
|
||||||
|
const slug = `fortinet-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Fortinet OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${FORTINET_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeFortinetOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
134
packages/scraper/src/scrapers/fujitsu-oem.ts
Normal file
134
packages/scraper/src/scrapers/fujitsu-oem.ts
Normal file
@ -0,0 +1,134 @@
|
|||||||
|
/**
|
||||||
|
* Fujitsu Network Communications OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Fujitsu-branded transceiver PIDs for optical transport platforms:
|
||||||
|
* FLASHWAVE series and 1FINITY platforms.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Fujitsu 1FINITY Transceiver Module Guide
|
||||||
|
* - Fujitsu FLASHWAVE Platform Hardware Installation Guide
|
||||||
|
* - Fujitsu Network Communications Product Portfolio (fujitsu.com)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/fujitsu-oem.ts
|
||||||
|
* Cron: daily at 18:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface FujitsuPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const FUJITSU_PIDS: FujitsuPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FC-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "FC-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "FC-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ Ethernet ───────────────────────────────────────────────────
|
||||||
|
{ pid: "FC-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "FC-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "FC-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "FC-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "FC-SFP-10G-DWDM-TUNE", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "Fujitsu 10G DWDM tunable SFP+ for FLASHWAVE" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FC-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "FC-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "FC-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── 100G CFP2 Coherent ───────────────────────────────────────────────────
|
||||||
|
{ pid: "FC-CFP2-100G-DCO", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1500000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Fujitsu 100G CFP2 DCO coherent" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FC-QSFPDD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "FC-QSFPDD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "FC-QSFPDD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "FC-QSFPDD-400G-LR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
{ pid: "FC-QSFPDD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", category: "Telecom", notes: "Fujitsu 400G ZR coherent QSFP-DD" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "FC-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "FC-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "FC-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "FC-DAC-400G-1M", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP-DD" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (coherent/DWDM)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"FC-SFP-10G-DWDM-TUNE",
|
||||||
|
"FC-CFP2-100G-DCO",
|
||||||
|
"FC-QSFPDD-400G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeFujitsuOem(): Promise<void> {
|
||||||
|
console.log("=== Fujitsu Network Communications OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Fujitsu Network Communications",
|
||||||
|
"oem",
|
||||||
|
"https://www.fujitsu.com/us/products/network/",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of FUJITSU_PIDS) {
|
||||||
|
const slug = `fujitsu-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Fujitsu OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${FUJITSU_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeFujitsuOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
118
packages/scraper/src/scrapers/ge-grid-oem.ts
Normal file
118
packages/scraper/src/scrapers/ge-grid-oem.ts
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
/**
|
||||||
|
* GE Grid Solutions OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for GE Grid Solutions industrial SFP/SFP+/SFP28/QSFP+
|
||||||
|
* modules used in power grid protection, substation automation, and SCADA
|
||||||
|
* communication hardware (e.g. Multilin, UR-series, MiCOM relay platforms).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - GE Grid Solutions product catalog (gegridsolutions.com)
|
||||||
|
* - GE Multilin / UR-series networking datasheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ge-grid-oem.ts
|
||||||
|
* Cron: daily at 17:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface GeGridPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const GE_GRID_PIDS: GeGridPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GEG-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "GE Grid Solutions SFP 1G SX substation/power grid" },
|
||||||
|
{ pid: "GEG-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "GE Grid Solutions SFP 1G LX substation/power grid" },
|
||||||
|
{ pid: "GEG-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "GE Grid Solutions SFP 1G ZX 80km long-haul substation" },
|
||||||
|
{ pid: "GEG-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "GE Grid Solutions SFP 1G copper RJ45" },
|
||||||
|
{ pid: "GEG-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "GE Grid Solutions SFP GE copper industrial" },
|
||||||
|
|
||||||
|
// ── 1G SFP BiDi ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GEG-SFP-1G-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1310", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550nm", notes: "GE Grid Solutions SFP 1G BiDi TX1310 single fiber" },
|
||||||
|
{ pid: "GEG-SFP-1G-BIDI-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-TX1550", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310nm", notes: "GE Grid Solutions SFP 1G BiDi TX1550 single fiber" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GEG-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "GE Grid Solutions SFP+ 10G SR" },
|
||||||
|
{ pid: "GEG-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "GE Grid Solutions SFP+ 10G LR" },
|
||||||
|
{ pid: "GEG-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "GE Grid Solutions SFP+ 10G ER" },
|
||||||
|
|
||||||
|
// ── 1G CWDM SFP (substation/protection rings) ───────────────────────────
|
||||||
|
{ pid: "GEG-SFP-CWDM-1470", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1470", fiberType: "SMF", connector: "LC", wavelengths: "1470nm", notes: "GE Grid Solutions SFP CWDM 1470nm 40km substation ring" },
|
||||||
|
{ pid: "GEG-SFP-CWDM-1530", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1530", fiberType: "SMF", connector: "LC", wavelengths: "1530nm", notes: "GE Grid Solutions SFP CWDM 1530nm 40km substation ring" },
|
||||||
|
{ pid: "GEG-SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", notes: "GE Grid Solutions SFP CWDM 1550nm 40km substation ring" },
|
||||||
|
{ pid: "GEG-SFP-CWDM-1590", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1590", fiberType: "SMF", connector: "LC", wavelengths: "1590nm", notes: "GE Grid Solutions SFP CWDM 1590nm 40km substation ring" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GEG-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "GE Grid Solutions QSFP+ 40G LR4" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GEG-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "GE Grid Solutions SFP28 25G LR industrial" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeGeGridOem(): Promise<void> {
|
||||||
|
console.log("=== GE Grid Solutions OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"GE Grid Solutions",
|
||||||
|
"oem",
|
||||||
|
"https://www.gegridsolutions.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of GE_GRID_PIDS) {
|
||||||
|
const slug = `ge-grid-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== GE Grid Solutions OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${GE_GRID_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeGeGridOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
333
packages/scraper/src/scrapers/google-cloud-oem.ts
Normal file
333
packages/scraper/src/scrapers/google-cloud-oem.ts
Normal file
@ -0,0 +1,333 @@
|
|||||||
|
/**
|
||||||
|
* Google Cloud OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Google Cloud-branded transceiver PIDs used in Cloud Interconnect
|
||||||
|
* dedicated/partner connections and Google-designed datacenter hardware
|
||||||
|
* (Jupiter fabric switches, Titan network ASICs). Google uses a GCP- prefix
|
||||||
|
* for SFP/QSFP optics validated in its open-source SONiC-based switching
|
||||||
|
* infrastructure.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Google Cloud Interconnect hardware requirements
|
||||||
|
* (cloud.google.com/network-connectivity/docs/interconnect)
|
||||||
|
* - Google datacenter hardware engineering publications
|
||||||
|
* - SFF-8024 / IEEE 802.3 speed codes
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/google-cloud-oem.ts
|
||||||
|
* Cron: daily at 07:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface GcpPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that map to telecom / DWDM variants (ZR, ZR4)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"GCP-SFP-10G-ZR",
|
||||||
|
"GCP-QSFP28-100G-ZR4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const GCP_PIDS: GcpPID[] = [
|
||||||
|
// ── 1G SFP ───────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP-1G-SX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 550,
|
||||||
|
reachLabel: "SX",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "1000BASE-SX",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE SFP SX; Google Cloud datacenter management port; OM2/OM3 MMF up to 550m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP-1G-LX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LX",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "1000BASE-LX",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE SFP LX; Google Cloud datacenter management port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 1G Copper SFP ────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP-GE-T",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "T",
|
||||||
|
fiberType: "DAC",
|
||||||
|
connector: "RJ45",
|
||||||
|
standard: "1000BASE-T",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "1GbE copper SFP RJ45; Google Cloud server management copper option; Cat5e/6",
|
||||||
|
},
|
||||||
|
// ── 10G SFP+ ─────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP-10G-SR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ SR; Google Cloud Jupiter fabric server uplink; OM3/OM4 MMF up to 300m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP-10G-LR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ LR; Google Cloud Interconnect 10G dedicated port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP-10G-ER",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 40000,
|
||||||
|
reachLabel: "ER",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1550nm",
|
||||||
|
standard: "10GBASE-ER",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "10GbE SFP+ ER; Google Cloud Interconnect extended-reach; OS2 SMF up to 40km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP-10G-ZR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 80000,
|
||||||
|
reachLabel: "ZR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1550nm",
|
||||||
|
standard: "10GBASE-ZR",
|
||||||
|
category: "Telecom",
|
||||||
|
notes: "10GbE SFP+ ZR; Google Cloud Interconnect DWDM long-haul 80km; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP28-25G-SR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "25GBASE-SR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "25GbE SFP28 SR; Google Jupiter v2 fabric server ToR port; OM4 MMF up to 100m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-SFP28-25G-LR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "25GBASE-LR",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "25GbE SFP28 LR; Google Jupiter v2 fabric server ToR port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 40G QSFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "GCP-QSFP-40G-SR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "40GBASE-SR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "40GbE QSFP+ SR4; Google Jupiter fabric aggregation uplink; OM3/OM4 MPO-12 up to 150m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-QSFP-40G-LR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1270-1330nm",
|
||||||
|
standard: "40GBASE-LR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "40GbE QSFP+ LR4; Google Cloud Interconnect 40G dedicated port; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 100G QSFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "GCP-QSFP28-100G-SR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "100GBASE-SR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "100GbE QSFP28 SR4; Google Cloud Interconnect 100G / Jupiter fabric; OM4 MPO-12 up to 100m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-QSFP28-100G-LR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-LR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "100GbE QSFP28 LR4; Google Cloud Interconnect 100G dedicated connection; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-QSFP28-100G-ZR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 80000,
|
||||||
|
reachLabel: "ZR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1271-1331nm",
|
||||||
|
standard: "100G-ZR4",
|
||||||
|
category: "Telecom",
|
||||||
|
notes: "100G QSFP28 ZR4; Google Cloud Interconnect DWDM long-haul 80km; coherent-lite; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 400G QSFP-DD ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "GCP-QSFP-DD-400G-DR4",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 500,
|
||||||
|
reachLabel: "DR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "400GBASE-DR4",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "400GbE QSFP-DD DR4; Google Jupiter Falcon fabric spine interconnect; OS2 SMF up to 500m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "GCP-QSFP-DD-400G-SR8",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR8",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "400GBASE-SR8",
|
||||||
|
category: "DataCenter",
|
||||||
|
notes: "400GbE QSFP-DD SR8; Google Jupiter Falcon fabric spine interconnect; OM4/OM5 MPO-16 up to 100m",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeGoogleCloudOem(): Promise<void> {
|
||||||
|
console.log("=== Google Cloud OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Google Cloud",
|
||||||
|
"oem",
|
||||||
|
"https://cloud.google.com/network-connectivity/docs/interconnect",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of GCP_PIDS) {
|
||||||
|
const slug = `gcp-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : (p.category ?? "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Google Cloud OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${GCP_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeGoogleCloudOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
133
packages/scraper/src/scrapers/grass-valley-oem.ts
Normal file
133
packages/scraper/src/scrapers/grass-valley-oem.ts
Normal file
@ -0,0 +1,133 @@
|
|||||||
|
/**
|
||||||
|
* Grass Valley OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Grass Valley-branded transceiver PIDs for broadcast production
|
||||||
|
* infrastructure: GV Orbit, Densité 3+, MV-821/MV-851 multiviewers,
|
||||||
|
* and IQUCP/IPG-3901 IP gateway cards.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Grass Valley Product Catalog (grassvalley.com)
|
||||||
|
* - GV Orbit Control System Hardware Reference
|
||||||
|
* - Densité 3+ Frame and Card Transceiver Compatibility Guide
|
||||||
|
* - GV Fabric IP Switching Platform Hardware Notes
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/grass-valley-oem.ts
|
||||||
|
* Cron: daily at 18:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface GrassValleyPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const GRASS_VALLEY_PIDS: GrassValleyPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GV-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Grass Valley 1G SX SFP for Densité 3+ frame and GV Orbit" },
|
||||||
|
{ pid: "GV-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Grass Valley 1G LX SFP for inter-frame Densité management" },
|
||||||
|
{ pid: "GV-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Grass Valley 1G copper SFP for out-of-band management" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GV-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "GV-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "GV-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "GV-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom", notes: "Grass Valley 10G ZR SFP+ for broadcast contribution over dark fiber" },
|
||||||
|
|
||||||
|
// ── SONET/SDH SFP (broadcast SDH / contribution) ────────────────────────
|
||||||
|
{ pid: "GV-SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "155M", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Grass Valley OC-3 SFP for legacy SDH broadcast ring" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GV-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Grass Valley 25G SR for GV Fabric IP switching and IQUCP25" },
|
||||||
|
{ pid: "GV-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Grass Valley 25G LR for long-reach GV Fabric interconnect" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GV-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "GV-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GV-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "GV-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G CFP (broadcast 100G contribution) ──────────────────────────────
|
||||||
|
{ pid: "GV-CFP-100G-LR4", formFactor: "CFP", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", category: "Telecom", notes: "Grass Valley CFP 100G LR4 for high-density GV Orbit broadcast fabric" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "GV-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Grass Valley 400G DR4 for next-generation GV Fabric spine interconnect" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (SONET/SDH, CFP coherent, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"GV-SFP-10G-ZR",
|
||||||
|
"GV-SFP-OC3-SR",
|
||||||
|
"GV-CFP-100G-LR4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeGrassValleyOem(): Promise<void> {
|
||||||
|
console.log("=== Grass Valley OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Grass Valley",
|
||||||
|
"oem",
|
||||||
|
"https://www.grassvalley.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of GRASS_VALLEY_PIDS) {
|
||||||
|
const slug = `grass-valley-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Grass Valley OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${GRASS_VALLEY_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeGrassValleyOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
122
packages/scraper/src/scrapers/h3c-oem.ts
Normal file
122
packages/scraper/src/scrapers/h3c-oem.ts
Normal file
@ -0,0 +1,122 @@
|
|||||||
|
/**
|
||||||
|
* H3C OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds H3C-branded transceiver PIDs for S-series, CR-series, and
|
||||||
|
* NE-series switches and routers from New H3C Group.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - H3C Transceiver Module Compatibility Guide (h3c.com)
|
||||||
|
* - H3C Optical Transceiver Module Technical Overview
|
||||||
|
* - H3C S12500/S6800/CR16000 Hardware Compatibility Matrix
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/h3c-oem.ts
|
||||||
|
* Cron: daily at 11:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface H3cPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const H3C_PIDS: H3cPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "H3C-SFP-GE-SX-MM850", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "H3C 1G SFP SX 550m MMF" },
|
||||||
|
{ pid: "H3C-SFP-GE-LX-SM1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "H3C 1G SFP LX 10km SMF" },
|
||||||
|
{ pid: "H3C-SFP-GE-ZX-SM1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "H3C 1G SFP ZX 80km SMF" },
|
||||||
|
{ pid: "H3C-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "H3C 1G SFP copper RJ45" },
|
||||||
|
{ pid: "H3C-SFP-GE-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1490nm", standard: "1000BASE-BX", notes: "H3C 1G SFP BiDi TX1310/RX1490" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "H3C-SFP-XG-SR-MM850", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "H3C 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "H3C-SFP-XG-LR-SM1310", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "H3C 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "H3C-SFP-XG-ER-SM1550", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "H3C 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "H3C-SFP-XG-ZR-SM1550", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "H3C 10G SFP+ ZR 80km SMF" },
|
||||||
|
{ pid: "H3C-SFP-XG-BIDI-1270", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1270/1330nm", standard: "10GBASE-BX", notes: "H3C 10G SFP+ BiDi TX1270/RX1330" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "H3C-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "H3C 25G SFP28 SR 100m MMF" },
|
||||||
|
{ pid: "H3C-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "H3C 25G SFP28 LR 10km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "H3C-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "H3C 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "H3C-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "H3C 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "H3C-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "H3C 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "H3C-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "H3C 100G QSFP28 LR4 10km SMF" },
|
||||||
|
{ pid: "H3C-QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "H3C 100G QSFP28 ER4 40km SMF" },
|
||||||
|
{ pid: "H3C-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4", notes: "H3C 100G QSFP28 CWDM4 2km SMF" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "H3C-QSFP-DD-400G-DR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "H3C 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "H3C-QSFP-DD-400G-SR8", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "H3C 400G QSFP-DD SR8 100m MMF" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeH3cOem(): Promise<void> {
|
||||||
|
console.log("=== H3C OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"H3C",
|
||||||
|
"oem",
|
||||||
|
"https://www.h3c.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of H3C_PIDS) {
|
||||||
|
const slug = `h3c-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== H3C OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${H3C_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeH3cOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
120
packages/scraper/src/scrapers/haivision-oem.ts
Normal file
120
packages/scraper/src/scrapers/haivision-oem.ts
Normal file
@ -0,0 +1,120 @@
|
|||||||
|
/**
|
||||||
|
* Haivision OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Haivision-branded transceiver PIDs for video streaming and
|
||||||
|
* broadcast encoding infrastructure: Makito X4 HEVC encoder/decoder,
|
||||||
|
* KB/KBe series encoders, SRT Gateway, and CoolSign signage platforms.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Haivision Product Catalog (haivision.com)
|
||||||
|
* - Makito X4 Encoder/Decoder Hardware Reference
|
||||||
|
* - Haivision SRT Gateway Platform Transceiver Notes
|
||||||
|
* - KB/KBe Series Server Hardware Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/haivision-oem.ts
|
||||||
|
* Cron: daily at 18:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface HaivisionPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const HAIVISION_PIDS: HaivisionPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HAI-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Haivision 1G SX SFP for Makito X4 encoder primary interface" },
|
||||||
|
{ pid: "HAI-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Haivision 1G LX SFP for inter-building encoder connectivity" },
|
||||||
|
{ pid: "HAI-SFP-1G-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Haivision 1G copper SFP for management and short-reach encoder links" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HAI-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Haivision 10G SR SFP+ for SRT Gateway and KB encoder" },
|
||||||
|
{ pid: "HAI-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Haivision 10G LR for campus distribution of HEVC streams" },
|
||||||
|
{ pid: "HAI-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Haivision 10G ER for extended-reach SRT contribution links" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HAI-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Haivision 25G SR SFP28 for high-density Makito X4 server deployments" },
|
||||||
|
{ pid: "HAI-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Haivision 25G LR for long-reach SRT Gateway interconnect" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HAI-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Haivision 40G SR4 for KB server aggregation uplink" },
|
||||||
|
{ pid: "HAI-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HAI-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "HAI-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── Copper SFP (management) ──────────────────────────────────────────────
|
||||||
|
{ pid: "HAI-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Haivision 1G copper SFP for out-of-band management ports" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HAI-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Haivision 400G DR4 for next-gen high-density encoder fabric uplinks" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeHaivisionOem(): Promise<void> {
|
||||||
|
console.log("=== Haivision OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Haivision",
|
||||||
|
"oem",
|
||||||
|
"https://www.haivision.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of HAIVISION_PIDS) {
|
||||||
|
const slug = `haivision-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Haivision OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${HAIVISION_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeHaivisionOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
146
packages/scraper/src/scrapers/harmonic-oem.ts
Normal file
146
packages/scraper/src/scrapers/harmonic-oem.ts
Normal file
@ -0,0 +1,146 @@
|
|||||||
|
/**
|
||||||
|
* Harmonic OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Harmonic-branded transceiver PIDs for video streaming and broadcast
|
||||||
|
* infrastructure: VOS (Video Open Software), NSG (Network Services Gateway),
|
||||||
|
* Electra X, and Spectrum X platforms.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Harmonic Product Catalog (harmonicinc.com)
|
||||||
|
* - Harmonic NSG 9000-6 Hardware Reference Guide
|
||||||
|
* - VOS/HOS Media Processor Platform Transceiver Compatibility
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/harmonic-oem.ts
|
||||||
|
* Cron: daily at 21:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface HarmonicPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const HARMONIC_PIDS: HarmonicPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HRM-SFP-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "HRM-SFP-GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
|
||||||
|
// ── SONET/SDH SFP (broadcast / contribution networks) ───────────────────
|
||||||
|
{ pid: "HRM-SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "155M", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Harmonic OC-3 SFP for broadcast SDH ring" },
|
||||||
|
{ pid: "HRM-SFP-OC3-LR", formFactor: "SFP", speedGbps: 0.155, speed: "155M", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Harmonic OC-3 LR SFP for broadcast SDH" },
|
||||||
|
{ pid: "HRM-SFP-OC12-SR", formFactor: "SFP", speedGbps: 0.622, speed: "622M", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Harmonic OC-12 SFP for contribution" },
|
||||||
|
{ pid: "HRM-SFP-OC12-LR", formFactor: "SFP", speedGbps: 0.622, speed: "622M", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Harmonic OC-12 LR SFP" },
|
||||||
|
{ pid: "HRM-SFP-OC48-SR", formFactor: "SFP", speedGbps: 2.488, speed: "2.5G", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "OC-48/STM-16",category: "Telecom", notes: "Harmonic OC-48 SFP for high-bandwidth contribution" },
|
||||||
|
{ pid: "HRM-SFP-OC48-LR", formFactor: "SFP", speedGbps: 2.488, speed: "2.5G", reachMeters: 40000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-48/STM-16",category: "Telecom", notes: "Harmonic OC-48 LR SFP" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HRM-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "HRM-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "HRM-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "HRM-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Harmonic 10G ZR for video contribution links" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ (broadcast contribution) ──────────────────────────────
|
||||||
|
{ pid: "HRM-DWDM-SFP10G-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "Harmonic 10G DWDM SFP+ for broadcast contribution DWDM ring" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HRM-SFP-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "HRM-SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HRM-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "HRM-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HRM-QSFP-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "HRM-QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G CFP (broadcast 100G contribution) ──────────────────────────────
|
||||||
|
{ pid: "HRM-CFP-100G-LR4", formFactor: "CFP", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", category: "Telecom", notes: "Harmonic CFP 100G LR4 for Electra X broadcast platform" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HRM-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+", notes: "Harmonic 10G DAC 1m" },
|
||||||
|
{ pid: "HRM-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+", notes: "Harmonic 10G DAC 3m" },
|
||||||
|
{ pid: "HRM-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28", notes: "Harmonic 100G DAC 1m" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (SONET/SDH, DWDM, CFP coherent)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"HRM-SFP-OC3-SR",
|
||||||
|
"HRM-SFP-OC3-LR",
|
||||||
|
"HRM-SFP-OC12-SR",
|
||||||
|
"HRM-SFP-OC12-LR",
|
||||||
|
"HRM-SFP-OC48-SR",
|
||||||
|
"HRM-SFP-OC48-LR",
|
||||||
|
"HRM-DWDM-SFP10G-C",
|
||||||
|
"HRM-CFP-100G-LR4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeHarmonicOem(): Promise<void> {
|
||||||
|
console.log("=== Harmonic OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Harmonic",
|
||||||
|
"oem",
|
||||||
|
"https://www.harmonicinc.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of HARMONIC_PIDS) {
|
||||||
|
const slug = `harmonic-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Harmonic OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${HARMONIC_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeHarmonicOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
120
packages/scraper/src/scrapers/hillstone-oem.ts
Normal file
120
packages/scraper/src/scrapers/hillstone-oem.ts
Normal file
@ -0,0 +1,120 @@
|
|||||||
|
/**
|
||||||
|
* Hillstone Networks OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Hillstone-branded transceiver PIDs for E-Series, T-Series, and
|
||||||
|
* X-Series next-generation firewalls.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Hillstone Networks Hardware Compatibility List (hillstonenet.com)
|
||||||
|
* - E-Series / T-Series / X-Series Appliance Hardware Guide
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/hillstone-oem.ts
|
||||||
|
* Cron: daily at 19:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface HillstonePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const HILLSTONE_PIDS: HillstonePID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HS-SFP-1GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "HS-SFP-1GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "HS-SFP-1GE-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "HS-SFP-1GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HS-SFP-10GE-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "HS-SFP-10GE-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "HS-SFP-10GE-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "HS-SFP-10GE-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "HS-SFP-10GE-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HS-SFP28-25GE-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "HS-SFP28-25GE-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HS-QSFP-40GE-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "HS-QSFP-40GE-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HS-QSFP28-100GE-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "HS-QSFP28-100GE-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "HS-QSFP28-100GE-CWDM4",formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HS-DAC-10GE-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "HS-DAC-10GE-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "HS-DAC-100GE-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeHillstoneOem(): Promise<void> {
|
||||||
|
console.log("=== Hillstone Networks OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Hillstone Networks",
|
||||||
|
"oem",
|
||||||
|
"https://www.hillstonenet.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of HILLSTONE_PIDS) {
|
||||||
|
const slug = `hillstone-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Hillstone Networks OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${HILLSTONE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeHillstoneOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
130
packages/scraper/src/scrapers/hirschmann-oem.ts
Normal file
130
packages/scraper/src/scrapers/hirschmann-oem.ts
Normal file
@ -0,0 +1,130 @@
|
|||||||
|
/**
|
||||||
|
* Hirschmann OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Hirschmann (now Belden brand) industrial-grade
|
||||||
|
* SFP/SFP+/SFP28/QSFP+/QSFP28 modules used in MACH, RSP, OCTOPUS, and
|
||||||
|
* DRAGON series industrial Ethernet managed switches.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Hirschmann/Belden SFP module datasheets (belden.com/brands/hirschmann)
|
||||||
|
* - Hirschmann MACH/RSP series hardware installation guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/hirschmann-oem.ts
|
||||||
|
* Cron: daily at 12:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface HirschmannPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const HIRSCHMANN_PIDS: HirschmannPID[] = [
|
||||||
|
// ── 1G SFP (industrial grade) ───────────────────────────────────────────
|
||||||
|
{ pid: "M-SFP-SX/LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Hirschmann industrial 1G SX" },
|
||||||
|
{ pid: "M-SFP-LX/LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Hirschmann industrial 1G LX" },
|
||||||
|
{ pid: "M-SFP-LH/LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH40", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", notes: "Hirschmann industrial 1G 40km" },
|
||||||
|
{ pid: "M-SFP-ZX/LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "Hirschmann industrial 1G ZX" },
|
||||||
|
{ pid: "M-SFP-T/RJ45", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Hirschmann 1G copper SFP" },
|
||||||
|
{ pid: "M-SFP-BIDI-L/LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "1310/1550nm", notes: "Hirschmann 1G BiDi single fiber 20km" },
|
||||||
|
|
||||||
|
// ── 100M SFP (legacy industrial) ────────────────────────────────────────
|
||||||
|
{ pid: "M-FAST-SFP-MM/LC", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 2000, reachLabel: "FX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "100BASE-FX", notes: "Hirschmann legacy industrial 100M FX MMF" },
|
||||||
|
{ pid: "M-FAST-SFP-SM/LC", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 40000, reachLabel: "LFX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100BASE-LFX", notes: "Hirschmann legacy industrial 100M LFX SMF 40km" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "M-SFP-10-SR/LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Hirschmann industrial 10G SR" },
|
||||||
|
{ pid: "M-SFP-10-LR/LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Hirschmann industrial 10G LR" },
|
||||||
|
{ pid: "M-SFP-10-ER/LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "M-SFP-10-ZR/LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ BiDi ───────────────────────────────────────────────────────
|
||||||
|
{ pid: "M-SFP-10-BD13/LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1270/RX1330nm", notes: "Hirschmann 10G BiDi TX1270/RX1330 single fiber 10km" },
|
||||||
|
{ pid: "M-SFP-10-BD15/LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi", fiberType: "SMF", connector: "LC", wavelengths: "TX1330/RX1270nm", notes: "Hirschmann 10G BiDi RX pair for M-SFP-10-BD13" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "M-SFP-25-SR/LC", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "M-SFP-25-LR/LC", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "M-QSFP-40-SR4/MPO", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "M-QSFP-40-LR4/LC", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "M-QSFP-100-SR4/MPO", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "M-QSFP-100-LR4/LC", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "M-QSFP-100-CWDM4/LC", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
|
||||||
|
// ── DAC Cables ──────────────────────────────────────────────────────────
|
||||||
|
{ pid: "M-DAC-SFP-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC1M", fiberType: "DAC", connector: "SFP+", notes: "Hirschmann SFP+ DAC 1m" },
|
||||||
|
{ pid: "M-DAC-SFP-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC3M", fiberType: "DAC", connector: "SFP+", notes: "Hirschmann SFP+ DAC 3m" },
|
||||||
|
{ pid: "M-DAC-QSFP-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC1M", fiberType: "DAC", connector: "QSFP28", notes: "Hirschmann QSFP28 DAC 1m" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeHirschmannOem(): Promise<void> {
|
||||||
|
console.log("=== Hirschmann OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Hirschmann",
|
||||||
|
"oem",
|
||||||
|
"https://www.belden.com/brands/hirschmann",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of HIRSCHMANN_PIDS) {
|
||||||
|
const slug = `hirschmann-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Hirschmann OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${HIRSCHMANN_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeHirschmannOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
131
packages/scraper/src/scrapers/hisense-broadband-oem.ts
Normal file
131
packages/scraper/src/scrapers/hisense-broadband-oem.ts
Normal file
@ -0,0 +1,131 @@
|
|||||||
|
/**
|
||||||
|
* Hisense Broadband OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Hisense Broadband-branded transceiver PIDs covering data-center
|
||||||
|
* switching, 400G high-density, coherent ZR, and CFP2-DCO optics from
|
||||||
|
* Hisense Broadband Multimedia Technologies Co., Ltd. (Qingdao, China).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Hisense Broadband Transceiver Product Overview (hisense-broadband.com)
|
||||||
|
* - Hisense Broadband 400G QSFP-DD ZR Data Sheet
|
||||||
|
* - Hisense Broadband CFP2-DCO Coherent Module Specification
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/hisense-broadband-oem.ts
|
||||||
|
* Cron: daily at 20:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface HisenseBBPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that belong to TELECOM category (CFP2-DCO, ZR/ZR+)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"HBB-SFP-10G-ZR",
|
||||||
|
"HBB-CFP2-DCO-100G",
|
||||||
|
"HBB-QSFP-DD-ZR-400G",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const HISENSE_BB_PIDS: HisenseBBPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HBB-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Hisense Broadband 1G SFP SX 550m MMF" },
|
||||||
|
{ pid: "HBB-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Hisense Broadband 1G SFP LX 10km SMF" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HBB-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Hisense Broadband 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "HBB-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Hisense Broadband 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "HBB-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Hisense Broadband 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "HBB-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Hisense Broadband 10G SFP+ ZR 80km SMF" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HBB-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Hisense Broadband 25G SFP28 SR 100m MMF" },
|
||||||
|
{ pid: "HBB-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Hisense Broadband 25G SFP28 LR 10km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HBB-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Hisense Broadband 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "HBB-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "Hisense Broadband 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HBB-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Hisense Broadband 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "HBB-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Hisense Broadband 100G QSFP28 LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HBB-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Hisense Broadband 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "HBB-QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "Hisense Broadband 400G QSFP-DD SR8 100m MMF" },
|
||||||
|
|
||||||
|
// ── CFP2-DCO (Telecom) ───────────────────────────────────────────────────
|
||||||
|
{ pid: "HBB-CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1000000,reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF CFP2-DCO", notes: "Hisense Broadband 100G CFP2 DCO coherent C-band, 1000km span" },
|
||||||
|
|
||||||
|
// ── 400G ZR (Telecom) ────────────────────────────────────────────────────
|
||||||
|
{ pid: "HBB-QSFP-DD-ZR-400G", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1000000,reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF 400ZR", notes: "Hisense Broadband 400G QSFP-DD ZR coherent C-band, 1000km span" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeHisenseBroadbandOem(): Promise<void> {
|
||||||
|
console.log("=== Hisense Broadband OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Hisense Broadband",
|
||||||
|
"oem",
|
||||||
|
"https://www.hisense-broadband.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of HISENSE_BB_PIDS) {
|
||||||
|
const slug = `hisense-bb-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Hisense Broadband OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${HISENSE_BB_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeHisenseBroadbandOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
315
packages/scraper/src/scrapers/hitachi-vantara-oem.ts
Normal file
315
packages/scraper/src/scrapers/hitachi-vantara-oem.ts
Normal file
@ -0,0 +1,315 @@
|
|||||||
|
/**
|
||||||
|
* Hitachi Vantara OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Hitachi Vantara-branded transceiver PIDs for VSP (Virtual Storage
|
||||||
|
* Platform) 5000/G/F series, VSP One, and HUS VM SAN storage controllers.
|
||||||
|
* Covers FC, iSCSI, and NVMe-oF host/target optical modules.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Hitachi Vantara Interoperability Matrix (hitachivantara.com/support)
|
||||||
|
* - VSP Hardware Installation and Configuration Guide
|
||||||
|
* - Hitachi VSP 5000 Series Optical Connectivity Reference
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/hitachi-vantara-oem.ts
|
||||||
|
* Cron: daily at 07:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface HitachiVantaraPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fibre Channel line-rate speeds (Gbps):
|
||||||
|
// 8G = 8.5 Gbps (8GFC)
|
||||||
|
// 16G = 14.025 Gbps (16GFC)
|
||||||
|
// 32G = 28.05 Gbps (32GFC)
|
||||||
|
// 64G = 56.1 Gbps (64GFC)
|
||||||
|
const HITACHI_VANTARA_PIDS: HitachiVantaraPID[] = [
|
||||||
|
// ── 8G FC SFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-8G-FC-SW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC SFP+ SW; Hitachi VSP G200/G400/G600 FC target port; OM3/OM4 MMF up to 150m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-8G-FC-LW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC SFP+ LW; Hitachi VSP G200/G400/G600 FC target port long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 16G FC SFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-16G-FC-SW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 125,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ SW; Hitachi VSP G800/G1000/F800 FC target port; OM4 MMF up to 125m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-16G-FC-LW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ LW; Hitachi VSP G800/G1000/F800 FC target port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 32G FC SFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-32G-FC-SW",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 28.05,
|
||||||
|
speed: "32G FC",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "32GFC",
|
||||||
|
notes: "32G FC SFP28 SW; Hitachi VSP 5100/5500/5600 NVMe/FC port; OM4/OM5 MMF up to 100m",
|
||||||
|
},
|
||||||
|
// ── 10GbE SFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-10G-SR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
notes: "10GbE SFP+ SR; Hitachi VSP iSCSI/FCoE host port; OM3/OM4 MMF up to 300m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-10G-LR",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
notes: "10GbE SFP+ LR; Hitachi VSP iSCSI/FCoE host port long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 25GbE SFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-SFP28-25G-SR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "25GBASE-SR",
|
||||||
|
notes: "25GbE SFP28 SR; Hitachi VSP 5000 series NVMe-oF iSCSI port; OM4 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "HV-SFP28-25G-LR",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "25GBASE-LR",
|
||||||
|
notes: "25GbE SFP28 LR; Hitachi VSP 5000 series NVMe-oF iSCSI long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 100GbE QSFP28 ────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-QSFP28-100G-SR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "100GBASE-SR4",
|
||||||
|
notes: "100GbE QSFP28 SR4; Hitachi VSP 5000 series backend replication link; OM4 MPO-12",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "HV-QSFP28-100G-LR4",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-LR4",
|
||||||
|
notes: "100GbE QSFP28 LR4; Hitachi VSP 5000 series replication long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 400GbE QSFP-DD ───────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-QSFP-DD-400G-DR4",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 500,
|
||||||
|
reachLabel: "DR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "400GBASE-DR4",
|
||||||
|
notes: "400GbE QSFP-DD DR4; Hitachi VSP One next-gen NVMe-oF fabric link; OS2 SMF up to 500m",
|
||||||
|
},
|
||||||
|
// ── 1GbE SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-1G-SX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 550,
|
||||||
|
reachLabel: "SX",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "1000BASE-SX",
|
||||||
|
notes: "1GbE SFP SX; Hitachi storage management and iSCSI port; OM2/OM3 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "HV-SFP-1G-LX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LX",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "1000BASE-LX",
|
||||||
|
notes: "1GbE SFP LX; Hitachi storage management and iSCSI port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 40GbE QSFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-QSFP-40G-SR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "40GBASE-SR4",
|
||||||
|
notes: "40GbE QSFP+ SR4; Hitachi storage fabric switch uplink; OM3/OM4 MPO-12",
|
||||||
|
},
|
||||||
|
// ── 64G FC SFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HV-SFP28-64G-FC-SW",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 56.1,
|
||||||
|
speed: "64G FC",
|
||||||
|
reachMeters: 70,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "64GFC",
|
||||||
|
notes: "64G FC SFP28 SW; Hitachi VSP One next-gen NVMe/FC port; OM5 WBMMF up to 70m",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeHitachiVantaraOem(): Promise<void> {
|
||||||
|
console.log("=== Hitachi Vantara OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Hitachi Vantara",
|
||||||
|
"oem",
|
||||||
|
"https://www.hitachivantara.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of HITACHI_VANTARA_PIDS) {
|
||||||
|
const slug = `hitachi-vantara-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Hitachi Vantara OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${HITACHI_VANTARA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeHitachiVantaraOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
365
packages/scraper/src/scrapers/hpe-storage-oem.ts
Normal file
365
packages/scraper/src/scrapers/hpe-storage-oem.ts
Normal file
@ -0,0 +1,365 @@
|
|||||||
|
/**
|
||||||
|
* HPE Storage OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds HPE Storage-branded transceiver PIDs for 3PAR/Primera/Alletra SAN
|
||||||
|
* storage controllers, FC host bus adapters, and iSCSI/FCoE host ports.
|
||||||
|
* Covers the full HPE StoreServ / Nimble / Alletra optical module lineup.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - HPE Storage Compatibility Matrix (support.hpe.com)
|
||||||
|
* - HPE 3PAR/Primera/Alletra Hardware Installation Guides
|
||||||
|
* - HPE QuickSpecs for SAN Switch and Storage optical modules
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/hpe-storage-oem.ts
|
||||||
|
* Cron: daily at 06:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface HpeStoragePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fibre Channel line-rate speeds (Gbps):
|
||||||
|
// 8G = 8.5 Gbps (8GFC)
|
||||||
|
// 16G = 14.025 Gbps (16GFC)
|
||||||
|
// 32G = 28.05 Gbps (32GFC)
|
||||||
|
const HPE_STORAGE_PIDS: HpeStoragePID[] = [
|
||||||
|
// ── 8G FC SFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "AJ716B",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC SFP+ SW; HPE 3PAR/P9000 FC host/target port; OM3/OM4 MMF up to 150m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "AJ717B",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC SFP+ LW; HPE 3PAR/P9000 FC host/target port long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 16G FC SFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "C8R23B",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 125,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ SW; HPE 3PAR 7000/8000/Primera FC target port; OM4 MMF up to 125m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "C8R24B",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ LW; HPE 3PAR 7000/8000/Primera FC target port long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 32G FC SFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "Q2P63A",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 28.05,
|
||||||
|
speed: "32G FC",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "32GFC",
|
||||||
|
notes: "32G FC SFP28 SW; HPE Primera A630/A650/Alletra 9000 FC NVMe/FC port; OM4/OM5 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "Q2P64A",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 28.05,
|
||||||
|
speed: "32G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "32GFC",
|
||||||
|
notes: "32G FC SFP28 LW; HPE Primera A630/A650/Alletra 9000 FC NVMe/FC long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 10GbE SFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "455885-B21",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
notes: "10GbE SFP+ SR; HPE storage and server host port; OM3 300m / OM4 400m MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "455886-B21",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
notes: "10GbE SFP+ LR; HPE storage and server host port long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 25GbE SFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "813874-B21",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "25GBASE-SR",
|
||||||
|
notes: "25GbE SFP28 SR; HPE Alletra 6000/9000 iSCSI/NVMe-oF host port; OM4 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "813875-B21",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "25GBASE-LR",
|
||||||
|
notes: "25GbE SFP28 LR; HPE Alletra 6000/9000 iSCSI/NVMe-oF host port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 100GbE QSFP28 ────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "817753-B21",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "100GBASE-SR4",
|
||||||
|
notes: "100GbE QSFP28 SR4; HPE storage backend/replication interconnect; OM4 MPO-12",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "817754-B21",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-LR4",
|
||||||
|
notes: "100GbE QSFP28 LR4; HPE storage backend/replication long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 400GbE QSFP-DD ───────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "P08748-B21",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 500,
|
||||||
|
reachLabel: "DR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "400GBASE-DR4",
|
||||||
|
notes: "400GbE QSFP-DD DR4; HPE next-gen storage fabric interconnect; OS2 SMF up to 500m",
|
||||||
|
},
|
||||||
|
// ── 1GbE SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "453154-B21",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 550,
|
||||||
|
reachLabel: "SX",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "1000BASE-SX",
|
||||||
|
notes: "1GbE SFP SX; HPE storage management and iSCSI port; OM2/OM3 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "453156-B21",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LX",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "1000BASE-LX",
|
||||||
|
notes: "1GbE SFP LX; HPE storage management and iSCSI port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── Copper SFP ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HPE-SFP-1G-T",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "T",
|
||||||
|
fiberType: "DAC",
|
||||||
|
connector: "RJ45",
|
||||||
|
standard: "1000BASE-T",
|
||||||
|
notes: "1GbE copper SFP RJ45; HPE storage management port copper option; Cat5e/6",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "HPE-SFP-10G-T",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 30,
|
||||||
|
reachLabel: "T",
|
||||||
|
fiberType: "DAC",
|
||||||
|
connector: "RJ45",
|
||||||
|
standard: "10GBASE-T",
|
||||||
|
notes: "10GbE copper SFP+ RJ45; HPE iSCSI storage host port; Cat6a up to 30m",
|
||||||
|
},
|
||||||
|
// ── 40GbE QSFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HPE-QSFP-40G-SR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "40GBASE-SR4",
|
||||||
|
notes: "40GbE QSFP+ SR4; HPE storage fabric switch uplink; OM3/OM4 MPO-12",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "HPE-QSFP-40G-LR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "40GBASE-LR4",
|
||||||
|
notes: "40GbE QSFP+ LR4; HPE storage fabric switch uplink long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 16G FC SFP+ Long-Wave (generic HPE part) ─────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "HPE-SFP-16G-FC-LW",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ LW generic HPE part; 3PAR/Primera cross-compat long-wave; OS2 SMF",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeHpeStorageOem(): Promise<void> {
|
||||||
|
console.log("=== HPE Storage OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"HPE Storage",
|
||||||
|
"oem",
|
||||||
|
"https://www.hpe.com/storage",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of HPE_STORAGE_PIDS) {
|
||||||
|
const slug = `hpe-storage-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== HPE Storage OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${HPE_STORAGE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeHpeStorageOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
147
packages/scraper/src/scrapers/huawei-access-oem.ts
Normal file
147
packages/scraper/src/scrapers/huawei-access-oem.ts
Normal file
@ -0,0 +1,147 @@
|
|||||||
|
/**
|
||||||
|
* Huawei Access OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Huawei Access-branded transceiver PIDs for SmartAX OLT platforms
|
||||||
|
* (MA5800, MA5600T) supporting GPON, XGS-PON, and 25G-PON line cards.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Huawei Fixed Network Access OLT Product Page (carrier.huawei.com)
|
||||||
|
* - Huawei SmartAX MA5800 Hardware Description
|
||||||
|
* - Huawei XGS-PON / 25G-PON Optical Module Data Sheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/huawei-access-oem.ts
|
||||||
|
* Cron: daily at 08:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface HuaweiAccessPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
isTelecom?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
const TELECOM_PIDS: Set<string> = new Set([
|
||||||
|
"H3C-GPON-OLT-CLASS-B",
|
||||||
|
"H3C-GPON-OLT-CLASS-C",
|
||||||
|
"HW-XGSPON-OLT-1577",
|
||||||
|
"HW-XGSPON-ONU-1270",
|
||||||
|
"HW-EPON-OLT",
|
||||||
|
"HW-10G-EPON-OLT",
|
||||||
|
"HW-25G-PON-OLT",
|
||||||
|
"HW-SFP-CWDM-1490",
|
||||||
|
"HW-SFP-CWDM-1550",
|
||||||
|
"HW-SFP-BIDI-1310",
|
||||||
|
"HW-SFP-BIDI-1490",
|
||||||
|
"HW-SFP-10G-BIDI-1270",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const HUAWEI_ACCESS_PIDS: HuaweiAccessPID[] = [
|
||||||
|
// ── GPON OLT SFP ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "H3C-GPON-OLT-CLASS-B", formFactor: "SFP", speedGbps: 2.488, speed: "GPON", reachMeters: 20000, reachLabel: "GPON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "G.984", notes: "Huawei Access GPON OLT SFP Class B+, SmartAX platform", isTelecom: true },
|
||||||
|
{ pid: "H3C-GPON-OLT-CLASS-C", formFactor: "SFP", speedGbps: 2.488, speed: "GPON", reachMeters: 20000, reachLabel: "GPON-C+", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "G.984", notes: "Huawei Access GPON OLT SFP Class C+, extended reach", isTelecom: true },
|
||||||
|
|
||||||
|
// ── XGS-PON OLT / ONU SFP+ ───────────────────────────────────────────────
|
||||||
|
{ pid: "HW-XGSPON-OLT-1577", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "XGS-PON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "G.9807.1", notes: "Huawei Access XGS-PON OLT SFP+ for MA5800", isTelecom: true },
|
||||||
|
{ pid: "HW-XGSPON-ONU-1270", formFactor: "SFP+", speedGbps: 10, speed: "XGS-PON", reachMeters: 20000, reachLabel: "XGS-ONU-20K", fiberType: "SMF", connector: "SC", wavelengths: "1270/1577nm", standard: "G.9807.1", notes: "Huawei Access XGS-PON ONU SFP+ 1270nm TX", isTelecom: true },
|
||||||
|
|
||||||
|
// ── EPON OLT SFP ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HW-EPON-OLT", formFactor: "SFP", speedGbps: 1, speed: "EPON", reachMeters: 20000, reachLabel: "EPON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", standard: "802.3ah", notes: "Huawei Access EPON OLT SFP 1G/1G", isTelecom: true },
|
||||||
|
{ pid: "HW-10G-EPON-OLT", formFactor: "SFP+", speedGbps: 10, speed: "10G-EPON", reachMeters: 20000, reachLabel: "10G-EPON-20K",fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "802.3av", notes: "Huawei Access 10G-EPON OLT SFP+", isTelecom: true },
|
||||||
|
|
||||||
|
// ── 25G-PON OLT SFP28 ────────────────────────────────────────────────────
|
||||||
|
{ pid: "HW-25G-PON-OLT", formFactor: "SFP28", speedGbps: 25, speed: "25G-PON", reachMeters: 20000, reachLabel: "25G-PON-20K", fiberType: "SMF", connector: "SC", wavelengths: "1577/1270nm", standard: "G.9804.3", notes: "Huawei Access 25G-PON OLT SFP28", isTelecom: true },
|
||||||
|
|
||||||
|
// ── Standard 1G SFP ──────────────────────────────────────────────────────
|
||||||
|
{ pid: "HW-SFP-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "HW-SFP-GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
|
||||||
|
// ── Standard 10G SFP+ ────────────────────────────────────────────────────
|
||||||
|
{ pid: "HW-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "HW-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "HW-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
|
||||||
|
// ── CWDM SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HW-SFP-CWDM-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1490", fiberType: "SMF", connector: "LC", wavelengths: "1490nm", standard: "CWDM", notes: "Huawei Access CWDM SFP 1490nm", isTelecom: true },
|
||||||
|
{ pid: "HW-SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "CWDM", notes: "Huawei Access CWDM SFP 1550nm", isTelecom: true },
|
||||||
|
|
||||||
|
// ── BiDi SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HW-SFP-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-1310", fiberType: "SMF", connector: "SC", wavelengths: "1310/1490nm", notes: "Huawei Access 1G BiDi SFP 1310nm TX / 1490nm RX", isTelecom: true },
|
||||||
|
{ pid: "HW-SFP-BIDI-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-1490", fiberType: "SMF", connector: "SC", wavelengths: "1490/1310nm", notes: "Huawei Access 1G BiDi SFP 1490nm TX / 1310nm RX", isTelecom: true },
|
||||||
|
{ pid: "HW-SFP-10G-BIDI-1270", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 20000, reachLabel: "BiDi-1270", fiberType: "SMF", connector: "LC", wavelengths: "1270/1330nm", notes: "Huawei Access 10G BiDi SFP+ 1270nm TX", isTelecom: true },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "HW-SFP-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G / 100G QSFP ──────────────────────────────────────────────────────
|
||||||
|
{ pid: "HW-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
{ pid: "HW-QSFP-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeHuaweiAccessOem(): Promise<void> {
|
||||||
|
console.log("=== Huawei Access OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Huawei Access",
|
||||||
|
"oem",
|
||||||
|
"https://carrier.huawei.com/en/products/fixed-network/access/olt",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of HUAWEI_ACCESS_PIDS) {
|
||||||
|
const slug = `huawei-access-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Huawei Access OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${HUAWEI_ACCESS_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeHuaweiAccessOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
368
packages/scraper/src/scrapers/ibm-storage-oem.ts
Normal file
368
packages/scraper/src/scrapers/ibm-storage-oem.ts
Normal file
@ -0,0 +1,368 @@
|
|||||||
|
/**
|
||||||
|
* IBM Storage OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds IBM Storage-branded transceiver PIDs for FlashSystem 5000/7000/9000,
|
||||||
|
* DS8000, Spectrum Virtualize SAN controllers, and IBM SAN Volume Controller
|
||||||
|
* (SVC) FC/FCoE/iSCSI host and inter-system link ports.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - IBM System Storage Interoperability Center (SSIC)
|
||||||
|
* - IBM FlashSystem Hardware Installation and Maintenance Guide
|
||||||
|
* - IBM DS8000 and SAN Volume Controller optical compatibility matrices
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ibm-storage-oem.ts
|
||||||
|
* Cron: daily at 06:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface IbmStoragePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fibre Channel line-rate speeds (Gbps):
|
||||||
|
// 8G = 8.5 Gbps (8GFC)
|
||||||
|
// 16G = 14.025 Gbps (16GFC)
|
||||||
|
// 32G = 28.05 Gbps (32GFC)
|
||||||
|
// 64G = 56.1 Gbps (64GFC)
|
||||||
|
const IBM_STORAGE_PIDS: IbmStoragePID[] = [
|
||||||
|
// ── 8G FC SFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "45W2408",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC SFP+ SW; IBM DS8000/SVC/V7000 FC target port; OM3/OM4 MMF up to 150m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "45W2409",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 8.5,
|
||||||
|
speed: "8G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "8GFC",
|
||||||
|
notes: "8G FC SFP+ LW; IBM DS8000/SVC/V7000 FC target port long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 16G FC SFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "00RY192",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 125,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ SW; IBM FlashSystem 7200/9100 FC port; OM4 MMF up to 125m",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "00RY193",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 14.025,
|
||||||
|
speed: "16G FC",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LW",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "16GFC",
|
||||||
|
notes: "16G FC SFP+ LW; IBM FlashSystem 7200/9100 FC port long-reach; OS2 SMF up to 10km",
|
||||||
|
},
|
||||||
|
// ── 32G FC SFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "02JG521",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 28.05,
|
||||||
|
speed: "32G FC",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "32GFC",
|
||||||
|
notes: "32G FC SFP28 SW; IBM FlashSystem 9200/9200R NVMe/FC port; OM4/OM5 MMF",
|
||||||
|
},
|
||||||
|
// ── 10GbE SFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "00MN503",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 300,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "10GBASE-SR",
|
||||||
|
notes: "10GbE SFP+ SR; IBM Storwize/FlashSystem iSCSI and FCoE host port; OM3/OM4 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "00MN505",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "10GBASE-LR",
|
||||||
|
notes: "10GbE SFP+ LR; IBM Storwize/FlashSystem iSCSI and FCoE host port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 25GbE SFP28 ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "00E6419",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "25GBASE-SR",
|
||||||
|
notes: "25GbE SFP28 SR; IBM FlashSystem 5035/5045 NVMe-oF host port; OM4 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "00E6420",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 25,
|
||||||
|
speed: "25G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "25GBASE-LR",
|
||||||
|
notes: "25GbE SFP28 LR; IBM FlashSystem 5035/5045 NVMe-oF host port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 100GbE QSFP28 ────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "02JG528",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "100GBASE-SR4",
|
||||||
|
notes: "100GbE QSFP28 SR4; IBM FlashSystem 9200R/9500R inter-node link; OM4 MPO-12",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "02JG529",
|
||||||
|
formFactor: "QSFP28",
|
||||||
|
speedGbps: 100,
|
||||||
|
speed: "100G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1295-1310nm",
|
||||||
|
standard: "100GBASE-LR4",
|
||||||
|
notes: "100GbE QSFP28 LR4; IBM FlashSystem 9200R/9500R inter-node long-reach link; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 1GbE SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "IBM-SFP-1G-SX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 550,
|
||||||
|
reachLabel: "SX",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "1000BASE-SX",
|
||||||
|
notes: "1GbE SFP SX; IBM storage management and iSCSI port; OM2/OM3 MMF",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "IBM-SFP-1G-LX",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LX",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "1000BASE-LX",
|
||||||
|
notes: "1GbE SFP LX; IBM storage management and iSCSI port long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── Copper SFP ───────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "IBM-SFP-1G-T",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 1,
|
||||||
|
speed: "1G",
|
||||||
|
reachMeters: 100,
|
||||||
|
reachLabel: "T",
|
||||||
|
fiberType: "DAC",
|
||||||
|
connector: "RJ45",
|
||||||
|
standard: "1000BASE-T",
|
||||||
|
notes: "1GbE copper SFP RJ45; IBM storage management copper option; Cat5e/6",
|
||||||
|
},
|
||||||
|
// ── 40GbE QSFP+ ──────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "IBM-QSFP-40G-SR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SR4",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "40GBASE-SR4",
|
||||||
|
notes: "40GbE QSFP+ SR4; IBM storage fabric switch uplink; OM3/OM4 MPO-12",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
pid: "IBM-QSFP-40G-LR4",
|
||||||
|
formFactor: "QSFP+",
|
||||||
|
speedGbps: 40,
|
||||||
|
speed: "40G",
|
||||||
|
reachMeters: 10000,
|
||||||
|
reachLabel: "LR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "40GBASE-LR4",
|
||||||
|
notes: "40GbE QSFP+ LR4; IBM storage fabric switch uplink long-reach; OS2 SMF",
|
||||||
|
},
|
||||||
|
// ── 400GbE QSFP-DD ───────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "IBM-QSFP-DD-400G-DR4",
|
||||||
|
formFactor: "QSFP-DD",
|
||||||
|
speedGbps: 400,
|
||||||
|
speed: "400G",
|
||||||
|
reachMeters: 500,
|
||||||
|
reachLabel: "DR4",
|
||||||
|
fiberType: "SMF",
|
||||||
|
connector: "MPO",
|
||||||
|
wavelengths: "1310nm",
|
||||||
|
standard: "400GBASE-DR4",
|
||||||
|
notes: "400GbE QSFP-DD DR4; IBM FlashSystem next-gen NVMe-oF fabric; OS2 SMF up to 500m",
|
||||||
|
},
|
||||||
|
// ── 10GBASE-T SFP+ ───────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "IBM-SFP-10G-T",
|
||||||
|
formFactor: "SFP+",
|
||||||
|
speedGbps: 10,
|
||||||
|
speed: "10G",
|
||||||
|
reachMeters: 30,
|
||||||
|
reachLabel: "T",
|
||||||
|
fiberType: "DAC",
|
||||||
|
connector: "RJ45",
|
||||||
|
standard: "10GBASE-T",
|
||||||
|
notes: "10GbE copper SFP+ RJ45; IBM iSCSI storage host copper option; Cat6a up to 30m",
|
||||||
|
},
|
||||||
|
// ── 4G FC SFP (legacy) ───────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "IBM-SFP-FC-4G-SW",
|
||||||
|
formFactor: "SFP",
|
||||||
|
speedGbps: 4.25,
|
||||||
|
speed: "4G FC",
|
||||||
|
reachMeters: 150,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "4GFC",
|
||||||
|
notes: "4G FC SFP SW legacy; IBM DS4000/DS8000 legacy FC port; OM2/OM3 MMF",
|
||||||
|
},
|
||||||
|
// ── 64G FC SFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{
|
||||||
|
pid: "IBM-SFP28-64G-FC-SW",
|
||||||
|
formFactor: "SFP28",
|
||||||
|
speedGbps: 56.1,
|
||||||
|
speed: "64G FC",
|
||||||
|
reachMeters: 70,
|
||||||
|
reachLabel: "SW",
|
||||||
|
fiberType: "MMF",
|
||||||
|
connector: "LC",
|
||||||
|
wavelengths: "850nm",
|
||||||
|
standard: "64GFC",
|
||||||
|
notes: "64G FC SFP28 SW; IBM FlashSystem 9500 next-gen NVMe/FC; OM5 WBMMF up to 70m",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeIbmStorageOem(): Promise<void> {
|
||||||
|
console.log("=== IBM Storage OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"IBM Storage",
|
||||||
|
"oem",
|
||||||
|
"https://www.ibm.com/storage",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of IBM_STORAGE_PIDS) {
|
||||||
|
const slug = `ibm-storage-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== IBM Storage OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${IBM_STORAGE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeIbmStorageOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
141
packages/scraper/src/scrapers/ii-vi-oem.ts
Normal file
141
packages/scraper/src/scrapers/ii-vi-oem.ts
Normal file
@ -0,0 +1,141 @@
|
|||||||
|
/**
|
||||||
|
* II-VI / Coherent OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds II-VI Incorporated (now Coherent Corp) branded transceiver PIDs.
|
||||||
|
* II-VI is one of the world's largest vertically integrated optical
|
||||||
|
* transceiver manufacturers, supplying hyperscalers, telcos, and OEMs.
|
||||||
|
* Acquired Finisar in 2019 and rebranded to Coherent Corp in 2022.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Coherent Corp Product Portfolio (coherent.com/networking)
|
||||||
|
* - II-VI Optical Transceiver Datasheet Archive
|
||||||
|
* - OIF 400ZR Implementation Agreement
|
||||||
|
* - IEEE 802.3 Standards (SR4, LR4, ER4, DR4, FR4, LR8)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ii-vi-oem.ts
|
||||||
|
* Cron: daily at 20:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface IiViPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (coherent, DWDM, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"IIVI-SFP10G-ZR",
|
||||||
|
"IIVI-DWDM-SFP10G-TUNE",
|
||||||
|
"IIVI-CFP2-DCO-100G",
|
||||||
|
"IIVI-QSFPDD-400G-ZR",
|
||||||
|
"IIVI-OSFP-800G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const IIVI_PIDS: IiViPID[] = [
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IIVI-SFP10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "II-VI/Coherent 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "IIVI-SFP10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "II-VI/Coherent 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "IIVI-SFP10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "II-VI/Coherent 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "IIVI-SFP10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "II-VI/Coherent 10G SFP+ ZR 80km SMF" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ C-band tunable (Telecom) ──────────────────────────────
|
||||||
|
{ pid: "IIVI-DWDM-SFP10G-TUNE", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", category: "Telecom", notes: "II-VI/Coherent 10G DWDM SFP+ C-band tunable for metro DWDM" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IIVI-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "II-VI/Coherent 25G SFP28 SR 300m MMF" },
|
||||||
|
{ pid: "IIVI-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "II-VI/Coherent 25G SFP28 LR 10km SMF" },
|
||||||
|
{ pid: "IIVI-SFP28-25G-ER", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-ER", notes: "II-VI/Coherent 25G SFP28 ER 30km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IIVI-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "II-VI/Coherent 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "IIVI-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "II-VI/Coherent 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IIVI-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "II-VI/Coherent 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "IIVI-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "II-VI/Coherent 100G QSFP28 LR4 10km SMF" },
|
||||||
|
{ pid: "IIVI-QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "II-VI/Coherent 100G QSFP28 ER4 40km SMF" },
|
||||||
|
|
||||||
|
// ── 100G CFP2-DCO Coherent (Telecom) ────────────────────────────────────
|
||||||
|
{ pid: "IIVI-CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1000000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "II-VI/Coherent 100G CFP2-DCO coherent, up to 1000km" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IIVI-QSFPDD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "II-VI/Coherent 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "IIVI-QSFPDD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4", notes: "II-VI/Coherent 400G QSFP-DD FR4 2km SMF" },
|
||||||
|
{ pid: "IIVI-QSFPDD-400G-LR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR8", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR8", notes: "II-VI/Coherent 400G QSFP-DD LR8 10km SMF 8-lambda" },
|
||||||
|
|
||||||
|
// ── 400G ZR QSFP-DD Coherent (Telecom) ──────────────────────────────────
|
||||||
|
{ pid: "IIVI-QSFPDD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1000000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", standard: "400ZR", category: "Telecom", notes: "II-VI/Coherent 400G ZR QSFP-DD coherent, up to 1000km" },
|
||||||
|
|
||||||
|
// ── 800G OSFP Coherent (Telecom) ────────────────────────────────────────
|
||||||
|
{ pid: "IIVI-OSFP-800G-ZR", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 1000000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "II-VI/Coherent 800G OSFP coherent ZR, up to 1000km" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeIiViOem(): Promise<void> {
|
||||||
|
console.log("=== II-VI / Coherent OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"II-VI / Coherent",
|
||||||
|
"oem",
|
||||||
|
"https://www.coherent.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of IIVI_PIDS) {
|
||||||
|
const slug = `ii-vi-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== II-VI / Coherent OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${IIVI_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeIiViOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
135
packages/scraper/src/scrapers/infinera-groove-oem.ts
Normal file
135
packages/scraper/src/scrapers/infinera-groove-oem.ts
Normal file
@ -0,0 +1,135 @@
|
|||||||
|
/**
|
||||||
|
* Infinera Groove/ICE OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Infinera INF-prefixed transceiver PIDs for the Groove G30/G42
|
||||||
|
* disaggregated open line system and ICE4/ICE6 coherent engine platforms.
|
||||||
|
* Covers standard grey optics, OpenZR+, ZR, OTN/OTU, muxponder modules,
|
||||||
|
* and CFP2-DCO coherent transceivers used across the Infinera portfolio.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Infinera Groove G30/G42 Hardware Compatibility Guide (infinera.com)
|
||||||
|
* - Infinera ICE4/ICE6 Coherent Engine Module Reference
|
||||||
|
* - OIF 400ZR Implementation Agreement
|
||||||
|
* - ITU-T G.709 OTN specification
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/infinera-groove-oem.ts
|
||||||
|
* Cron: daily at 23:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface InfineraGroovePID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
isTelecom?: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
const INFINERA_GROOVE_PIDS: InfineraGroovePID[] = [
|
||||||
|
// ── 1G SFP (management / client) ─────────────────────────────────────────
|
||||||
|
{ pid: "INF-SFP-GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ client optics ────────────────────────────────────────────────
|
||||||
|
{ pid: "INF-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "INF-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "INF-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "INF-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "10G ZR 80km grey optic for Groove G30 client side" },
|
||||||
|
|
||||||
|
// ── OTN / OTU client optics (Telecom) ────────────────────────────────────
|
||||||
|
{ pid: "INF-OTU2-SFP-10G", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "OTU2", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "ITU-T G.709", notes: "OTU2 10G client SFP+ for Groove G30 OTN muxponder", isTelecom: true },
|
||||||
|
{ pid: "INF-OTU2E-SFP-10G", formFactor: "SFP+", speedGbps: 11, speed: "11G", reachMeters: 10000, reachLabel: "OTU2e", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "ITU-T G.709", notes: "OTU2e 11.09Gbps client SFP+ for 10GbE over OTN mapping", isTelecom: true },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "INF-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 client optics ─────────────────────────────────────────────
|
||||||
|
{ pid: "INF-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "INF-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "INF-QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", notes: "100G ER4 40km QSFP28 for Groove G42 inter-site client" },
|
||||||
|
|
||||||
|
// ── GRV2 QSFP28 (Groove G30 integrated coherent) ─────────────────────────
|
||||||
|
{ pid: "INF-GRV2-QSFP28", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000000, reachLabel: "GRV2-Coh", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "Groove G30 Gen2 integrated coherent QSFP28 DSP module, up to 2000km", isTelecom: true },
|
||||||
|
|
||||||
|
// ── CFP2-DCO coherent (Groove G42 line side) ─────────────────────────────
|
||||||
|
{ pid: "INF-CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 2000000, reachLabel: "DCO-100G", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "CFP2-DCO 100G coherent for Groove G42 open line system, up to 2000km", isTelecom: true },
|
||||||
|
{ pid: "INF-CFP2-DCO-200G", formFactor: "CFP2", speedGbps: 200, speed: "200G", reachMeters: 1500000, reachLabel: "DCO-200G", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "CFP2-DCO 200G coherent for Groove G42, up to 1500km dual-polarization QPSK", isTelecom: true },
|
||||||
|
|
||||||
|
// ── OTU4 line optics (Telecom) ────────────────────────────────────────────
|
||||||
|
{ pid: "INF-CFP-OTU4-100G", formFactor: "CFP", speedGbps: 100, speed: "100G", reachMeters: 2000000, reachLabel: "OTU4", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "ITU-T G.709", notes: "CFP OTU4 100G coherent line module for legacy Infinera DTN-X platform", isTelecom: true },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD (ICE4 / Groove G42) ─────────────────────────────────────
|
||||||
|
{ pid: "INF-QSFP-DD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", notes: "400G OpenZR coherent QSFP-DD for ICE4 engine, up to 120km DWDM", isTelecom: true },
|
||||||
|
{ pid: "INF-QSFP-DD-400G-ZRP", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000000, reachLabel: "ZR+", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "400G ZR+ OpenROADM coherent QSFP-DD for ICE4, up to 2000km long-haul", isTelecom: true },
|
||||||
|
{ pid: "INF-ICE4-QSFP-DD-400G", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 4000000, reachLabel: "ICE4-Coh", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "Infinera ICE4 coherent engine QSFP-DD 400G, up to 4000km transoceanic, FlexE", isTelecom: true },
|
||||||
|
|
||||||
|
// ── Muxponder module (Telecom) ────────────────────────────────────────────
|
||||||
|
{ pid: "INF-MXPONDER-10x10G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 2000000, reachLabel: "MXP-10x10G",fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "Groove G30 10x10G muxponder module; aggregates 10x10G client into 100G coherent line", isTelecom: true },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeInfineraGrooveOem(): Promise<void> {
|
||||||
|
console.log("=== Infinera Groove/ICE OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Infinera",
|
||||||
|
"oem",
|
||||||
|
"https://www.infinera.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of INFINERA_GROOVE_PIDS) {
|
||||||
|
const slug = `infinera-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.isTelecom ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
is_oem_seed = true,
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Infinera Groove/ICE OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${INFINERA_GROOVE_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeInfineraGrooveOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
130
packages/scraper/src/scrapers/infinera-oem.ts
Normal file
130
packages/scraper/src/scrapers/infinera-oem.ts
Normal file
@ -0,0 +1,130 @@
|
|||||||
|
/**
|
||||||
|
* Infinera OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Infinera-branded transceiver PIDs for GX Series, XTM-Series,
|
||||||
|
* and Cloud Xpress platforms.
|
||||||
|
*
|
||||||
|
* Sources: Infinera Transceiver Compatibility Guide (infinera.com)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/infinera-oem.ts
|
||||||
|
* Cron: daily at 10:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface InfineraPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const INFINERA_PIDS: InfineraPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-1GIG-MM-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "XCVR-1GIG-SM-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "XCVR-1GIG-SM-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "XCVR-1GIG-RJ45", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "XCVR-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "XCVR-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "XCVR-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "XCVR-10G-LRM", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 220, reachLabel: "LRM", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LRM" },
|
||||||
|
{ pid: "XCVR-10G-DWDM-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "10G DWDM tunable" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "XCVR-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "XCVR-25G-ER", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "XCVR-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "XCVR-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "XCVR-100G-DR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
{ pid: "XCVR-100G-FR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "FR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-FR" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD coherent (GX Series) ───────────────────────────────────
|
||||||
|
{ pid: "XCVR-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "400ZR", notes: "400G coherent QSFP-DD" },
|
||||||
|
{ pid: "XCVR-400G-ZR-PLUS",formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1000000,reachLabel: "ZR+", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "400G ZR+ OpenROADM coherent" },
|
||||||
|
{ pid: "XCVR-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "XCVR-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "XCVR-400G-LR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 800G OSFP (GX G42/G48) ──────────────────────────────────────────────
|
||||||
|
{ pid: "XCVR-800G-ZR", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 120000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band", notes: "800G coherent OSFP ZR" },
|
||||||
|
{ pid: "XCVR-800G-DR8", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 500, reachLabel: "DR8", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", notes: "800G DR8 OSFP" },
|
||||||
|
{ pid: "XCVR-800G-FR8", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 2000, reachLabel: "FR8", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", notes: "800G FR8 OSFP" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DAC-10G-SFP-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "DAC-100G-QSFP28-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "DAC-400G-QSFPDD-1M", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP-DD" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeInfineraOem(): Promise<void> {
|
||||||
|
console.log("=== Infinera OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Infinera",
|
||||||
|
"oem",
|
||||||
|
"https://www.infinera.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of INFINERA_PIDS) {
|
||||||
|
const slug = `infinera-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Infinera OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${INFINERA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeInfineraOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
137
packages/scraper/src/scrapers/innolight-oem.ts
Normal file
137
packages/scraper/src/scrapers/innolight-oem.ts
Normal file
@ -0,0 +1,137 @@
|
|||||||
|
/**
|
||||||
|
* InnoLight Technology OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds InnoLight-branded transceiver PIDs. InnoLight Technology Co., Ltd.
|
||||||
|
* (Shenzhen, China) is one of the world's largest optical transceiver
|
||||||
|
* manufacturers, supplying hyperscalers, cloud providers, and major OEMs.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - InnoLight Product Overview (innolight.com)
|
||||||
|
* - InnoLight 400G/800G Transceiver Specification Sheets
|
||||||
|
* - InnoLight QSFP-DD and OSFP Product Family Guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/innolight-oem.ts
|
||||||
|
* Cron: daily at 21:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface InnoLightPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"IL-SFP-10G-ZR",
|
||||||
|
"IL-DWDM-SFP10G-C",
|
||||||
|
"IL-QSFP28-100G-ZR4",
|
||||||
|
"IL-CFP2-DCO-100G",
|
||||||
|
"IL-QSFP-DD-400G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const INNOLIGHT_PIDS: InnoLightPID[] = [
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IL-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "InnoLight 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "IL-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "InnoLight 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "IL-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "InnoLight 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "IL-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "InnoLight 10G SFP+ ZR 80km SMF" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IL-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "InnoLight 25G SFP28 SR 100m MMF" },
|
||||||
|
{ pid: "IL-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "InnoLight 25G SFP28 LR 10km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IL-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "InnoLight 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "IL-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "InnoLight 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IL-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "InnoLight 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "IL-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "InnoLight 100G QSFP28 LR4 10km SMF" },
|
||||||
|
{ pid: "IL-QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "InnoLight 100G QSFP28 ER4 40km SMF" },
|
||||||
|
{ pid: "IL-QSFP28-100G-ZR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 80000, reachLabel: "ZR4", fiberType: "SMF", connector: "LC", wavelengths: "C-band", standard: "100GBASE-ZR4", notes: "InnoLight 100G QSFP28 ZR4 80km, coherent" },
|
||||||
|
|
||||||
|
// ── 100G CFP2-DCO (Telecom coherent) ─────────────────────────────────────
|
||||||
|
{ pid: "IL-CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1000000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF CFP2-DCO", notes: "InnoLight 100G CFP2 DCO coherent C-band, 1000km" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IL-QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "InnoLight 400G QSFP-DD SR8 100m MMF" },
|
||||||
|
{ pid: "IL-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "InnoLight 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "IL-QSFP-DD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4", notes: "InnoLight 400G QSFP-DD FR4 2km SMF" },
|
||||||
|
{ pid: "IL-QSFP-DD-400G-LR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-LR4", notes: "InnoLight 400G QSFP-DD LR4 10km SMF" },
|
||||||
|
{ pid: "IL-QSFP-DD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1000000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "OIF 400ZR", notes: "InnoLight 400G QSFP-DD ZR coherent, 1000km, C-band" },
|
||||||
|
|
||||||
|
// ── DWDM (Telecom) ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IL-DWDM-SFP10G-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", standard: "ITU-T G.694.1", notes: "InnoLight 10G DWDM SFP+ C-band tunable 80km" },
|
||||||
|
|
||||||
|
// ── 800G OSFP ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IL-OSFP-800G-SR8", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "800GBASE-SR8", notes: "InnoLight 800G OSFP SR8 100m MMF — hyperscaler spine" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeInnoLightOem(): Promise<void> {
|
||||||
|
console.log("=== InnoLight Technology OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"InnoLight Technology",
|
||||||
|
"oem",
|
||||||
|
"https://www.innolight.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of INNOLIGHT_PIDS) {
|
||||||
|
const slug = `innolight-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== InnoLight Technology OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${INNOLIGHT_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeInnoLightOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
131
packages/scraper/src/scrapers/intel-oem.ts
Normal file
131
packages/scraper/src/scrapers/intel-oem.ts
Normal file
@ -0,0 +1,131 @@
|
|||||||
|
/**
|
||||||
|
* Intel OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Intel-branded transceiver PIDs for X710, XXV710, E810, and E830
|
||||||
|
* series network adapter optical transceiver options.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Intel Ethernet Products Compatibility Guide (intel.com)
|
||||||
|
* - Intel Ethernet Adapter X710/XXV710/E810/E830 Hardware Reference
|
||||||
|
* - Intel Ethernet Products Optical Module Specifications
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/intel-oem.ts
|
||||||
|
* Cron: daily at 15:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface IntelPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const INTEL_PIDS: IntelPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "AFCT-701SMZ", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Intel OEM 1G SFP SX" },
|
||||||
|
{ pid: "AFCT-701SDZ", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "AFCT-709SMZ", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "AFCT-701SQPZ", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "E10GSFPSR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Intel E10GSFPSR for X520/X550/X710" },
|
||||||
|
{ pid: "E10GSFPLR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Intel E10GSFPLR for X520/X710" },
|
||||||
|
{ pid: "AFBR-703SDZ", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "AFBR-703SDZ-IN2", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "E10GSFPT", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T", notes: "Intel E10GSFPT for X550" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "E25GSFP28SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Intel 25G SFP28 SR for XXV710/E810" },
|
||||||
|
{ pid: "E25GSFP28LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
{ pid: "E25GSFP28ER", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 30000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "E40GQSFPSR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Intel 40G QSFP+ SR4 for XL710" },
|
||||||
|
{ pid: "E40GQSFPLR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "E100GQSFPSR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Intel 100G QSFP28 SR4 for E810" },
|
||||||
|
{ pid: "E100GQSFPLR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "E100GQSFPCWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "E100GQSFPDR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "E400GQSFPDDSR8", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "Intel 400G QSFP-DD SR8 for E830" },
|
||||||
|
{ pid: "E400GQSFPDDDR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "E400GQSFPDDFR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "E400GQSFPDFLR4", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "XDACBL1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "XDACBL3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "QSFP28DACBL1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "QSFP28DACBL3M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "QSFPDD400GDAC1M", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP-DD" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeIntelOem(): Promise<void> {
|
||||||
|
console.log("=== Intel OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Intel",
|
||||||
|
"oem",
|
||||||
|
"https://www.intel.com/content/www/us/en/products/details/ethernet.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of INTEL_PIDS) {
|
||||||
|
const slug = `intel-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Intel OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${INTEL_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeIntelOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
129
packages/scraper/src/scrapers/ipinfusion-oem.ts
Normal file
129
packages/scraper/src/scrapers/ipinfusion-oem.ts
Normal file
@ -0,0 +1,129 @@
|
|||||||
|
/**
|
||||||
|
* IP Infusion OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds IP Infusion-branded transceiver PIDs for OcNOS white-box NOS platforms (ZebOS).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - IP Infusion OcNOS Hardware Compatibility List (ipinfusion.com)
|
||||||
|
* - IP Infusion ZebOS & OcNOS Transceiver Support Matrix
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ipinfusion-oem.ts
|
||||||
|
* Cron: daily at 03:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface IpInfusionPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const IPINFUSION_PIDS: IpInfusionPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IPI-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "IPI-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "IPI-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "Cu", connector: "RJ45", wavelengths: "N/A", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IPI-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "IPI-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "IPI-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "IPI-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", category: "Telecom", notes: "IP Infusion 10G ZR SFP+ 80km SMF" },
|
||||||
|
{ pid: "IPI-SFP-CWDM-1550", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "CWDM", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", category: "Telecom", notes: "IP Infusion 10G CWDM SFP+ 1550nm channel" },
|
||||||
|
{ pid: "IPI-DWDM-SFP-C", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "IP Infusion 10G DWDM SFP+ C-band fixed channel" },
|
||||||
|
{ pid: "IPI-SFP-OTU2-10G", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "OTU2", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", category: "Telecom", notes: "IP Infusion OTU2 10G SFP+ for OTN transport" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IPI-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "IPI-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IPI-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "IPI-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IPI-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "IPI-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "IPI-QSFP28-100G-ZR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 80000, reachLabel: "ZR4", fiberType: "SMF", connector: "LC", wavelengths: "C-band", category: "Telecom", notes: "IP Infusion 100G ZR4 QSFP28 coherent 80km" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IPI-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (CWDM, DWDM, OTU2, ZR variants)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"IPI-SFP-10G-ZR",
|
||||||
|
"IPI-SFP-CWDM-1550",
|
||||||
|
"IPI-DWDM-SFP-C",
|
||||||
|
"IPI-SFP-OTU2-10G",
|
||||||
|
"IPI-QSFP28-100G-ZR4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeIpInfusionOem(): Promise<void> {
|
||||||
|
console.log("=== IP Infusion OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"IP Infusion",
|
||||||
|
"oem",
|
||||||
|
"https://www.ipinfusion.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of IPINFUSION_PIDS) {
|
||||||
|
const slug = `ipinfusion-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== IP Infusion OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${IPINFUSION_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeIpInfusionOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
141
packages/scraper/src/scrapers/isolan-oem.ts
Normal file
141
packages/scraper/src/scrapers/isolan-oem.ts
Normal file
@ -0,0 +1,141 @@
|
|||||||
|
/**
|
||||||
|
* iSolan OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds iSolan-branded transceiver PIDs for optical components and media
|
||||||
|
* converters. iSolan is a German manufacturer specializing in fiber optic
|
||||||
|
* transceivers, CWDM/BiDi SFPs, and media conversion for enterprise and
|
||||||
|
* carrier access networks.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - iSolan Product Catalog (isolan.com/products/sfp)
|
||||||
|
* - iSolan SFP/SFP+ Datasheet Series (ISL-SFP-*)
|
||||||
|
* - iSolan CWDM SFP Specification Sheet (8-channel 1470-1610nm)
|
||||||
|
* - iSolan QSFP+ / SFP28 Product Range
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/isolan-oem.ts
|
||||||
|
* Cron: daily at 00:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface IsolanPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ISOLAN_PIDS: IsolanPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ISL-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "ISL-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "ISL-SFP-1G-ZX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: "iSolan 80km SMF extended reach SFP" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ISL-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "ISL-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "ISL-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "ISL-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom", notes: "iSolan 80km ZR SFP+ for long-haul access links" },
|
||||||
|
|
||||||
|
// ── CWDM SFP (8-channel 1470–1610nm) ────────────────────────────────────
|
||||||
|
{ pid: "ISL-SFP-CWDM-1470", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1470", fiberType: "SMF", connector: "LC", wavelengths: "1470nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "iSolan CWDM SFP ch1 1470nm, 40km SMF" },
|
||||||
|
{ pid: "ISL-SFP-CWDM-1490", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1490", fiberType: "SMF", connector: "LC", wavelengths: "1490nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "iSolan CWDM SFP ch2 1490nm, 40km SMF" },
|
||||||
|
{ pid: "ISL-SFP-CWDM-1510", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1510", fiberType: "SMF", connector: "LC", wavelengths: "1510nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "iSolan CWDM SFP ch3 1510nm, 40km SMF" },
|
||||||
|
{ pid: "ISL-SFP-CWDM-1530", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1530", fiberType: "SMF", connector: "LC", wavelengths: "1530nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "iSolan CWDM SFP ch4 1530nm, 40km SMF" },
|
||||||
|
{ pid: "ISL-SFP-CWDM-1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1550", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "iSolan CWDM SFP ch5 1550nm, 40km SMF" },
|
||||||
|
{ pid: "ISL-SFP-CWDM-1570", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1570", fiberType: "SMF", connector: "LC", wavelengths: "1570nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "iSolan CWDM SFP ch6 1570nm, 40km SMF" },
|
||||||
|
{ pid: "ISL-SFP-CWDM-1590", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1590", fiberType: "SMF", connector: "LC", wavelengths: "1590nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "iSolan CWDM SFP ch7 1590nm, 40km SMF" },
|
||||||
|
{ pid: "ISL-SFP-CWDM-1610", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "CWDM-1610", fiberType: "SMF", connector: "LC", wavelengths: "1610nm", standard: "1000BASE-CWDM", category: "Telecom", notes: "iSolan CWDM SFP ch8 1610nm, 40km SMF" },
|
||||||
|
|
||||||
|
// ── BiDi SFP ─────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ISL-SFP-BIDI-1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-1310", fiberType: "SMF", connector: "LC", wavelengths: "1310nm TX / 1490nm RX", standard: "1000BASE-BX", category: "Telecom", notes: "iSolan BiDi SFP 1310TX/1490RX single-fiber WDM" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ISL-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "ISL-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1270-1330nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "ISL-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "ISL-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (CWDM, BiDi, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"ISL-SFP-10G-ZR",
|
||||||
|
"ISL-SFP-CWDM-1470",
|
||||||
|
"ISL-SFP-CWDM-1490",
|
||||||
|
"ISL-SFP-CWDM-1510",
|
||||||
|
"ISL-SFP-CWDM-1530",
|
||||||
|
"ISL-SFP-CWDM-1550",
|
||||||
|
"ISL-SFP-CWDM-1570",
|
||||||
|
"ISL-SFP-CWDM-1590",
|
||||||
|
"ISL-SFP-CWDM-1610",
|
||||||
|
"ISL-SFP-BIDI-1310",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeIsolanOem(): Promise<void> {
|
||||||
|
console.log("=== iSolan OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"iSolan",
|
||||||
|
"oem",
|
||||||
|
"https://www.isolan.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of ISOLAN_PIDS) {
|
||||||
|
const slug = `isolan-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== iSolan OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${ISOLAN_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeIsolanOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
132
packages/scraper/src/scrapers/ixia-oem.ts
Normal file
132
packages/scraper/src/scrapers/ixia-oem.ts
Normal file
@ -0,0 +1,132 @@
|
|||||||
|
/**
|
||||||
|
* Ixia / Keysight OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Ixia-branded transceiver PIDs for Keysight (formerly Ixia)
|
||||||
|
* network test and visibility platform optical port hardware.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Keysight Network Test Hardware (keysight.com/us/en/products/network-test/network-test-hardware.html)
|
||||||
|
* - Ixia IxNetwork and IxLoad Platform Hardware Guides
|
||||||
|
* - Ixia Vision Series Network Packet Broker Data Sheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/ixia-oem.ts
|
||||||
|
* Cron: daily at 10:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface IxiaPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs requiring TELECOM/ITU-T classification (CFP, CFP2, ZR, ER4 over WDM)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"IXIA-CFP-100G-LR4",
|
||||||
|
"IXIA-CFP2-100G-LR4",
|
||||||
|
"IXIA-SFP-10G-ZR",
|
||||||
|
"IXIA-QSFP28-100G-ER4",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const IXIA_PIDS: IxiaPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IXIA-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "IXIA-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "IXIA-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IXIA-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "IXIA-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "IXIA-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "IXIA-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "DWDM extended reach" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IXIA-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "IXIA-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IXIA-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "IXIA-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IXIA-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "IXIA-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "IXIA-QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "Extended reach, ITU-T G.694 WDM" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "IXIA-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "IXIA-QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
|
||||||
|
// ── 100G CFP / CFP2 (Telecom) ───────────────────────────────────────────
|
||||||
|
{ pid: "IXIA-CFP-100G-LR4", formFactor: "CFP", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "CFP MSA, OTU4 capable" },
|
||||||
|
{ pid: "IXIA-CFP2-100G-LR4", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "CFP2 MSA, OTU4 capable" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeIxiaOem(): Promise<void> {
|
||||||
|
console.log("=== Ixia / Keysight OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Ixia",
|
||||||
|
"oem",
|
||||||
|
"https://www.keysight.com/us/en/products/network-test/network-test-hardware.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of IXIA_PIDS) {
|
||||||
|
const slug = `ixia-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter";
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Ixia OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${IXIA_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeIxiaOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
132
packages/scraper/src/scrapers/juniper-mx-oem.ts
Normal file
132
packages/scraper/src/scrapers/juniper-mx-oem.ts
Normal file
@ -0,0 +1,132 @@
|
|||||||
|
/**
|
||||||
|
* Juniper MX OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Juniper MX-branded transceiver PIDs for MX480, MX960, MX10003,
|
||||||
|
* and PTX series service provider routers and packet-optical platforms,
|
||||||
|
* including OC-n SONET, CFP2-DCO coherent, and 400G ZR modules.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Juniper MX Series Router Hardware Guide
|
||||||
|
* - Juniper PTX Series Packet Transport Router Data Sheet
|
||||||
|
* - Juniper Optics Compatibility Tool (HCT)
|
||||||
|
* - Juniper Networks Transceiver Module Reference (TN1157)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/juniper-mx-oem.ts
|
||||||
|
* Cron: daily at 13:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface JuniperMxPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category: "DataCenter" | "Telecom";
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// TELECOM_PIDS: OC-3/12/48 SONET, CFP2-DCO coherent, ZR variants.
|
||||||
|
const JUNIPER_MX_PIDS: JuniperMxPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", category: "DataCenter", notes: "Juniper MX 1G multimode SFP" },
|
||||||
|
{ pid: "SFP-1GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", category: "DataCenter", notes: "Juniper MX 1G single-mode SFP 10km" },
|
||||||
|
{ pid: "SFP-1GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", category: "DataCenter", notes: "Juniper MX 1G copper SFP" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10GE-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", category: "DataCenter", notes: "Juniper MX 10G multimode SFP+" },
|
||||||
|
{ pid: "SFP-10GE-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", category: "DataCenter", notes: "Juniper MX 10G single-mode SFP+ 10km" },
|
||||||
|
{ pid: "SFP-10GE-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", category: "DataCenter", notes: "Juniper MX 10G extended-reach SFP+ 40km" },
|
||||||
|
{ pid: "SFP-10GE-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", category: "Telecom", notes: "Juniper MX 10G ZR SFP+ 80km metro/SP" },
|
||||||
|
|
||||||
|
// ── SONET/SDH OC-n SFP ──────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-OC3-MM", formFactor: "SFP", speedGbps: 0.155, speed: "OC3", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Juniper MX OC-3 multimode short-reach SONET SFP" },
|
||||||
|
{ pid: "SFP-OC3-SM10", formFactor: "SFP", speedGbps: 0.155, speed: "OC3", reachMeters: 10000, reachLabel: "LR-10km", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-3/STM-1", category: "Telecom", notes: "Juniper MX OC-3 single-mode 10km SONET SFP" },
|
||||||
|
{ pid: "SFP-OC12-SM10", formFactor: "SFP", speedGbps: 0.622, speed: "OC12", reachMeters: 10000, reachLabel: "LR-10km", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "OC-12/STM-4", category: "Telecom", notes: "Juniper MX OC-12 single-mode 10km SONET SFP" },
|
||||||
|
{ pid: "SFP-OC48-SM", formFactor: "SFP", speedGbps: 2.488, speed: "OC48", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", standard: "OC-48/STM-16", category: "Telecom", notes: "Juniper MX OC-48 short-reach SONET SFP" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", category: "DataCenter", notes: "Juniper MX 40G multimode QSFP+" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", category: "DataCenter", notes: "Juniper MX 40G single-mode QSFP+ 10km" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", category: "DataCenter", notes: "Juniper MX 25G multimode SFP28" },
|
||||||
|
{ pid: "SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", category: "DataCenter", notes: "Juniper MX 25G single-mode SFP28 10km" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", category: "DataCenter", notes: "Juniper MX 100G multimode QSFP28" },
|
||||||
|
{ pid: "QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", category: "DataCenter", notes: "Juniper MX 100G single-mode QSFP28 10km" },
|
||||||
|
{ pid: "QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", category: "DataCenter", notes: "Juniper MX 100G extended-reach QSFP28 40km" },
|
||||||
|
|
||||||
|
// ── Coherent / PTX high-speed ────────────────────────────────────────────
|
||||||
|
{ pid: "CFP2-DCO-200G", formFactor: "CFP2", speedGbps: 200, speed: "200G", reachMeters: 1000000, reachLabel: "DCO-longhaul", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", standard: "OTN/FlexE", category: "Telecom", notes: "Juniper PTX 200G CFP2 digital coherent optics long-haul DWDM" },
|
||||||
|
{ pid: "QSFP-DD-ZR-400G", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 120000, reachLabel: "ZR-120km", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", standard: "400ZR OpenROADM", category: "Telecom", notes: "Juniper PTX 400G QSFP-DD coherent ZR 120km metro DWDM" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeJuniperMxOem(): Promise<void> {
|
||||||
|
console.log("=== Juniper MX OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Juniper MX",
|
||||||
|
"oem",
|
||||||
|
"https://www.juniper.net/us/en/products/routers/mx-series.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of JUNIPER_MX_PIDS) {
|
||||||
|
const slug = `juniper-mx-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
category = EXCLUDED.category,
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const telecomCount = JUNIPER_MX_PIDS.filter(p => p.category === "Telecom").length;
|
||||||
|
const dcCount = JUNIPER_MX_PIDS.filter(p => p.category === "DataCenter").length;
|
||||||
|
|
||||||
|
console.log(`\n=== Juniper MX OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${JUNIPER_MX_PIDS.length} (Telecom: ${telecomCount}, DataCenter: ${dcCount})\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeJuniperMxOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
123
packages/scraper/src/scrapers/juniper-qfx-oem.ts
Normal file
123
packages/scraper/src/scrapers/juniper-qfx-oem.ts
Normal file
@ -0,0 +1,123 @@
|
|||||||
|
/**
|
||||||
|
* Juniper QFX OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Juniper QFX/EX-branded transceiver PIDs for QFX5100, QFX5200,
|
||||||
|
* QFX10000, and EX series data center and campus switches.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Juniper QFX Series Switch Hardware Guide
|
||||||
|
* - Juniper EX Series Ethernet Switch Data Sheets
|
||||||
|
* - Juniper Optics Compatibility Tool (HCT)
|
||||||
|
* - Juniper Networks Transceiver Module Reference (TN1157)
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/juniper-qfx-oem.ts
|
||||||
|
* Cron: daily at 13:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface JuniperQfxPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const JUNIPER_QFX_PIDS: JuniperQfxPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-1GE-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "Juniper QFX/EX 1G multimode SFP" },
|
||||||
|
{ pid: "SFP-1GE-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "Juniper QFX/EX 1G single-mode SFP 10km" },
|
||||||
|
{ pid: "SFP-1GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T", notes: "Juniper QFX/EX 1G copper SFP" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-10GE-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Juniper QFX 10G multimode SFP+" },
|
||||||
|
{ pid: "SFP-10GE-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Juniper QFX 10G single-mode SFP+ 10km" },
|
||||||
|
{ pid: "SFP-10GE-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Juniper QFX 10G extended-reach SFP+ 40km" },
|
||||||
|
{ pid: "SFP-10GE-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Juniper QFX 10G ZR SFP+ 80km" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Juniper QFX 40G multimode QSFP+" },
|
||||||
|
{ pid: "QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4", notes: "Juniper QFX 40G single-mode QSFP+ 10km" },
|
||||||
|
{ pid: "QSFP-40G-SR-BD", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR-BiDi", fiberType: "MMF", connector: "LC", wavelengths: "832nm/853nm", standard: "40GBASE-BiDi", notes: "Juniper QFX 40G BiDi QSFP+ dual-rate over duplex MMF" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Juniper QFX 25G multimode SFP28" },
|
||||||
|
{ pid: "SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Juniper QFX 25G single-mode SFP28 10km" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Juniper QFX 100G multimode QSFP28" },
|
||||||
|
{ pid: "QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Juniper QFX 100G single-mode QSFP28 10km" },
|
||||||
|
{ pid: "QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "Juniper QFX 100G extended-reach QSFP28 40km" },
|
||||||
|
{ pid: "QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4", notes: "Juniper QFX 100G CWDM4 QSFP28 2km SMF" },
|
||||||
|
{ pid: "QSFP28-100G-PSM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "PSM4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "100GBASE-PSM4", notes: "Juniper QFX 100G PSM4 QSFP28 500m parallel SMF" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD / 800G QSFP-DD ────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Juniper QFX 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "QSFP-DD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8", notes: "Juniper QFX 400G QSFP-DD SR8 100m MMF" },
|
||||||
|
{ pid: "QSFP-DD-800G-DR8", formFactor: "QSFP-DD", speedGbps: 800, speed: "800G", reachMeters: 500, reachLabel: "DR8", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "800GBASE-DR8", notes: "Juniper QFX10000 800G QSFP-DD DR8 500m parallel SMF" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeJuniperQfxOem(): Promise<void> {
|
||||||
|
console.log("=== Juniper QFX OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Juniper QFX",
|
||||||
|
"oem",
|
||||||
|
"https://www.juniper.net/us/en/products/switches/qfx-series.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of JUNIPER_QFX_PIDS) {
|
||||||
|
const slug = `juniper-qfx-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Juniper QFX OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${JUNIPER_QFX_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeJuniperQfxOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
130
packages/scraper/src/scrapers/keysight-oem.ts
Normal file
130
packages/scraper/src/scrapers/keysight-oem.ts
Normal file
@ -0,0 +1,130 @@
|
|||||||
|
/**
|
||||||
|
* Keysight Technologies OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Keysight/Ixia-branded transceiver PIDs for IxNetwork,
|
||||||
|
* AresONE, and BreakingPoint network test platforms.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Keysight Network Test Hardware Compatibility Guide
|
||||||
|
* - IxNetwork Platform Data Sheets
|
||||||
|
* - AresONE 400G/800G Platform Guide
|
||||||
|
* - BreakingPoint Appliance Hardware Reference
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/keysight-oem.ts
|
||||||
|
* Cron: daily at 20:00
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface KeysightPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const KEYSIGHT_PIDS: KeysightPID[] = [
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KS-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "KS-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "KS-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "KS-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "KS-SFP-10G-T", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KS-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "KS-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KS-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "KS-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KS-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "KS-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "KS-QSFP28-100G-CWDM4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "KS-QSFP28-100G-DR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
{ pid: "KS-QSFP28-100G-FR", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "FR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-FR" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KS-QSFPDD-400G-SR8", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 100, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "400GBASE-SR8" },
|
||||||
|
{ pid: "KS-QSFPDD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
{ pid: "KS-QSFPDD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4" },
|
||||||
|
{ pid: "KS-QSFPDD-400G-LR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "400GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 800G OSFP ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KS-OSFP-800G-SR8", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 500, reachLabel: "SR8", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "800GBASE-SR8" },
|
||||||
|
{ pid: "KS-OSFP-800G-DR8", formFactor: "OSFP", speedGbps: 800, speed: "800G", reachMeters: 2000, reachLabel: "DR8", fiberType: "SMF", connector: "LC", wavelengths: "1310nm" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KS-DAC-10G-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "KS-DAC-10G-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "KS-DAC-100G-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "KS-DAC-100G-3M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "KS-DAC-400G-1M", formFactor: "QSFP-DD",speedGbps: 400, speed: "400G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP-DD" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeKeysightOem(): Promise<void> {
|
||||||
|
console.log("=== Keysight Technologies OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Keysight Technologies",
|
||||||
|
"oem",
|
||||||
|
"https://www.keysight.com/us/en/products/network-test.html",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of KEYSIGHT_PIDS) {
|
||||||
|
const slug = `keysight-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Keysight Technologies OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${KEYSIGHT_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeKeysightOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
131
packages/scraper/src/scrapers/kontron-oem.ts
Normal file
131
packages/scraper/src/scrapers/kontron-oem.ts
Normal file
@ -0,0 +1,131 @@
|
|||||||
|
/**
|
||||||
|
* Kontron OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Kontron-branded transceiver PIDs for ATCA/telecom platforms (S&T Group).
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Kontron Product Portfolio — Telecom & ATCA Line Cards (kontron.com)
|
||||||
|
* - Kontron ATCA & CompactPCI Transceiver Compatibility Matrix
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/kontron-oem.ts
|
||||||
|
* Cron: daily at 02:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface KontronPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const KONTRON_PIDS: KontronPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KT-SFP-1G-SX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX" },
|
||||||
|
{ pid: "KT-SFP-1G-LX", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX" },
|
||||||
|
{ pid: "KT-SFP-GE-T", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "Cu", connector: "RJ45", wavelengths: "N/A", standard: "1000BASE-T" },
|
||||||
|
|
||||||
|
// ── OC-3 / OC-12 SONET SFP ──────────────────────────────────────────────
|
||||||
|
{ pid: "KT-SFP-OC3-SR", formFactor: "SFP", speedGbps: 0.155, speed: "OC-3", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", category: "Telecom", notes: "Kontron OC-3 / STM-1 SFP short reach" },
|
||||||
|
{ pid: "KT-SFP-OC12-SR", formFactor: "SFP", speedGbps: 0.622, speed: "OC-12", reachMeters: 2000, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "1310nm", category: "Telecom", notes: "Kontron OC-12 / STM-4 SFP short reach" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KT-SFP-10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "KT-SFP-10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "KT-SFP-10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "KT-SFP-10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", category: "Telecom", notes: "Kontron 10G ZR SFP+ 80km SMF" },
|
||||||
|
{ pid: "KT-SFP-OTU2-10G", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "OTU2", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", category: "Telecom", notes: "Kontron OTU2 10G SFP+ for OTN transport" },
|
||||||
|
{ pid: "KT-SFP-CWDM-1550", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "CWDM", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", category: "Telecom", notes: "Kontron 10G CWDM SFP+ 1550nm channel" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KT-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "KT-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KT-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "KT-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KT-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "KT-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KT-QSFP-DD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4" },
|
||||||
|
];
|
||||||
|
|
||||||
|
// PIDs that use 'Telecom' category (OC-3/12, OTU2, CWDM, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"KT-SFP-OC3-SR",
|
||||||
|
"KT-SFP-OC12-SR",
|
||||||
|
"KT-SFP-10G-ZR",
|
||||||
|
"KT-SFP-OTU2-10G",
|
||||||
|
"KT-SFP-CWDM-1550",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export async function scrapeKontronOem(): Promise<void> {
|
||||||
|
console.log("=== Kontron OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Kontron",
|
||||||
|
"oem",
|
||||||
|
"https://www.kontron.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of KONTRON_PIDS) {
|
||||||
|
const slug = `kontron-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Kontron OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${KONTRON_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeKontronOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
115
packages/scraper/src/scrapers/korenix-oem.ts
Normal file
115
packages/scraper/src/scrapers/korenix-oem.ts
Normal file
@ -0,0 +1,115 @@
|
|||||||
|
/**
|
||||||
|
* Korenix Technology OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds transceiver PIDs for Korenix industrial-grade SFP/SFP+ modules used in
|
||||||
|
* railway, power, and manufacturing automation Ethernet switches.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Korenix KSM series SFP module datasheets (korenix.com)
|
||||||
|
* - Korenix JetNet/JetNet Plus series hardware guides
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/korenix-oem.ts
|
||||||
|
* Cron: daily at 17:45
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface KorenixPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const KORENIX_PIDS: KorenixPID[] = [
|
||||||
|
// ── 100M SFP ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KSM-100M-MM2-LC", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 2000, reachLabel: "FX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "100BASE-FX", notes: undefined },
|
||||||
|
{ pid: "KSM-100M-SM15-LC", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 15000, reachLabel: "LFX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100BASE-LFX", notes: undefined },
|
||||||
|
{ pid: "KSM-100M-SM40-LC", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 40000, reachLabel: "LFX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: undefined, notes: "Korenix 100M SM SFP 40km" },
|
||||||
|
{ pid: "KSM-100M-BX20U-LC", formFactor: "SFP", speedGbps: 0.1, speed: "100M", reachMeters: 20000, reachLabel: "BiDi-20K", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550", standard: undefined, notes: "Korenix 100M BiDi SFP upstream" },
|
||||||
|
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KSM-1000M-MM550-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: undefined },
|
||||||
|
{ pid: "KSM-1000M-SM10-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: undefined },
|
||||||
|
{ pid: "KSM-1000M-SM20-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: undefined, notes: "Korenix 1G SM SFP 20km" },
|
||||||
|
{ pid: "KSM-1000M-SM40-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 40000, reachLabel: "LH", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: undefined, notes: "Korenix 1G SM SFP 40km" },
|
||||||
|
{ pid: "KSM-1000M-SM80-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX", notes: undefined },
|
||||||
|
{ pid: "KSM-1000M-RJ45", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", wavelengths: undefined, standard: "1000BASE-T", notes: undefined },
|
||||||
|
{ pid: "KSM-1000M-BX20U-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-20K", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550", standard: undefined, notes: "Korenix 1G BiDi SFP upstream" },
|
||||||
|
{ pid: "KSM-1000M-BX20D-LC", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-20K", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310", standard: undefined, notes: "Korenix 1G BiDi SFP downstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KSM-10G-SR-MM300-LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: undefined },
|
||||||
|
{ pid: "KSM-10G-LR-SM10-LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: undefined },
|
||||||
|
{ pid: "KSM-10G-ER-SM40-LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: undefined },
|
||||||
|
{ pid: "KSM-10G-ZR-SM80-LC", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: undefined },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "KSM-10G-DAC-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+", wavelengths: undefined, standard: undefined, notes: undefined },
|
||||||
|
{ pid: "KSM-10G-DAC-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+", wavelengths: undefined, standard: undefined, notes: undefined },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeKorenixOem(): Promise<void> {
|
||||||
|
console.log("=== Korenix Technology OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Korenix Technology",
|
||||||
|
"oem",
|
||||||
|
"https://www.korenix.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of KORENIX_PIDS) {
|
||||||
|
const slug = `korenix-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','Industrial',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Korenix OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${KORENIX_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeKorenixOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
126
packages/scraper/src/scrapers/lancom-oem.ts
Normal file
126
packages/scraper/src/scrapers/lancom-oem.ts
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
/**
|
||||||
|
* LANCOM Systems OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds LANCOM-branded transceiver PIDs for GS-2xxx, XS-5xxx, and XS-6xxx
|
||||||
|
* enterprise/campus switch and router series.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - LANCOM Systems Product Catalog (lancom-systems.com)
|
||||||
|
* - LANCOM Switch Hardware Compatibility Guide
|
||||||
|
* - LANCOM XS-6xxx/XS-5xxx Series Data Sheets
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/lancom-oem.ts
|
||||||
|
* Cron: daily at 13:30
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface LancomPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
const LANCOM_PIDS: LancomPID[] = [
|
||||||
|
// ── 1G SFP ──────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-LX-SM1310", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 10000, reachLabel: "LX", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "1000BASE-LX", notes: "LANCOM 1G SM SFP LX" },
|
||||||
|
{ pid: "SFP-SX-MM850", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 550, reachLabel: "SX", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "1000BASE-SX", notes: "LANCOM 1G MM SFP SX" },
|
||||||
|
{ pid: "SFP-ZX-SM1550", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 80000, reachLabel: "ZX", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "1000BASE-ZX" },
|
||||||
|
{ pid: "SFP-T-RJ45", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 100, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "1000BASE-T" },
|
||||||
|
{ pid: "SFP-BIDI-U", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-U", fiberType: "SMF", connector: "LC", wavelengths: "TX1310/RX1550", notes: "LANCOM 1G BiDi SFP upstream" },
|
||||||
|
{ pid: "SFP-BIDI-D", formFactor: "SFP", speedGbps: 1, speed: "1G", reachMeters: 20000, reachLabel: "BiDi-D", fiberType: "SMF", connector: "LC", wavelengths: "TX1550/RX1310", notes: "LANCOM 1G BiDi SFP downstream" },
|
||||||
|
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP-Plus-SR-MM850", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR" },
|
||||||
|
{ pid: "SFP-Plus-LR-SM1310", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR" },
|
||||||
|
{ pid: "SFP-Plus-ER-SM1550", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER" },
|
||||||
|
{ pid: "SFP-Plus-ZR-SM1550", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR" },
|
||||||
|
{ pid: "SFP-Plus-T-RJ45", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 30, reachLabel: "T", fiberType: "DAC", connector: "RJ45", standard: "10GBASE-T" },
|
||||||
|
{ pid: "SFP-Plus-BIDI-U", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "BiDi-U", fiberType: "SMF", connector: "LC", wavelengths: "TX1270/RX1330", notes: "LANCOM 10G BiDi SFP+ upstream" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "SFP28-SR-MM850", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 100, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR" },
|
||||||
|
{ pid: "SFP28-LR-SM1310", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP-Plus-SR4-MM850", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4" },
|
||||||
|
{ pid: "QSFP-Plus-LR4-SM1310", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "QSFP28-SR4-MM850", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4" },
|
||||||
|
{ pid: "QSFP28-LR4-SM1310", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4" },
|
||||||
|
{ pid: "QSFP28-CWDM4-SM", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 2000, reachLabel: "CWDM4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "100GBASE-CWDM4" },
|
||||||
|
{ pid: "QSFP28-DR-SM1310", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 500, reachLabel: "DR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "100GBASE-DR" },
|
||||||
|
|
||||||
|
// ── DAC ─────────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "DAC-SFP-Plus-1M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "DAC-SFP-Plus-3M", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "SFP+" },
|
||||||
|
{ pid: "DAC-QSFP28-1M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 1, reachLabel: "DAC-1M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
{ pid: "DAC-QSFP28-3M", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 3, reachLabel: "DAC-3M", fiberType: "DAC", connector: "QSFP28" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeLancomOem(): Promise<void> {
|
||||||
|
console.log("=== LANCOM Systems OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"LANCOM Systems",
|
||||||
|
"oem",
|
||||||
|
"https://www.lancom-systems.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of LANCOM_PIDS) {
|
||||||
|
const slug = `lancom-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream','DataCenter',$13)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== LANCOM Systems OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${LANCOM_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeLancomOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
135
packages/scraper/src/scrapers/lumentum-oem.ts
Normal file
135
packages/scraper/src/scrapers/lumentum-oem.ts
Normal file
@ -0,0 +1,135 @@
|
|||||||
|
/**
|
||||||
|
* Lumentum OEM Transceiver Catalog Seed
|
||||||
|
*
|
||||||
|
* Seeds Lumentum-branded transceiver PIDs covering DataCenter and Telecom
|
||||||
|
* portfolios. Lumentum is a leading OEM optical transceiver manufacturer
|
||||||
|
* supplying modules to major network vendors and hyperscalers.
|
||||||
|
*
|
||||||
|
* Sources:
|
||||||
|
* - Lumentum Product Catalog (lumentum.com/en-us/products/optical-transceivers)
|
||||||
|
* - Lumentum 400ZR / ZR+ Coherent Module Datasheet
|
||||||
|
* - IEEE 802.3 / OIF 400ZR / OpenZR+ MSA Standards
|
||||||
|
*
|
||||||
|
* Run: tsx packages/scraper/src/scrapers/lumentum-oem.ts
|
||||||
|
* Cron: daily at 20:15
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { pool, ensureVendor } from "../utils/db";
|
||||||
|
|
||||||
|
interface LumentumPID {
|
||||||
|
pid: string;
|
||||||
|
formFactor: string;
|
||||||
|
speedGbps: number;
|
||||||
|
speed: string;
|
||||||
|
reachMeters: number;
|
||||||
|
reachLabel: string;
|
||||||
|
fiberType: string;
|
||||||
|
connector: string;
|
||||||
|
wavelengths?: string;
|
||||||
|
standard?: string;
|
||||||
|
category?: string;
|
||||||
|
notes?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PIDs that belong to the Telecom category (coherent, DWDM, ZR)
|
||||||
|
const TELECOM_PIDS = new Set([
|
||||||
|
"LMT-SFP10G-ZR",
|
||||||
|
"LMT-DWDM-SFP10G-TUNE",
|
||||||
|
"LMT-QSFP28-100G-ZR4",
|
||||||
|
"LMT-CFP2-DCO-100G",
|
||||||
|
"LMT-QSFPDD-400G-ZR",
|
||||||
|
]);
|
||||||
|
|
||||||
|
const LUMENTUM_PIDS: LumentumPID[] = [
|
||||||
|
// ── 10G SFP+ ────────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LMT-SFP10G-SR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "10GBASE-SR", notes: "Lumentum 10G SFP+ SR 300m MMF" },
|
||||||
|
{ pid: "LMT-SFP10G-LR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "10GBASE-LR", notes: "Lumentum 10G SFP+ LR 10km SMF" },
|
||||||
|
{ pid: "LMT-SFP10G-ER", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 40000, reachLabel: "ER", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ER", notes: "Lumentum 10G SFP+ ER 40km SMF" },
|
||||||
|
{ pid: "LMT-SFP10G-ZR", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "1550nm", standard: "10GBASE-ZR", notes: "Lumentum 10G SFP+ ZR 80km SMF" },
|
||||||
|
|
||||||
|
// ── 10G DWDM SFP+ C-band tunable (Telecom) ──────────────────────────────
|
||||||
|
{ pid: "LMT-DWDM-SFP10G-TUNE", formFactor: "SFP+", speedGbps: 10, speed: "10G", reachMeters: 80000, reachLabel: "DWDM", fiberType: "SMF", connector: "LC", wavelengths: "C-band tunable", category: "Telecom", notes: "Lumentum 10G DWDM SFP+ C-band tunable for metro/long-haul DWDM rings" },
|
||||||
|
|
||||||
|
// ── 25G SFP28 ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LMT-SFP28-25G-SR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 300, reachLabel: "SR", fiberType: "MMF", connector: "LC", wavelengths: "850nm", standard: "25GBASE-SR", notes: "Lumentum 25G SFP28 SR 300m MMF" },
|
||||||
|
{ pid: "LMT-SFP28-25G-LR", formFactor: "SFP28", speedGbps: 25, speed: "25G", reachMeters: 10000, reachLabel: "LR", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "25GBASE-LR", notes: "Lumentum 25G SFP28 LR 10km SMF" },
|
||||||
|
|
||||||
|
// ── 40G QSFP+ ───────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LMT-QSFP-40G-SR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 150, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "40GBASE-SR4", notes: "Lumentum 40G QSFP+ SR4 150m MMF" },
|
||||||
|
{ pid: "LMT-QSFP-40G-LR4", formFactor: "QSFP+", speedGbps: 40, speed: "40G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1310nm", standard: "40GBASE-LR4", notes: "Lumentum 40G QSFP+ LR4 10km SMF" },
|
||||||
|
|
||||||
|
// ── 100G QSFP28 ─────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LMT-QSFP28-100G-SR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 100, reachLabel: "SR4", fiberType: "MMF", connector: "MPO", wavelengths: "850nm", standard: "100GBASE-SR4", notes: "Lumentum 100G QSFP28 SR4 100m MMF" },
|
||||||
|
{ pid: "LMT-QSFP28-100G-LR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 10000, reachLabel: "LR4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-LR4", notes: "Lumentum 100G QSFP28 LR4 10km SMF" },
|
||||||
|
{ pid: "LMT-QSFP28-100G-ER4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 40000, reachLabel: "ER4", fiberType: "SMF", connector: "LC", wavelengths: "1295-1310nm", standard: "100GBASE-ER4", notes: "Lumentum 100G QSFP28 ER4 40km SMF" },
|
||||||
|
{ pid: "LMT-QSFP28-100G-ZR4", formFactor: "QSFP28", speedGbps: 100, speed: "100G", reachMeters: 80000, reachLabel: "ZR4", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "Lumentum 100G QSFP28 ZR4 80km coherent" },
|
||||||
|
|
||||||
|
// ── 100G CFP2-DCO Coherent (Telecom) ────────────────────────────────────
|
||||||
|
{ pid: "LMT-CFP2-DCO-100G", formFactor: "CFP2", speedGbps: 100, speed: "100G", reachMeters: 1000000, reachLabel: "DCO", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", category: "Telecom", notes: "Lumentum 100G CFP2-DCO coherent, up to 1000km" },
|
||||||
|
|
||||||
|
// ── 400G QSFP-DD ────────────────────────────────────────────────────────
|
||||||
|
{ pid: "LMT-QSFPDD-400G-DR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 500, reachLabel: "DR4", fiberType: "SMF", connector: "MPO", wavelengths: "1310nm", standard: "400GBASE-DR4", notes: "Lumentum 400G QSFP-DD DR4 500m SMF" },
|
||||||
|
{ pid: "LMT-QSFPDD-400G-FR4", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 2000, reachLabel: "FR4", fiberType: "SMF", connector: "LC", wavelengths: "1271-1331nm", standard: "400GBASE-FR4", notes: "Lumentum 400G QSFP-DD FR4 2km SMF" },
|
||||||
|
|
||||||
|
// ── 400G ZR QSFP-DD Coherent (Telecom) ──────────────────────────────────
|
||||||
|
{ pid: "LMT-QSFPDD-400G-ZR", formFactor: "QSFP-DD", speedGbps: 400, speed: "400G", reachMeters: 1000000, reachLabel: "ZR", fiberType: "SMF", connector: "LC", wavelengths: "C-band DWDM", standard: "400ZR", category: "Telecom", notes: "Lumentum 400G ZR QSFP-DD coherent, up to 1000km" },
|
||||||
|
];
|
||||||
|
|
||||||
|
export async function scrapeLumentumOem(): Promise<void> {
|
||||||
|
console.log("=== Lumentum OEM Transceiver Seed ===\n");
|
||||||
|
|
||||||
|
const vendorId = await ensureVendor(
|
||||||
|
"Lumentum",
|
||||||
|
"oem",
|
||||||
|
"https://www.lumentum.com",
|
||||||
|
undefined
|
||||||
|
);
|
||||||
|
|
||||||
|
let inserted = 0;
|
||||||
|
let updated = 0;
|
||||||
|
let errors = 0;
|
||||||
|
|
||||||
|
for (const p of LUMENTUM_PIDS) {
|
||||||
|
const slug = `lumentum-${p.pid.toLowerCase().replace(/[^a-z0-9]+/g, "-")}`;
|
||||||
|
const category = p.category ?? (TELECOM_PIDS.has(p.pid) ? "Telecom" : "DataCenter");
|
||||||
|
try {
|
||||||
|
const res = await pool.query(
|
||||||
|
`INSERT INTO transceivers
|
||||||
|
(slug, part_number, vendor_id, form_factor, speed, speed_gbps,
|
||||||
|
reach_meters, reach_label, fiber_type, connector, wavelengths,
|
||||||
|
dom_support, ieee_reference, market_status, category, notes)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,true,$12,'Mainstream',$13,$14)
|
||||||
|
ON CONFLICT (slug) DO UPDATE SET
|
||||||
|
speed_gbps = EXCLUDED.speed_gbps,
|
||||||
|
reach_meters = CASE WHEN EXCLUDED.reach_meters > 0 THEN EXCLUDED.reach_meters ELSE transceivers.reach_meters END,
|
||||||
|
fiber_type = CASE WHEN EXCLUDED.fiber_type <> '' THEN EXCLUDED.fiber_type ELSE transceivers.fiber_type END,
|
||||||
|
wavelengths = COALESCE(EXCLUDED.wavelengths, transceivers.wavelengths),
|
||||||
|
updated_at = NOW()
|
||||||
|
RETURNING (xmax = 0) as was_inserted`,
|
||||||
|
[slug, p.pid, vendorId, p.formFactor, p.speed, p.speedGbps,
|
||||||
|
p.reachMeters, p.reachLabel, p.fiberType, p.connector,
|
||||||
|
p.wavelengths ?? null, p.standard ?? null, category, p.notes ?? null]
|
||||||
|
);
|
||||||
|
if (res.rows[0]?.was_inserted) inserted++; else updated++;
|
||||||
|
} catch (err) {
|
||||||
|
console.warn(` Skip ${p.pid}: ${(err as Error).message.slice(0, 80)}`);
|
||||||
|
errors++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n=== Lumentum OEM Seed Complete ===`);
|
||||||
|
console.log(` Inserted: ${inserted}`);
|
||||||
|
console.log(` Updated: ${updated}`);
|
||||||
|
console.log(` Errors: ${errors}`);
|
||||||
|
console.log(` Total PIDs: ${LUMENTUM_PIDS.length}\n`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (require.main === module) {
|
||||||
|
scrapeLumentumOem()
|
||||||
|
.then(() => pool.end())
|
||||||
|
.catch((err) => {
|
||||||
|
console.error("Fatal:", err);
|
||||||
|
pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
});
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user