rene.fichtmueller/transceiver-db
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity
506 Commits 5 Branches 9 Tags
Commit Graph

22 Commits

Author SHA1 Message Date
Rene Fichtmueller
86d1531d52 fix(security): scrub hardcoded DB credentials, untrack training data, add scan allowlist 
- Replace plaintext tip (transceiver_db) and llm_gateway DB passwords with
  env-var references guarded by fail-fast checks, across 15 scripts + sql runbook.
  Covers quoted exports, unquoted psql calls, python env dicts and
  env-or-hardcoded-default fallbacks.
- Untrack training-data/*.jsonl (~48MB, kept on disk) and gitignore *.jsonl.
- Add .security-scan-allowlist for legit false positives: vendor name flexoptix,
  own homelab infra, sibling-project codenames in the sync journal, env-resolved
  DB URIs, example placeholders, ML tokenizer tokens.
- Pre-push leak scanner goes from 397 to 0 findings, green without --no-verify.

Follow-up (not in this commit): rotate the leaked values (llm_gateway first,
tip fleet-wide coordinated); git history still holds the old passwords.
 2026-07-07 12:06:07 +02:00
Rene Fichtmueller
db6b97186a feat: OPN+spec equivalence matchers, 400G pricing, TIP_LLM training data 
- Add OPN-based equivalence matcher robot (7,245 manufacturer-confirmed matches, confidence=1.0)
- Add spec-based equivalence matcher robot (683 matches, confidence=0.85)
  - Matches by form_factor + speed_gbps + reach_tier + wavelength ±10nm
  - Safety cap: skip FX products matching >30 competitors (too generic)
  - Daily schedule: 04:30 UTC via pg-boss
- SQL migrations 116 (OPN) + 117 (spec) with tip_extract_wavelength_nm() + tip_reach_tier() helpers
- Fix tenGtek.ts: add 3 missing 400G categories (QSFP-DD, QSFP112) — closes pricing gap
- Generate tip-llm-pricing-v1.jsonl: 80 DB-grounded QA pairs (pricing, equivalences, 400G)
- Rebuild TIP_LLM training pool: 11,999 pairs (+127 vs prev), deployed to Erik
- FX product equivalence coverage: 88.1% (959/1089)
 2026-05-13 21:33:19 +02:00
Rene Fichtmueller
a0657ee565 fix: filter TIP hot topics quality 2026-05-10 15:54:38 +02:00
Rene Fichtmueller
e5917a2250 fix: show active TIP product scope 2026-05-10 15:46:41 +02:00
Rene Fichtmueller
58a2570842 fix: show TIP research status on overview 2026-05-10 15:01:22 +02:00
Rene Fichtmueller
5eb1b07183 fix: close stale TIP manual review queue 2026-05-10 10:23:07 +02:00
Rene Fichtmueller
cf0e471fa4 feat: close TIP research resolution states 2026-05-10 10:13:09 +02:00
Rene Fichtmueller
7e36236d2b fix: quarantine GAO catalog artifacts 2026-05-10 09:48:43 +02:00
Rene Fichtmueller
d691745c7b feat: clean TIP cable rows from active base 2026-05-10 09:41:59 +02:00
Rene Fichtmueller
0599991431 sync: record TIP price closure follow-up 2026-05-10 01:49:42 +02:00
Rene Fichtmueller
2be61f2441 feat: close TIP retail price research states 2026-05-10 01:42:24 +02:00
Rene Fichtmueller
b58f7cee41 feat: resolve OEM price status and part details 2026-05-10 01:16:49 +02:00
Rene Fichtmueller
adb2661fac feat: add targeted product page asset verifier 2026-05-10 00:31:33 +02:00
Rene Fichtmueller
0d4bcb6924 fix: preserve explicit competitor states in reconcile 2026-05-10 00:17:26 +02:00
Rene Fichtmueller
635a102932 feat: close open competitor research states 2026-05-10 00:03:42 +02:00
Rene Fichtmueller
fb9db56617 fix: quarantine fs numeric sku aliases 2026-05-09 23:35:01 +02:00
Rene Fichtmueller
7b8e229cf0 sync: record no-valid matcher closure 2026-05-09 23:24:55 +02:00
Rene Fichtmueller
79a57a5ac6 feat: add no-valid competitor resolver 2026-05-09 23:16:04 +02:00
Rene Fichtmueller
650de6ba9a feat: add verification evidence state model 2026-05-09 23:06:21 +02:00
Rene Fichtmueller
1af4f090f7 fix: harden TIP verification cleanup 2026-05-09 22:16:29 +02:00
Rene Fichtmueller
8e367b3c33 feat: TIP_LLM 5-capability training data + updated system prompt 
- Add scripts/seed-tip-llm-capabilities.ts: generator for 34 SFT pairs
  covering all 5 TIP_LLM capabilities (transceiver research, switch
  research, Blog_LLM data evaluation, crawler/scraper design, Hype Cycle)
- Add training-data/tip-llm-capabilities-v1.jsonl: generated output (34 pairs)
- Update tip-learning-pool-build.ts: expanded 5-capability system prompt
  replaces single-line prompt; register capabilities file in files.tip_llm
- Regenerate tip_llm runpod outputs: 12141 raw pairs → 11872 training pairs
  (up from 10654 before capabilities addition)
- Published tip_llm (11872 pairs) + blog_llm (11408 pairs) to HuggingFace
 2026-04-26 00:01:21 +02:00
Rene Fichtmueller
8c625ff1d2 feat(training): add TIP selflearning workflows 2026-04-25 12:21:56 +02:00