134 lines
5.6 KiB
Markdown
134 lines
5.6 KiB
Markdown
# Open Source Gap Analysis
|
|
|
|
This document maps the current Context-X LLM Gateway to the planned open-source Adaptive LLM Gateway.
|
|
|
|
## Current Strengths
|
|
|
|
Already present in the repository:
|
|
|
|
| Area | Current State | Notes |
|
|
|---|---|---|
|
|
| Gateway API | Present | Fastify gateway in `packages/gateway`. |
|
|
| Completion API | Present | Main route: `/v1/completion`. |
|
|
| Classification | Present | `/v1/classify` and pre-classifier pipeline. |
|
|
| Batch jobs | Present | `/v1/batch` and PgBoss queue integration. |
|
|
| Health checks | Present | `/health`, `/health/live`, `/health/ready`. |
|
|
| Metrics | Present | Prometheus metrics and dashboard metrics. |
|
|
| Dashboard | Present | Operational dashboard exists in `packages/gateway/public`. |
|
|
| Routing rules | Present | YAML routing rules and model tiers. |
|
|
| Local model routing | Present | Ollama-based routing and fallback chains. |
|
|
| Hosted providers | Partial | External provider registry exists. Needs OSS cleanup and discovery. |
|
|
| Cost tracking | Present | Cost analytics, token tracking, cost stream. |
|
|
| Compression accounting | Partial | TokenVault/cost hooks exist. Needs first-class compression engine. |
|
|
| Learning engine | Present | Learning cycles, model performance tracking, fine-tuner package. |
|
|
| Client SDK | Present | `@llm-gateway/client`. |
|
|
| OpenAI compatibility | Partial | `chatgpt-api-adapter` and `openai-bridge` exist. Needs clean OSS path. |
|
|
| Codex integration | Partial | `packages/codex-lsp-adapter` exists. Needs production hardening. |
|
|
| Claude Code integration | Partial | `packages/claude-code-bridge` exists. Needs MCP-first flow. |
|
|
| LightRAG/RAG | Present | LightRAG sidecar exists. Needs generic connector story. |
|
|
| Handoff sync | Partial | `sync/` handoff folder exists. Needs protocol and tools. |
|
|
| Gitea use | Present internally | Needs generic Gitea memory backend. |
|
|
|
|
## Missing For Open Source
|
|
|
|
These features need to be added or extracted:
|
|
|
|
| Feature | Status | Priority | Target Package/Area |
|
|
|---|---|---:|---|
|
|
| First-run setup wizard | Missing | P0 | `packages/cli`, `packages/discovery` |
|
|
| Local AI discovery | Missing | P0 | `packages/discovery` |
|
|
| Public provider discovery | Partial | P0 | `packages/discovery`, `packages/providers` |
|
|
| AI client detection | Missing | P0 | `packages/discovery` |
|
|
| MCP server | Missing | P0 | `packages/mcp-server` |
|
|
| Trust Router | Missing | P0 | `packages/trust-router` |
|
|
| Consent Ledger | Missing | P0 | `packages/consent-ledger` |
|
|
| Shared Gitea Memory | Missing | P0 | `packages/memory-sync` |
|
|
| Context Receipt | Missing | P0 | `packages/context-receipts` |
|
|
| AI Handoff Protocol | Partial | P0 | `packages/handoff` |
|
|
| Safe Config Writer | Missing | P0 | `packages/config-writer` |
|
|
| Setup Doctor | Missing | P0 | `packages/doctor` |
|
|
| Offline Mode | Missing | P0 | gateway config/policy |
|
|
| Capability Benchmark Lab | Missing | P1 | `packages/benchmark-lab` |
|
|
| Agent Reputation Score | Missing | P1 | `packages/agent-reputation` |
|
|
| Reproducible Runs | Missing | P1 | `packages/run-ledger` |
|
|
| Visual Topology Map | Missing | P1 | dashboard UI/API |
|
|
| Integration Marketplace | Missing | P1 | `packages/integrations` + UI |
|
|
| Data source connectors | Missing | P1 | `packages/connectors` |
|
|
| Context Compression Engine | Partial | P1 | `packages/context-compression` |
|
|
| Semantic cache | Missing/mentioned | P1 | `packages/cache` |
|
|
| Team mode | Missing | P2 | auth/policy/admin UI |
|
|
| Prompt/agent versioning | Partial | P2 | memory/git/prompt registry |
|
|
| Migration/import wizard | Missing | P2 | `packages/import-wizard` |
|
|
|
|
## Context-X Assumptions To Remove
|
|
|
|
Before public release, remove or move behind an example profile:
|
|
|
|
- hardcoded `context-x.org` domains
|
|
- hardcoded `fichtmueller.org` Ollama endpoint
|
|
- Erik-specific paths such as `/opt/llm-gateway`
|
|
- private project callers and templates as defaults
|
|
- internal IP assumptions
|
|
- private training data
|
|
- private bridge assumptions
|
|
- secret-looking examples
|
|
- Context-X branding as default OSS UI
|
|
|
|
Keep them as:
|
|
|
|
```text
|
|
examples/profiles/context-x/
|
|
```
|
|
|
|
or as a private deployment overlay.
|
|
|
|
## Proposed New Packages
|
|
|
|
```text
|
|
packages/
|
|
cli/ # init, doctor, integrate, import, mode
|
|
discovery/ # detects models, clients, runtimes, providers
|
|
mcp-server/ # MCP tools/resources
|
|
trust-router/ # sensitivity + policy routing
|
|
consent-ledger/ # append-only permissions ledger
|
|
memory-sync/ # local/git/gitea memory backend
|
|
handoff/ # AI Handoff Protocol schema + helpers
|
|
context-receipts/ # receipts and audit artifacts
|
|
config-writer/ # safe config diffs and rollback
|
|
benchmark-lab/ # model/agent benchmark suite
|
|
agent-reputation/ # agent scorecards
|
|
run-ledger/ # reproducible AI runs
|
|
context-compression/ # compression + token budget manager
|
|
integrations/ # integration catalog manifests
|
|
connectors/ # data source connectors
|
|
import-wizard/ # migration/import helpers
|
|
```
|
|
|
|
## MVP Cut
|
|
|
|
The first useful OSS release should not try to ship everything.
|
|
|
|
MVP must include:
|
|
|
|
- CLI with `init`, `doctor`, `start`, `integrate`
|
|
- local AI discovery: Ollama + LM Studio + OpenAI-compatible `/v1/models`
|
|
- provider env discovery with consent
|
|
- MCP server with safe gateway and memory tools
|
|
- Trust Router with four trust levels
|
|
- Gitea/Git memory backend
|
|
- Context Receipts
|
|
- AI Handoff Protocol
|
|
- Safe Config Writer
|
|
- Offline Mode
|
|
- basic topology dashboard
|
|
|
|
MVP can defer:
|
|
|
|
- full benchmark lab
|
|
- team RBAC
|
|
- all data connectors
|
|
- full import wizard
|
|
- advanced compression comparisons
|
|
- agent reputation automation
|
|
|