rene/llm-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
llm-gateway/OPEN_SOURCE_IMPLEMENTATION_ROADMAP.md

4.5 KiB
Raw Blame History

Open Source Implementation Roadmap

Phase 0: Sanitize And Productize

Goal: make the current codebase safe to publish and understandable outside Context-X.

Tasks:

  • Add OSS name and package naming decision.
  • Move Context-X-only files into examples/profiles/context-x/.
  • Add .env.example without private domains or secrets.
  • Replace hardcoded defaults with generated config.
  • Add license, contributing guide, security policy, and public README.
  • Run secret scan and dependency/license audit.
  • Decide which training data can be published.

Exit criteria:

  • Fresh clone can install without private services.
  • No private domains or internal IPs are required for default startup.
  • Public README explains local-only setup.

Phase 1: Adaptive Init

Goal: detect the user's AI environment and create config.

Packages:

  • packages/cli
  • packages/discovery
  • packages/config-writer

Commands:

adaptive-llm-gateway init
adaptive-llm-gateway doctor
adaptive-llm-gateway integrate <target>
adaptive-llm-gateway mode offline
adaptive-llm-gateway simulate <request-file>

Detection targets:

  • Ollama
  • LM Studio
  • LocalAI
  • llama.cpp server
  • vLLM
  • Open WebUI
  • OpenAI-compatible endpoints
  • OpenAI/Anthropic/Groq/Mistral/OpenRouter env keys
  • Claude Code
  • Codex
  • Cursor
  • VS Code
  • Continue.dev
  • n8n
  • Docker containers
  • Git/Gitea availability

Exit criteria:

  • init writes ~/.adaptive-llm-gateway/config.yaml.
  • No external integration is enabled without approval.
  • doctor reports actionable health and setup status.

Phase 2: Trust, Consent, Receipts

Goal: every request goes through policy and produces an audit artifact.

Packages:

  • packages/trust-router
  • packages/policy-engine
  • packages/consent-ledger
  • packages/context-receipts
  • packages/run-ledger
  • packages/provider-router

Features:

  • four trust levels: public, internal, confidential, secret
  • local-only/offline routing mode
  • simulation mode with no execution
  • provider router route constraints and fallbacks
  • append-only consent ledger
  • receipt for context used, blocked, redacted, routed
  • reproducible run folder

Exit criteria:

  • External providers are blocked for confidential/secret data by default.
  • Receipts can be viewed from CLI and dashboard.
  • Consent changes are append-only and reversible.

Phase 3: Shared Memory And MCP

Goal: make the gateway the shared memory and tool layer for all AI clients.

Packages:

  • packages/memory-sync
  • packages/handoff
  • packages/mcp-server
  • packages/route-reflector-memory

Features:

  • local memory repo
  • Git/Gitea sync
  • typed memory folders
  • MCP tools for memory and gateway calls
  • AI Handoff Protocol
  • Route Reflector Memory for routing outcomes
  • conflict-safe append-first writes

MCP tools:

  • gateway.complete
  • gateway.chat
  • gateway.health
  • gateway.route_preview
  • memory.search
  • memory.read
  • memory.write
  • memory.append_session
  • memory.record_decision
  • memory.record_task
  • memory.pull
  • memory.push

Exit criteria:

  • Claude Code and Codex can access the same memory through MCP.
  • Handoffs are stored in Git/Gitea.
  • Memory sync refuses to commit secrets.

Phase 4: Compression And Knowledge

Goal: reduce token use and retrieve only the right context.

Packages:

  • packages/context-compression
  • packages/connectors
  • packages/cache

Features:

  • token budget manager
  • session compaction
  • repo/doc summarization
  • memory dedupe
  • semantic cache
  • SQLite vector default
  • Postgres/Qdrant optional
  • approved data source connectors

Exit criteria:

  • Context packages include budget, source refs, and compression stats.
  • Receipts show compressed-from and final token counts.
  • Indexing requires explicit allowed roots.

Phase 5: Benchmarking And Reputation

Goal: route based on evidence instead of static assumptions.

Packages:

  • packages/benchmark-lab
  • packages/agent-reputation

Features:

  • model capability tests
  • agent scorecards
  • latency/cost/quality tracking
  • JSON reliability test
  • code patch/test benchmark
  • local vs hosted comparison

Exit criteria:

  • Trust Router can use benchmark scores.
  • Dashboard shows model and agent strengths.
  • Routing decisions explain benchmark influence.

Phase 6: Product UI

Goal: turn the operational dashboard into a usable OSS app.

UI areas:

  • Topology
  • Models
  • Agents
  • Memory
  • Policies
  • Receipts
  • Benchmarks
  • Costs
  • Integrations
  • Doctor
  • Settings

Exit criteria:

  • First screen is topology/status.
  • User can enable integrations from UI with diff preview.
  • User can inspect receipts and memory sync status.