LLM Gateway is a TypeScript control plane for routing AI requests across local models, hosted providers, and user-owned subscription tools through one API surface.

The project focuses on practical AI operations:

one OpenAI-compatible endpoint for chat and automation clients
provider routing with local-first defaults
request validation, output checks, and audit-friendly observability
token and context reduction helpers
dashboard and metrics endpoints
subscription bridge discovery for CLIs such as Claude Code, Codex, ChatGPT, Gemini, and Copilot

This repository is a public-ready starter snapshot. It does not include private training data, private deployment profiles, private hostnames, production credentials, or local operator runbooks.

Repository Status

Maintainer: Rene Fichtmueller
Primary language: TypeScript
Runtime: Node.js
Web framework: Fastify
Database support: PostgreSQL
Bridge style: local HTTP bridge processes, OpenAI-compatible where possible

See CONTRIBUTORS.md for the contributor list.

What The Gateway Does

LLM Gateway accepts a request from a client, classifies and routes it, optionally compresses or validates context, calls a selected provider or bridge, and returns a normalized response.

The intended flow is:

client
  -> gateway API
  -> policy and routing
  -> local model, provider API, or subscription bridge
  -> validation and metrics
  -> normalized response

The gateway can be used by:

local developer tools
internal dashboards
automation workflows
CLI assistants
model experiments
subscription bridge tests
apps that expect OpenAI-style chat completions

Features

Unified API

The gateway exposes familiar endpoints such as:

GET /health
GET /metrics
POST /v1/chat/completions
POST /v1/embeddings
GET /api/subscriptions
POST /api/subscriptions/scan
POST /api/subscriptions/join
POST /api/subscriptions/bridges
POST /api/subscriptions/test
POST /api/subscriptions/:subscription_id/v1/chat/completions

Subscription Bridge Discovery

The gateway can scan for locally available subscription tools and present them as routable providers. A detected and joined subscription can receive an automatically created local bridge.

Current catalog entries include:

Claude Code
GitHub Copilot
Microsoft 365 Copilot
ChatGPT
Gemini
Codex
Aider

Subscription OAuth sessions and credentials stay inside the local CLI or bridge process. The gateway only exposes a local bridge URL and a gateway access card.

Local-First Routing

The project is designed so local model providers can be preferred for sensitive or private workflows. Hosted providers and subscription bridges can be enabled deliberately when the operator chooses to use them.

Observability

The gateway includes metrics, structured logging, request tracking, fallback tracking, and cost-oriented helpers. The goal is to make model routing visible instead of guessing what happened after a request completes.

Quick Start

Requirements

Node.js 22 or newer
npm
optional: PostgreSQL for persistence-backed routes
optional: a local model server or subscription CLI for live model calls

Install

npm install

Run In Development

npm run dev

The gateway starts on http://localhost:3100 unless PORT is set.

Build

npm run build

Start The Built Server

npm run start

Smoke Test

curl http://localhost:3100/health

Chat Completion Test

curl -X POST http://localhost:3100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -H "X-Caller-ID: local-test" \
  -d '{
    "model": "codex-default",
    "messages": [
      { "role": "user", "content": "Say hello from the gateway." }
    ]
  }'

The selected model must be available through a configured provider or subscription bridge.

Subscription Bridge Quick Start

Scan for subscriptions:

curl -X POST http://localhost:3100/api/subscriptions/scan \
  -H "Authorization: Bearer <key>"

Search the subscription list:

curl "http://localhost:3100/api/subscriptions?search=codex" \
  -H "Authorization: Bearer <key>"

Join a subscription:

curl -X POST http://localhost:3100/api/subscriptions/join \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{ "subscription_id": "codex", "auto_spawn": true }'

Spawn bridges for joined or detected subscriptions:

curl -X POST http://localhost:3100/api/subscriptions/bridges \
  -H "Authorization: Bearer <key>"

Test a subscription bridge:

curl -X POST http://localhost:3100/api/subscriptions/test \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{
    "subscription_id": "codex",
    "message": "Return a one sentence bridge status."
  }'

Call a joined subscription through its own gateway API:

curl -X POST http://localhost:3100/api/subscriptions/codex/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{
    "model": "codex-default",
    "messages": [
      { "role": "user", "content": "Return one sentence through the Codex bridge." }
    ]
  }'

Joined subscriptions also work through the unified /v1/chat/completions endpoint when the request sends X-LLM-Gateway-Subscription with the subscription ID. The gateway compresses chat context before bridge dispatch and returns compression metadata in the response when available.

See docs/subscription-bridges.md for a deeper guide.

Configuration

Configuration is environment-driven. Common variables:

Variable	Purpose
`PORT`	Gateway HTTP port
`HOST`	Gateway bind host
`LOG_LEVEL`	Log verbosity
`DATABASE_URL`	PostgreSQL connection string
`CLAUDE_BRIDGE_URL`	Claude bridge URL
`CODEX_BRIDGE_URL`	Codex bridge URL
`CHATGPT_BRIDGE_URL`	ChatGPT bridge URL
`COPILOT_BRIDGE_URL`	Copilot bridge URL
`GEMINI_BRIDGE_URL`	Gemini bridge URL
`AUTO_SPAWN_BRIDGES`	Enables bridge startup on boot when set to `1`
`DISCOVERY_INTERVAL_MS`	Subscription discovery interval
`WATCHDOG_ENABLED`	Enables bridge watchdog mode when set to `1`

Do not commit real credentials, local OAuth artifacts, database passwords, cookies, or production deployment values.

See docs/configuration.md.

Project Layout

packages/gateway              Main Fastify gateway
packages/client               TypeScript client helper
packages/chatgpt-api-adapter  Local ChatGPT adapter
packages/claude-code-bridge   Claude Code bridge helper
packages/codex-lsp-adapter    Codex integration helper
packages/learning             Learning worker pieces
packages/learning-integration Feedback and metrics integration
packages/prompt-optimizer     Prompt analysis helpers
openai-bridge                 Minimal OpenAI-compatible bridge
copilot-bridge                Copilot bridge prototype

Development Workflow

Create a branch.
Keep changes small and reviewable.
Run the relevant build or tests.
Run a secret scan before pushing.
Open a pull request with a clear description and validation notes.

Recommended checks:

npm run build
npm test
gitleaks dir . --redact

Adding A Provider Or Subscription

Most subscription additions start in:

packages/gateway/src/modules/subscription-discovery.ts
packages/gateway/src/modules/bridge-spawner.ts
packages/gateway/src/routes/subscriptions.ts
packages/gateway/src/pipeline/external-providers.ts

The short version:

Add a descriptor to the subscription catalog.
Add a bridge implementation or external bridge URL.
Add one or more model IDs.
Run discovery.
Join the subscription.
Spawn and test the bridge.
Add or update tests.

See HOWTO.md for the full walkthrough.

Security

This project intentionally avoids publishing private operator data. Please keep it that way.

Before opening a pull request, check for:

credentials or OAuth artifacts
private hostnames or personal paths
production deployment URLs
private IP addresses
training data or logs
database dumps
screenshots that expose account state

See SECURITY.md for reporting and handling guidance.

Contributing

Contributions are welcome when they keep the gateway safer, easier to run, and easier to understand. Start with CONTRIBUTING.md.

License

No public license has been selected in this snapshot. Do not assume reuse rights until a license file is added by the maintainer.