llm-gateway-public/README.md

# LLM Gateway

LLM Gateway is a TypeScript control plane for routing AI requests across local models,
hosted providers, and user-owned subscription tools through one API surface.

The project focuses on practical AI operations:

- one OpenAI-compatible endpoint for chat and automation clients
- provider routing with local-first defaults
- request validation, output checks, and audit-friendly observability
- token and context reduction helpers
- dashboard and metrics endpoints
- subscription bridge discovery for CLIs such as Claude Code, Codex, ChatGPT, Gemini, and Copilot

This repository is a public-ready starter snapshot. It does not include private training
data, private deployment profiles, private hostnames, production credentials, or local
operator runbooks.

## Repository Status

- Maintainer: Rene Fichtmueller
- Primary language: TypeScript
- Runtime: Node.js
- Web framework: Fastify
- Database support: PostgreSQL
- Bridge style: local HTTP bridge processes, OpenAI-compatible where possible

See [CONTRIBUTORS.md](CONTRIBUTORS.md) for the contributor list.

## What The Gateway Does

LLM Gateway accepts a request from a client, classifies and routes it, optionally compresses
or validates context, calls a selected provider or bridge, and returns a normalized response.

The intended flow is:

```text
client
  -> gateway API
  -> policy and routing
  -> local model, provider API, or subscription bridge
  -> validation and metrics
  -> normalized response
```

The gateway can be used by:

- local developer tools
- internal dashboards
- automation workflows
- CLI assistants
- model experiments
- subscription bridge tests
- apps that expect OpenAI-style chat completions

## Features

### Unified API

The gateway exposes familiar endpoints such as:

- `GET /health`
- `GET /metrics`
- `POST /v1/chat/completions`
- `POST /v1/embeddings`
- `GET /api/subscriptions`
- `POST /api/subscriptions/scan`
- `POST /api/subscriptions/join`
- `POST /api/subscriptions/bridges`
- `POST /api/subscriptions/test`
- `POST /api/subscriptions/:subscription_id/v1/chat/completions`

### Subscription Bridge Discovery

The gateway can scan for locally available subscription tools and present them as routable
providers. A detected and joined subscription can receive an automatically created local bridge.

Current catalog entries include:

- Claude Code
- GitHub Copilot
- Microsoft 365 Copilot
- ChatGPT
- Gemini
- Codex
- Aider

Subscription OAuth sessions and credentials stay inside the local CLI or bridge process.
The gateway only exposes a local bridge URL and a gateway access card.

### Local-First Routing

The project is designed so local model providers can be preferred for sensitive or private
workflows. Hosted providers and subscription bridges can be enabled deliberately when the
operator chooses to use them.

### Observability

The gateway includes metrics, structured logging, request tracking, fallback tracking, and
cost-oriented helpers. The goal is to make model routing visible instead of guessing what
happened after a request completes.

## Quick Start

### Requirements

- Node.js 22 or newer
- npm
- optional: PostgreSQL for persistence-backed routes
- optional: a local model server or subscription CLI for live model calls

### Install

```bash
npm install
```

### Run In Development

```bash
npm run dev
```

The gateway starts on `http://localhost:3100` unless `PORT` is set.

### Build

```bash
npm run build
```

### Start The Built Server

```bash
npm run start
```

### Smoke Test

```bash
curl http://localhost:3100/health
```

### Chat Completion Test

```bash
curl -X POST http://localhost:3100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -H "X-Caller-ID: local-test" \
  -d '{
    "model": "codex-default",
    "messages": [
      { "role": "user", "content": "Say hello from the gateway." }
    ]
  }'
```

The selected model must be available through a configured provider or subscription bridge.

## Subscription Bridge Quick Start

Scan for subscriptions:

```bash
curl -X POST http://localhost:3100/api/subscriptions/scan \
  -H "Authorization: Bearer <key>"
```

Search the subscription list:

```bash
curl "http://localhost:3100/api/subscriptions?search=codex" \
  -H "Authorization: Bearer <key>"
```

Join a subscription:

```bash
curl -X POST http://localhost:3100/api/subscriptions/join \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{ "subscription_id": "codex", "auto_spawn": true }'
```

Spawn bridges for joined or detected subscriptions:

```bash
curl -X POST http://localhost:3100/api/subscriptions/bridges \
  -H "Authorization: Bearer <key>"
```

Test a subscription bridge:

```bash
curl -X POST http://localhost:3100/api/subscriptions/test \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{
    "subscription_id": "codex",
    "message": "Return a one sentence bridge status."
  }'
```

Call a joined subscription through its own gateway API:

```bash
curl -X POST http://localhost:3100/api/subscriptions/codex/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{
    "model": "codex-default",
    "messages": [
      { "role": "user", "content": "Return one sentence through the Codex bridge." }
    ]
  }'
```

Joined subscriptions also work through the unified `/v1/chat/completions` endpoint when the
request sends `X-LLM-Gateway-Subscription` with the subscription ID. The gateway compresses
chat context before bridge dispatch and returns compression metadata in the response when
available.

See [docs/subscription-bridges.md](docs/subscription-bridges.md) for a deeper guide.

## Configuration

Configuration is environment-driven. Common variables:

| Variable | Purpose |
|---|---|
| `PORT` | Gateway HTTP port |
| `HOST` | Gateway bind host |
| `LOG_LEVEL` | Log verbosity |
| `DATABASE_URL` | PostgreSQL connection string |
| `CLAUDE_BRIDGE_URL` | Claude bridge URL |
| `CODEX_BRIDGE_URL` | Codex bridge URL |
| `CHATGPT_BRIDGE_URL` | ChatGPT bridge URL |
| `COPILOT_BRIDGE_URL` | Copilot bridge URL |
| `GEMINI_BRIDGE_URL` | Gemini bridge URL |
| `AUTO_SPAWN_BRIDGES` | Enables bridge startup on boot when set to `1` |
| `DISCOVERY_INTERVAL_MS` | Subscription discovery interval |
| `WATCHDOG_ENABLED` | Enables bridge watchdog mode when set to `1` |

Do not commit real credentials, local OAuth artifacts, database passwords, cookies, or
production deployment values.

See [docs/configuration.md](docs/configuration.md).

## Project Layout

```text
packages/gateway              Main Fastify gateway
packages/client               TypeScript client helper
packages/chatgpt-api-adapter  Local ChatGPT adapter
packages/claude-code-bridge   Claude Code bridge helper
packages/codex-lsp-adapter    Codex integration helper
packages/learning             Learning worker pieces
packages/learning-integration Feedback and metrics integration
packages/prompt-optimizer     Prompt analysis helpers
openai-bridge                 Minimal OpenAI-compatible bridge
copilot-bridge                Copilot bridge prototype
```

## Development Workflow

1. Create a branch.
2. Keep changes small and reviewable.
3. Run the relevant build or tests.
4. Run a secret scan before pushing.
5. Open a pull request with a clear description and validation notes.

Recommended checks:

```bash
npm run build
npm test
gitleaks dir . --redact
```

## Adding A Provider Or Subscription

Most subscription additions start in:

- `packages/gateway/src/modules/subscription-discovery.ts`
- `packages/gateway/src/modules/bridge-spawner.ts`
- `packages/gateway/src/routes/subscriptions.ts`
- `packages/gateway/src/pipeline/external-providers.ts`

The short version:

1. Add a descriptor to the subscription catalog.
2. Add a bridge implementation or external bridge URL.
3. Add one or more model IDs.
4. Run discovery.
5. Join the subscription.
6. Spawn and test the bridge.
7. Add or update tests.

See [HOWTO.md](HOWTO.md) for the full walkthrough.

## Security

This project intentionally avoids publishing private operator data. Please keep it that way.

Before opening a pull request, check for:

- credentials or OAuth artifacts
- private hostnames or personal paths
- production deployment URLs
- private IP addresses
- training data or logs
- database dumps
- screenshots that expose account state

See [SECURITY.md](SECURITY.md) for reporting and handling guidance.

## Contributing

Contributions are welcome when they keep the gateway safer, easier to run, and easier to
understand. Start with [CONTRIBUTING.md](CONTRIBUTING.md).

## License

No public license has been selected in this snapshot. Do not assume reuse rights until a
license file is added by the maintainer.