llm-gateway/HOWTO.md

# How To Work With LLM Gateway

This guide covers the most common operator and contributor tasks.

## 1. Run The Gateway Locally

Install dependencies:

```bash
npm install
```

Start development mode:

```bash
npm run dev
```

Check health:

```bash
curl http://localhost:3100/health
```

Build:

```bash
npm run build
```

Start the built server:

```bash
npm run start
```

## 2. Configure A Local Provider

Use environment variables for local provider URLs and bridge URLs. Avoid committing any
machine-specific configuration.

Common examples:

```bash
export PORT=3100
export HOST=localhost
export LOG_LEVEL=info
```

For provider and bridge variables, prefer your shell profile, process manager, or deployment
environment. Keep real values out of git.

## 3. Scan For Subscriptions

Start the gateway, then run:

```bash
curl -X POST http://localhost:3100/api/subscriptions/scan \
  -H "Authorization: Bearer <key>"
```

The scan checks the known subscription catalog and reports:

- whether the CLI is installed
- whether authentication can be detected
- whether a bridge is already running
- which model IDs the gateway can expose
- whether the subscription can be joined or spawned

## 4. Join A Subscription

Joining means the gateway records that the operator wants this subscription available.
It does not publish the subscription credentials.

```bash
curl -X POST http://localhost:3100/api/subscriptions/join \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{ "subscription_id": "claude-code", "auto_spawn": true }'
```

After joining, fetch the access card:

```bash
curl "http://localhost:3100/api/subscriptions?search=claude" \
  -H "Authorization: Bearer <key>"
```

The response includes the gateway URL, chat endpoint, model IDs, and bridge status.
It also includes `subscriptionChatCompletionsUrl`, the per-subscription API endpoint that
clients can call directly after the bridge is available.

## 5. Spawn A Bridge

```bash
curl -X POST http://localhost:3100/api/subscriptions/bridges \
  -H "Authorization: Bearer <key>"
```

For a single subscription:

```bash
curl -X POST http://localhost:3100/api/subscriptions/bridges \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{ "subscription_id": "codex" }'
```

Bridges listen on localhost ports and expose:

- `GET /health`
- `POST /api/generate`
- `POST /v1/chat/completions`

## 6. Test A Bridge

```bash
curl -X POST http://localhost:3100/api/subscriptions/test \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{
    "subscription_id": "codex",
    "message": "Return a short bridge test response."
  }'
```

A healthy response should report bridge URL, latency, selected model, and a short output.

## 7. Call A Subscription As An API

After scan, join, and bridge spawn, each joined subscription can be called through its own
OpenAI-compatible gateway endpoint:

```bash
curl -X POST http://localhost:3100/api/subscriptions/claude-code/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [
      { "role": "user", "content": "Explain what the gateway does in one sentence." }
    ]
  }'
```

The same subscription can also be forced through the unified endpoint with:

```text
X-LLM-Gateway-Subscription: claude-code
```

The gateway compresses chat context before subscription bridge dispatch and includes
compression metadata in responses where available.

## 8. Call The Unified Chat Endpoint

```bash
curl -X POST http://localhost:3100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -H "X-LLM-Gateway-Subscription: claude-code" \
  -H "X-Caller-ID: local-test" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [
      { "role": "user", "content": "Explain what the gateway does in one sentence." }
    ]
  }'
```

If the selected model is not available, check:

- the subscription scan result
- bridge health
- environment variables
- gateway logs
- whether the CLI is authenticated locally

## 9. Add A New Subscription

1. Add a new entry to `SUBSCRIPTION_CATALOG` in
   `packages/gateway/src/modules/subscription-discovery.ts`.
2. Choose a stable `subscription_id`.
3. Define the CLI command and version probe.
4. Define the bridge port and env var.
5. Add model IDs and tiers.
6. Add or reuse a bridge implementation in
   `packages/gateway/src/modules/bridge-spawner.ts`.
7. Add provider routing in `packages/gateway/src/pipeline/external-providers.ts` if needed.
8. Run `npm run build`.
9. Test scan, join, spawn, and unified chat.

Descriptor shape:

```typescript
{
  id: 'example-provider',
  label: 'Example Provider',
  command: 'example-cli',
  versionArgs: ['--version'],
  bridgePort: 3260,
  bridgeEnvKey: 'EXAMPLE_BRIDGE_URL',
  providerName: 'example-bridge',
  bridgeImplementation: 'inline-openai',
  models: [
    { id: 'example-large', tier: 'large' }
  ]
}
```

## 10. Add A New Contributor

Update [CONTRIBUTORS.md](CONTRIBUTORS.md).

Use a short entry:

```text
- Name - role or contribution area
```

Avoid adding private email addresses, local usernames, personal machine paths, or account
details unless the contributor explicitly wants that information public.

## 11. Prepare A Pull Request

Before opening a pull request:

```bash
npm run build
npm test
gitleaks dir . --redact
```

In the pull request, include:

- what changed
- why it changed
- how it was tested
- any risks or follow-ups

## 12. Common Troubleshooting

### The Gateway Starts But Chat Fails

Check that a provider or bridge is configured for the requested model.

### A Subscription Is Detected But Not Ready

The CLI may be installed but not authenticated. Log in through that CLI first, then rescan.

### The Bridge Starts But Returns An Error

Run the same CLI command outside the gateway to confirm the subscription can answer locally.

### The API Says Unauthorized

Set the gateway access value in your runtime environment and send it as:

```text
Authorization: Bearer <key>
```

### The Build Fails

Run `npm install`, check Node.js version, and verify that no private-only package paths were
introduced.