llm-gateway/HOWTO.md

How To Work With LLM Gateway

This guide covers the most common operator and contributor tasks.

1. Run The Gateway Locally

Install dependencies:

npm install

Start development mode:

npm run dev

Check health:

curl http://localhost:3100/health

Build:

npm run build

Start the built server:

npm run start

2. Configure A Local Provider

Use environment variables for local provider URLs and bridge URLs. Avoid committing any machine-specific configuration.

Common examples:

export PORT=3100
export HOST=localhost
export LOG_LEVEL=info

For provider and bridge variables, prefer your shell profile, process manager, or deployment environment. Keep real values out of git.

3. Scan For Subscriptions

Start the gateway, then run:

curl -X POST http://localhost:3100/api/subscriptions/scan \
  -H "Authorization: Bearer <key>"

The scan checks the known subscription catalog and reports:

• whether the CLI is installed
• whether authentication can be detected
• whether a bridge is already running
• which model IDs the gateway can expose
• whether the subscription can be joined or spawned

4. Join A Subscription

Joining means the gateway records that the operator wants this subscription available. It does not publish the subscription credentials.

curl -X POST http://localhost:3100/api/subscriptions/join \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{ "subscription_id": "claude-code", "auto_spawn": true }'

After joining, fetch the access card:

curl "http://localhost:3100/api/subscriptions?search=claude" \
  -H "Authorization: Bearer <key>"

The response includes the gateway URL, chat endpoint, model IDs, and bridge status. It also includes subscriptionChatCompletionsUrl, the per-subscription API endpoint that clients can call directly after the bridge is available.

5. Spawn A Bridge

curl -X POST http://localhost:3100/api/subscriptions/bridges \
  -H "Authorization: Bearer <key>"

For a single subscription:

curl -X POST http://localhost:3100/api/subscriptions/bridges \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{ "subscription_id": "codex" }'

Bridges listen on localhost ports and expose:

• GET /health
• POST /api/generate
• POST /v1/chat/completions

6. Test A Bridge

curl -X POST http://localhost:3100/api/subscriptions/test \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{
    "subscription_id": "codex",
    "message": "Return a short bridge test response."
  }'

A healthy response should report bridge URL, latency, selected model, and a short output.

7. Call A Subscription As An API

After scan, join, and bridge spawn, each joined subscription can be called through its own OpenAI-compatible gateway endpoint:

curl -X POST http://localhost:3100/api/subscriptions/claude-code/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [
      { "role": "user", "content": "Explain what the gateway does in one sentence." }
    ]
  }'

The same subscription can also be forced through the unified endpoint with:

X-LLM-Gateway-Subscription: claude-code

The gateway compresses chat context before subscription bridge dispatch and includes compression metadata in responses where available.

8. Call The Unified Chat Endpoint

curl -X POST http://localhost:3100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <key>" \
  -H "X-LLM-Gateway-Subscription: claude-code" \
  -H "X-Caller-ID: local-test" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [
      { "role": "user", "content": "Explain what the gateway does in one sentence." }
    ]
  }'

If the selected model is not available, check:

• the subscription scan result
• bridge health
• environment variables
• gateway logs
• whether the CLI is authenticated locally

9. Add A New Subscription

1. Add a new entry to SUBSCRIPTION_CATALOG in packages/gateway/src/modules/subscription-discovery.ts.
2. Choose a stable subscription_id.
3. Define the CLI command and version probe.
4. Define the bridge port and env var.
5. Add model IDs and tiers.
6. Add or reuse a bridge implementation in packages/gateway/src/modules/bridge-spawner.ts.
7. Add provider routing in packages/gateway/src/pipeline/external-providers.ts if needed.
8. Run npm run build.
9. Test scan, join, spawn, and unified chat.

Descriptor shape:

{
  id: 'example-provider',
  label: 'Example Provider',
  command: 'example-cli',
  versionArgs: ['--version'],
  bridgePort: 3260,
  bridgeEnvKey: 'EXAMPLE_BRIDGE_URL',
  providerName: 'example-bridge',
  bridgeImplementation: 'inline-openai',
  models: [
    { id: 'example-large', tier: 'large' }
  ]
}

10. Add A New Contributor

Update CONTRIBUTORS.md.

Use a short entry:

- Name - role or contribution area

Avoid adding private email addresses, local usernames, personal machine paths, or account details unless the contributor explicitly wants that information public.

11. Prepare A Pull Request

Before opening a pull request:

npm run build
npm test
gitleaks dir . --redact

In the pull request, include:

• what changed
• why it changed
• how it was tested
• any risks or follow-ups

12. Common Troubleshooting

The Gateway Starts But Chat Fails

Check that a provider or bridge is configured for the requested model.

A Subscription Is Detected But Not Ready

The CLI may be installed but not authenticated. Log in through that CLI first, then rescan.

The Bridge Starts But Returns An Error

Run the same CLI command outside the gateway to confirm the subscription can answer locally.

The API Says Unauthorized

Set the gateway access value in your runtime environment and send it as:

Authorization: Bearer <key>

The Build Fails

Run npm install, check Node.js version, and verify that no private-only package paths were introduced.