llm-gateway/DEPLOYMENT-BRIDGES.md

Bridge Services Deployment Guide

Overview

This document covers deploying bridge services to Erik for third-party LLM provider integration:

• openai-bridge (Port 3251): OpenAI ChatGPT and Codex API wrapper
• copilot-bridge (Port 3252): GitHub Copilot API proxy

Architecture

• openai-bridge (Port 3251): HTTP wrapper around OpenAI CLI, provides OpenAI-compatible /v1/chat/completions endpoint
• copilot-bridge (Port 3252): HTTP proxy managing GitHub Copilot API (copilot-api), provides OpenAI-compatible /v1/chat/completions endpoint
• Integration: Both registered in external-providers.ts as fallback providers
• Authentication:
  • openai-bridge: Uses OPENAI_API_KEY (OpenAI subscription required)
  • copilot-bridge: Uses GitHub Copilot subscription (auth handled internally by copilot-api)

Prerequisites on Erik

• OpenAI CLI tool: openai command available (homebrew: brew install openai-cli)
• Node.js 20+ installed
• PM2 process manager available
• /opt directory writable (for service deployment)
• /var/log/llm-gateway directory exists for logs

Deployment Steps

1. Automatic Setup (Recommended)

The ensure-bridges.sh script handles automatic deployment:

# Run on Erik
cd /opt/llm-gateway
bash scripts/ensure-bridges.sh

This will:

• Create /opt/openai-bridge directory if missing
• Deploy server.js and package.json
• Create logs directory

2. Manual Setup (Alternative)

If automatic script fails, deploy manually:

# On Erik
mkdir -p /opt/openai-bridge /var/log/llm-gateway

# Copy openai-bridge files from repo
cp /opt/llm-gateway/openai-bridge/server.js /opt/openai-bridge/
cp /opt/llm-gateway/openai-bridge/package.json /opt/openai-bridge/

3. Configuration

Update ecosystem.config.cjs

Edit /opt/llm-gateway/deploy/ecosystem.config.cjs:

env: {
  OPENAI_API_KEY: 'sk-proj-xxxxx...', // Your OpenAI subscription key
  OPENAI_BRIDGE_PORT: 3251,
  OPENAI_MODEL: 'gpt-4-turbo',
}

Important: Do NOT commit API keys to git. Use environment variables or .env files.

4. Deploy with PM2

# On Erik
cd /opt/llm-gateway
pm2 start deploy/ecosystem.config.cjs --update-env

This will start:

• llm-gateway (main service, port 3103)
• openai-bridge (ChatGPT bridge, port 3251)
• llm-learning (auto-learning engine)

5. Verification

# Check service status
pm2 status

# Health check
curl http://localhost:3251/health

# Sample request
curl -X POST http://localhost:3251/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4-turbo",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

# View logs
pm2 logs openai-bridge

Environment Variables

Variable	Required	Default	Notes
OPENAI_API_KEY	Yes	(empty)	OpenAI subscription API key
OPENAI_BRIDGE_PORT	No	3251	Service port
OPENAI_MODEL	No	gpt-4-turbo	Default model
OPENAI_BRIDGE_URL	Yes*	http://localhost:3251	Set in ecosystem.config.cjs

*Required in gateway's environment for proper routing

Copilot Bridge Deployment

Prerequisites on Erik

• Node.js 20+ installed
• PM2 process manager available
• GitHub account with active GitHub Copilot subscription
• /opt directory writable (for service deployment)
• /var/log/llm-gateway directory exists for logs

Deployment Steps

1. Automatic Setup (Recommended)

The ensure-bridges.sh script handles automatic deployment:

# Run on Erik
cd /opt/llm-gateway
bash scripts/ensure-bridges.sh

This will:

• Create /opt/copilot-bridge directory if missing
• Deploy server.js and package.json
• Run npm install to install dependencies
• Create logs directory

2. Manual Setup (Alternative)

If automatic script fails, deploy manually:

# On Erik
mkdir -p /opt/copilot-bridge /var/log/llm-gateway

# Copy copilot-bridge files from repo
cp /opt/llm-gateway/copilot-bridge/server.js /opt/copilot-bridge/
cp /opt/llm-gateway/copilot-bridge/package.json /opt/copilot-bridge/

# Install dependencies
cd /opt/copilot-bridge
npm install

3. Authentication

GitHub Copilot authentication requires user interaction and is per-machine:

# On Erik (as the user who will run PM2)
cd /opt/copilot-bridge
npm run auth

This will:

1. Open a browser window (or provide a GitHub auth URL)
2. Prompt you to authorize GitHub Copilot API access
3. Save tokens locally in ~/.copilot-api/ directory

4. Configuration

Update deploy/ecosystem.config.cjs:

{
  name: 'copilot-bridge',
  env: {
    COPILOT_BRIDGE_PORT: 3252,
    COPILOT_API_INTERNAL_PORT: 4141,
  }
}

The gateway configuration should include:

COPILOT_BRIDGE_URL: 'http://localhost:3252'

5. Verification

# Check service status
pm2 status | grep copilot-bridge

# Health check
curl http://localhost:3252/health

# Sample request
curl -X POST http://localhost:3252/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Write a hello world function"}]
  }'

# View logs
pm2 logs copilot-bridge

Troubleshooting

Bridge Service Won't Start

# Check if openai CLI is installed
which openai
openai --version

# Install if missing (requires homebrew)
brew install openai-cli

Port 3251 Already in Use

# Find process using port
lsof -i :3251
# Kill process if needed
kill -9 <PID>

OPENAI_API_KEY Not Configured

# Check environment
pm2 show openai-bridge | grep OPENAI_API_KEY

# Update if missing
pm2 set openai-bridge OPENAI_API_KEY "sk-proj-xxxxx..."
pm2 restart openai-bridge

Health Check Fails

# Manual test of openai CLI
OPENAI_API_KEY="sk-proj-xxxxx..." openai api chat.completions.create \
  -m gpt-4-turbo \
  -g user "hello" \
  -t 0.3 \
  -M 100

Copilot Bridge Won't Start

Authentication Failed

# Clear cached tokens and re-authenticate
rm -rf ~/.copilot-api
cd /opt/copilot-bridge
npm run auth

Port 4141 Already in Use

# Find process using internal copilot-api port
lsof -i :4141
# Kill if needed
kill -9 <PID>

GitHub Copilot Subscription Issues

Verify you have an active GitHub Copilot subscription:

1. Check https://github.com/settings/copilot
2. Verify payment method is valid
3. Re-authenticate: npm run auth
4. Test directly: npm run debug

Port 3252 Already in Use

# Find process using copilot-bridge port
lsof -i :3252
# Kill if needed
kill -9 <PID>

Integration with LLM Gateway

Once deployed, openai-bridge is automatically:

1. Registered in provider fallback chain (after claude-bridge, cerebras, groq, etc.)
2. Used when:
   • Ollama is unavailable
   • Request specifically targets "openai" or "chatgpt" provider
   • Confidence threshold requires higher-tier reasoning
3. Monitored for:
   • Rate limits (90 requests/min)
   • Response quality and latency
   • Error rates

Rollback / Removal

# Stop openai-bridge service
pm2 stop openai-bridge
pm2 delete openai-bridge

# Remove service directory (keeps config for reinstall)
rm -rf /opt/openai-bridge

# Or keep for reinstall (without logs)
rm -rf /opt/openai-bridge/node_modules

Provider Fallback Chain

Providers are tried in order of tier and availability:

1. Ollama (local, fastest, free)
2. claude-bridge (Claude subscription)
3. openai-bridge (OpenAI subscription)
4. copilot-bridge (GitHub Copilot subscription) ← NEW
5. Free LLM APIs (Cerebras, Groq, Mistral, NVIDIA, Cloudflare)

When a higher-tier provider is unavailable or rate-limited, the gateway automatically routes to the next available provider.

Next Steps

1. Cost Tracking: Monitor API usage and costs across providers
2. Monitoring: Add Prometheus metrics for bridge health and latency
3. Performance Optimization: Cache responses and optimize provider selection
4. Additional Bridges: Anthropic Claude API, Azure OpenAI, Cohere, others

---

Last Updated: 2026-04-25
Status: Ready for deployment (OpenAI + Copilot)