llm-gateway/DEPLOYMENT-BRIDGES.md

# Bridge Services Deployment Guide

## Overview
This document covers deploying bridge services to Erik for third-party LLM provider integration:
- **openai-bridge** (Port 3251): OpenAI ChatGPT and Codex API wrapper
- **copilot-bridge** (Port 3252): GitHub Copilot API proxy

## Architecture
- **openai-bridge** (Port 3251): HTTP wrapper around OpenAI CLI, provides OpenAI-compatible /v1/chat/completions endpoint
- **copilot-bridge** (Port 3252): HTTP proxy managing GitHub Copilot API (copilot-api), provides OpenAI-compatible /v1/chat/completions endpoint
- **Integration**: Both registered in `external-providers.ts` as fallback providers
- **Authentication**: 
  - openai-bridge: Uses OPENAI_API_KEY (OpenAI subscription required)
  - copilot-bridge: Uses GitHub Copilot subscription (auth handled internally by copilot-api)

## Prerequisites on Erik
- OpenAI CLI tool: `openai` command available (homebrew: `brew install openai-cli`)
- Node.js 20+ installed
- PM2 process manager available
- /opt directory writable (for service deployment)
- /var/log/llm-gateway directory exists for logs

## Deployment Steps

### 1. Automatic Setup (Recommended)
The `ensure-bridges.sh` script handles automatic deployment:

```bash
# Run on Erik
cd /opt/llm-gateway
bash scripts/ensure-bridges.sh
```

This will:
- Create `/opt/openai-bridge` directory if missing
- Deploy `server.js` and `package.json`
- Create logs directory

### 2. Manual Setup (Alternative)
If automatic script fails, deploy manually:

```bash
# On Erik
mkdir -p /opt/openai-bridge /var/log/llm-gateway

# Copy openai-bridge files from repo
cp /opt/llm-gateway/openai-bridge/server.js /opt/openai-bridge/
cp /opt/llm-gateway/openai-bridge/package.json /opt/openai-bridge/
```

### 3. Configuration

#### Update ecosystem.config.cjs
Edit `/opt/llm-gateway/deploy/ecosystem.config.cjs`:

```javascript
env: {
  OPENAI_API_KEY: 'sk-proj-xxxxx...', // Your OpenAI subscription key
  OPENAI_BRIDGE_PORT: 3251,
  OPENAI_MODEL: 'gpt-4-turbo',
}
```

**Important**: Do NOT commit API keys to git. Use environment variables or .env files.

### 4. Deploy with PM2

```bash
# On Erik
cd /opt/llm-gateway
pm2 start deploy/ecosystem.config.cjs --update-env
```

This will start:
- `llm-gateway` (main service, port 3103)
- `openai-bridge` (ChatGPT bridge, port 3251)
- `llm-learning` (auto-learning engine)

### 5. Verification

```bash
# Check service status
pm2 status

# Health check
curl http://localhost:3251/health

# Sample request
curl -X POST http://localhost:3251/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4-turbo",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

# View logs
pm2 logs openai-bridge
```

## Environment Variables

| Variable | Required | Default | Notes |
|----------|----------|---------|-------|
| OPENAI_API_KEY | Yes | (empty) | OpenAI subscription API key |
| OPENAI_BRIDGE_PORT | No | 3251 | Service port |
| OPENAI_MODEL | No | gpt-4-turbo | Default model |
| OPENAI_BRIDGE_URL | Yes* | http://localhost:3251 | Set in ecosystem.config.cjs |

*Required in gateway's environment for proper routing

## Copilot Bridge Deployment

### Prerequisites on Erik
- Node.js 20+ installed
- PM2 process manager available
- GitHub account with active GitHub Copilot subscription
- /opt directory writable (for service deployment)
- /var/log/llm-gateway directory exists for logs

### Deployment Steps

#### 1. Automatic Setup (Recommended)
The `ensure-bridges.sh` script handles automatic deployment:

```bash
# Run on Erik
cd /opt/llm-gateway
bash scripts/ensure-bridges.sh
```

This will:
- Create `/opt/copilot-bridge` directory if missing
- Deploy `server.js` and `package.json`
- Run `npm install` to install dependencies
- Create logs directory

#### 2. Manual Setup (Alternative)
If automatic script fails, deploy manually:

```bash
# On Erik
mkdir -p /opt/copilot-bridge /var/log/llm-gateway

# Copy copilot-bridge files from repo
cp /opt/llm-gateway/copilot-bridge/server.js /opt/copilot-bridge/
cp /opt/llm-gateway/copilot-bridge/package.json /opt/copilot-bridge/

# Install dependencies
cd /opt/copilot-bridge
npm install
```

#### 3. Authentication

GitHub Copilot authentication requires user interaction and is **per-machine**:

```bash
# On Erik (as the user who will run PM2)
cd /opt/copilot-bridge
npm run auth
```

This will:
1. Open a browser window (or provide a GitHub auth URL)
2. Prompt you to authorize GitHub Copilot API access
3. Save tokens locally in `~/.copilot-api/` directory

#### 4. Configuration

Update `deploy/ecosystem.config.cjs`:

```javascript
{
  name: 'copilot-bridge',
  env: {
    COPILOT_BRIDGE_PORT: 3252,
    COPILOT_API_INTERNAL_PORT: 4141,
  }
}
```

The gateway configuration should include:
```javascript
COPILOT_BRIDGE_URL: 'http://localhost:3252'
```

#### 5. Verification

```bash
# Check service status
pm2 status | grep copilot-bridge

# Health check
curl http://localhost:3252/health

# Sample request
curl -X POST http://localhost:3252/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Write a hello world function"}]
  }'

# View logs
pm2 logs copilot-bridge
```

## Troubleshooting

### Bridge Service Won't Start
```bash
# Check if openai CLI is installed
which openai
openai --version

# Install if missing (requires homebrew)
brew install openai-cli
```

### Port 3251 Already in Use
```bash
# Find process using port
lsof -i :3251
# Kill process if needed
kill -9 <PID>
```

### OPENAI_API_KEY Not Configured
```bash
# Check environment
pm2 show openai-bridge | grep OPENAI_API_KEY

# Update if missing
pm2 set openai-bridge OPENAI_API_KEY "sk-proj-xxxxx..."
pm2 restart openai-bridge
```

### Health Check Fails
```bash
# Manual test of openai CLI
OPENAI_API_KEY="sk-proj-xxxxx..." openai api chat.completions.create \
  -m gpt-4-turbo \
  -g user "hello" \
  -t 0.3 \
  -M 100
```

### Copilot Bridge Won't Start

#### Authentication Failed
```bash
# Clear cached tokens and re-authenticate
rm -rf ~/.copilot-api
cd /opt/copilot-bridge
npm run auth
```

#### Port 4141 Already in Use
```bash
# Find process using internal copilot-api port
lsof -i :4141
# Kill if needed
kill -9 <PID>
```

#### GitHub Copilot Subscription Issues
Verify you have an active GitHub Copilot subscription:
1. Check https://github.com/settings/copilot
2. Verify payment method is valid
3. Re-authenticate: `npm run auth`
4. Test directly: `npm run debug`

### Port 3252 Already in Use
```bash
# Find process using copilot-bridge port
lsof -i :3252
# Kill if needed
kill -9 <PID>
```

## Integration with LLM Gateway

Once deployed, openai-bridge is automatically:
1. Registered in provider fallback chain (after claude-bridge, cerebras, groq, etc.)
2. Used when:
   - Ollama is unavailable
   - Request specifically targets "openai" or "chatgpt" provider
   - Confidence threshold requires higher-tier reasoning
3. Monitored for:
   - Rate limits (90 requests/min)
   - Response quality and latency
   - Error rates

## Rollback / Removal

```bash
# Stop openai-bridge service
pm2 stop openai-bridge
pm2 delete openai-bridge

# Remove service directory (keeps config for reinstall)
rm -rf /opt/openai-bridge

# Or keep for reinstall (without logs)
rm -rf /opt/openai-bridge/node_modules
```

## Provider Fallback Chain

Providers are tried in order of tier and availability:

1. **Ollama** (local, fastest, free)
2. **claude-bridge** (Claude subscription)
3. **openai-bridge** (OpenAI subscription)
4. **copilot-bridge** (GitHub Copilot subscription) ← NEW
5. **Free LLM APIs** (Cerebras, Groq, Mistral, NVIDIA, Cloudflare)

When a higher-tier provider is unavailable or rate-limited, the gateway automatically routes to the next available provider.

## Next Steps

1. **Cost Tracking**: Monitor API usage and costs across providers
2. **Monitoring**: Add Prometheus metrics for bridge health and latency
3. **Performance Optimization**: Cache responses and optimize provider selection
4. **Additional Bridges**: Anthropic Claude API, Azure OpenAI, Cohere, others

---

**Last Updated**: 2026-04-25  
**Status**: Ready for deployment (OpenAI + Copilot)