diff --git a/DEPLOYMENT-BRIDGES.md b/DEPLOYMENT-BRIDGES.md new file mode 100644 index 0000000..abd3ca6 --- /dev/null +++ b/DEPLOYMENT-BRIDGES.md @@ -0,0 +1,183 @@ +# OpenAI Bridge Deployment Guide + +## Overview +This document covers deploying the OpenAI Bridge service to Erik for ChatGPT/Codex integration. + +## Architecture +- **openai-bridge** (Port 3251): HTTP wrapper around OpenAI CLI, provides OpenAI-compatible /v1/chat/completions endpoint +- **Integration**: Registered in `external-providers.ts` as fallback provider after Ollama and Claude +- **Authentication**: Uses OPENAI_API_KEY environment variable (OpenAI subscription required) + +## Prerequisites on Erik +- OpenAI CLI tool: `openai` command available (homebrew: `brew install openai-cli`) +- Node.js 20+ installed +- PM2 process manager available +- /opt directory writable (for service deployment) +- /var/log/llm-gateway directory exists for logs + +## Deployment Steps + +### 1. Automatic Setup (Recommended) +The `ensure-bridges.sh` script handles automatic deployment: + +```bash +# Run on Erik +cd /opt/llm-gateway +bash scripts/ensure-bridges.sh +``` + +This will: +- Create `/opt/openai-bridge` directory if missing +- Deploy `server.js` and `package.json` +- Create logs directory + +### 2. Manual Setup (Alternative) +If automatic script fails, deploy manually: + +```bash +# On Erik +mkdir -p /opt/openai-bridge /var/log/llm-gateway + +# Copy openai-bridge files from repo +cp /opt/llm-gateway/openai-bridge/server.js /opt/openai-bridge/ +cp /opt/llm-gateway/openai-bridge/package.json /opt/openai-bridge/ +``` + +### 3. Configuration + +#### Update ecosystem.config.cjs +Edit `/opt/llm-gateway/deploy/ecosystem.config.cjs`: + +```javascript +env: { + OPENAI_API_KEY: 'sk-proj-xxxxx...', // Your OpenAI subscription key + OPENAI_BRIDGE_PORT: 3251, + OPENAI_MODEL: 'gpt-4-turbo', +} +``` + +**Important**: Do NOT commit API keys to git. Use environment variables or .env files. + +### 4. Deploy with PM2 + +```bash +# On Erik +cd /opt/llm-gateway +pm2 start deploy/ecosystem.config.cjs --update-env +``` + +This will start: +- `llm-gateway` (main service, port 3103) +- `openai-bridge` (ChatGPT bridge, port 3251) +- `llm-learning` (auto-learning engine) + +### 5. Verification + +```bash +# Check service status +pm2 status + +# Health check +curl http://localhost:3251/health + +# Sample request +curl -X POST http://localhost:3251/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "gpt-4-turbo", + "messages": [{"role": "user", "content": "Hello"}] + }' + +# View logs +pm2 logs openai-bridge +``` + +## Environment Variables + +| Variable | Required | Default | Notes | +|----------|----------|---------|-------| +| OPENAI_API_KEY | Yes | (empty) | OpenAI subscription API key | +| OPENAI_BRIDGE_PORT | No | 3251 | Service port | +| OPENAI_MODEL | No | gpt-4-turbo | Default model | +| OPENAI_BRIDGE_URL | Yes* | http://localhost:3251 | Set in ecosystem.config.cjs | + +*Required in gateway's environment for proper routing + +## Troubleshooting + +### Bridge Service Won't Start +```bash +# Check if openai CLI is installed +which openai +openai --version + +# Install if missing (requires homebrew) +brew install openai-cli +``` + +### Port 3251 Already in Use +```bash +# Find process using port +lsof -i :3251 +# Kill process if needed +kill -9 +``` + +### OPENAI_API_KEY Not Configured +```bash +# Check environment +pm2 show openai-bridge | grep OPENAI_API_KEY + +# Update if missing +pm2 set openai-bridge OPENAI_API_KEY "sk-proj-xxxxx..." +pm2 restart openai-bridge +``` + +### Health Check Fails +```bash +# Manual test of openai CLI +OPENAI_API_KEY="sk-proj-xxxxx..." openai api chat.completions.create \ + -m gpt-4-turbo \ + -g user "hello" \ + -t 0.3 \ + -M 100 +``` + +## Integration with LLM Gateway + +Once deployed, openai-bridge is automatically: +1. Registered in provider fallback chain (after claude-bridge, cerebras, groq, etc.) +2. Used when: + - Ollama is unavailable + - Request specifically targets "openai" or "chatgpt" provider + - Confidence threshold requires higher-tier reasoning +3. Monitored for: + - Rate limits (90 requests/min) + - Response quality and latency + - Error rates + +## Rollback / Removal + +```bash +# Stop openai-bridge service +pm2 stop openai-bridge +pm2 delete openai-bridge + +# Remove service directory (keeps config for reinstall) +rm -rf /opt/openai-bridge + +# Or keep for reinstall (without logs) +rm -rf /opt/openai-bridge/node_modules +``` + +## Next Steps + +1. **Microsoft Copilot Integration**: Similar bridge pattern once CLI available +2. **Rate Limiting**: Implement adaptive rate limiting per provider +3. **Cost Tracking**: Monitor OpenAI API usage and costs +4. **Monitoring**: Add Prometheus metrics for bridge health + +--- + +**Last Updated**: 2026-04-25 +**Status**: Ready for deployment diff --git a/scripts/ensure-bridges.sh b/scripts/ensure-bridges.sh new file mode 100755 index 0000000..18d2c8d --- /dev/null +++ b/scripts/ensure-bridges.sh @@ -0,0 +1,182 @@ +#!/bin/bash +# Ensure bridge services are deployed and ready +# This script runs as part of PM2 initialization to deploy bridges if not present + +set -e + +OPENAI_BRIDGE_DIR="/opt/openai-bridge" +CLAUDE_BRIDGE_DIR="/opt/claude-bridge" + +echo "[$(date +'%Y-%m-%d %H:%M:%S')] Checking bridge services..." + +# Ensure openai-bridge exists +if [ ! -d "$OPENAI_BRIDGE_DIR" ]; then + echo "[$(date +'%Y-%m-%d %H:%M:%S')] Deploying openai-bridge..." + mkdir -p "$OPENAI_BRIDGE_DIR" + + # Deploy server.js + cat > "$OPENAI_BRIDGE_DIR/server.js" << 'BRIDGE_EOF' +import { execFile } from 'child_process' +import { createServer } from 'http' +import { promisify } from 'util' +import { readFileSync } from 'fs' + +const exec = promisify(execFile) +const PORT = process.env.OPENAI_BRIDGE_PORT || 3251 +const API_KEY = process.env.OPENAI_API_KEY +const DEFAULT_MODEL = process.env.OPENAI_MODEL || 'gpt-4-turbo' + +const SYSTEM_CODEX = `You are an expert code generation AI. +Generate clean, well-documented, production-ready code. +Output only the code — no explanations, no markdown blocks, no preamble.` + +const SYSTEM_CHATGPT = `You are a helpful AI assistant. +Provide clear, concise, accurate responses. +Output only the response — no preamble.` + +async function callOpenAI(messages, model, temperature = 0.3, maxTokens = 2048) { + if (!API_KEY) { + throw new Error('OPENAI_API_KEY not configured') + } + + const args = [ + 'api', 'chat.completions.create', + '-m', model, + '-t', String(temperature), + '-M', String(maxTokens), + ] + + for (const msg of messages) { + args.push('-g', msg.role, msg.content) + } + + return new Promise((resolve) => { + const env = { + ...process.env, + OPENAI_API_KEY: API_KEY + } + + exec('openai', args, { env, timeout: 300_000, maxBuffer: 1024 * 1024 * 10 }, (err, stdout) => { + if (err) { + resolve({ + success: false, + content: null, + error: err.message.slice(0, 500), + stderr: err.stderr?.slice(0, 500) + }) + } else { + try { + const result = JSON.parse(stdout) + const content = result.choices?.[0]?.message?.content || result.message?.content || stdout + resolve({ success: true, content, error: null }) + } catch (e) { + resolve({ success: true, content: stdout.trim(), error: null }) + } + } + }) + }) +} + +const server = createServer(async (req, res) => { + res.setHeader('Content-Type', 'application/json') + res.setHeader('Access-Control-Allow-Origin', '*') + res.setHeader('Access-Control-Allow-Methods', 'POST, GET, OPTIONS') + res.setHeader('Access-Control-Allow-Headers', 'Content-Type') + + if (req.method === 'OPTIONS') { res.writeHead(200); res.end(); return } + + if (req.method === 'GET' && req.url === '/health') { + res.writeHead(200) + res.end(JSON.stringify({ + status: 'ok', + version: '1.0.0', + provider: 'openai', + model: DEFAULT_MODEL, + configured: !!API_KEY + })) + return + } + + if (req.method === 'POST' && req.url === '/v1/chat/completions') { + let body = '' + req.on('data', chunk => body += chunk) + req.on('end', async () => { + try { + const { model, messages, temperature, max_tokens, type } = JSON.parse(body) + + if (!messages || !Array.isArray(messages)) { + res.writeHead(400) + res.end(JSON.stringify({ error: 'messages array required' })) + return + } + + const selectedModel = model || DEFAULT_MODEL + const temp = temperature ?? 0.3 + const maxTok = max_tokens ?? 2048 + + console.log(`[${new Date().toISOString()}] OpenAI ${selectedModel} (${type || 'chat'})`) + + const result = await callOpenAI(messages, selectedModel, temp, maxTok) + + if (result.success) { + res.writeHead(200) + res.end(JSON.stringify({ + success: true, + content: result.content, + provider: 'openai', + model: selectedModel + })) + } else { + res.writeHead(500) + res.end(JSON.stringify({ + success: false, + error: result.error, + stderr: result.stderr + })) + } + } catch (e) { + console.error('Error:', e.message) + res.writeHead(500) + res.end(JSON.stringify({ error: e.message })) + } + }) + return + } + + res.writeHead(404) + res.end(JSON.stringify({ error: 'not found' })) +}) + +server.listen(PORT, () => { + console.log(`openai-bridge running on port ${PORT}`) + console.log(` POST http://localhost:${PORT}/v1/chat/completions`) + console.log(` GET http://localhost:${PORT}/health`) + console.log(` Model: ${DEFAULT_MODEL}`) + console.log(` API Key configured: ${!!API_KEY}`) +}) +BRIDGE_EOF + + # Deploy package.json + cat > "$OPENAI_BRIDGE_DIR/package.json" << 'PACKAGE_EOF' +{ + "name": "openai-bridge", + "version": "1.0.0", + "description": "OpenAI API bridge for ChatGPT and Codex integration", + "type": "module", + "main": "server.js", + "scripts": { + "start": "node server.js", + "pm2": "pm2 start server.js --name openai-bridge" + } +} +PACKAGE_EOF + + echo "[$(date +'%Y-%m-%d %H:%M:%S')] ✓ openai-bridge deployed" +fi + +# Ensure claude-bridge exists (similar check) +if [ ! -d "$CLAUDE_BRIDGE_DIR" ]; then + echo "[$(date +'%Y-%m-%d %H:%M:%S')] Note: claude-bridge directory not found. Run deployment script separately if needed." +fi + +echo "[$(date +'%Y-%m-%d %H:%M:%S')] Bridge initialization complete"