Skip to content

EC2-DEPLOY.md

Latest commit

 

History

History
262 lines (180 loc) · 9.55 KB

EC2-DEPLOY.md

File metadata and controls

262 lines (180 loc) · 9.55 KB
Raw

Deploy CodeStream API + Worker to Amazon EC2

The frontend is hosted on Vercel. EC2 runs api-service, worker-service, and nginx (HTTPS) at api.codestream.npsolver.io. The host Docker engine runs isolated Python sandboxes for the worker.

Architecture

Vercel (frontend)  ──HTTPS──►  api.codestream.npsolver.io
                                        │
                                        ▼
                                  nginx :443 (TLS)
                                        │
                                        ▼
                                 api-service :8082
                                        │
                                        ▼
                                  SQS (submissions)
                                        │
                                        ▼
                               worker-service (systemd)
                                        │
                                        ▼
                        docker run codestream-python-runner
                                        │
                                        ▼
                                  SQS (results) ──► api-service ──► PostgreSQL

Vercel env: set API_SERVICE_URL=https://api.codestream.npsolver.io. The browser calls /api/* on Vercel; Next.js rewrites proxy to the API (see env.vercel.example). Redeploy after changing env vars.

Why worker runs on the host: it shells out to the docker CLI. Running natively with SupplementaryGroups=docker is simpler than mounting /var/run/docker.sock into a container.

Port 8082 stays on localhost only — 80/443 are public via nginx.

Prerequisites (AWS)

Resource Notes
SQS queues code-submissions and execution-results (see root README)
PostgreSQL Managed DB — set DATABASE_URL on EC2
IAM role EC2 instance profile with SQS send/receive/delete on both queues
DNS api.codestream.npsolver.io A record → EC2 Elastic IP
Vercel API_SERVICE_URL=https://api.codestream.npsolver.io (see env.vercel.example)
Gemini API key Optional

Step 1 — Launch EC2

  1. AMI: Amazon Linux 2023 or Ubuntu 22.04 LTS.
  2. Instance type: t3.medium or larger.
  3. Storage: 20 GB+ gp3.
  4. Security group inbound:
    • SSH 22 — your IP only
    • HTTP 80 — 0.0.0.0/0 (certbot + redirect)
    • HTTPS 443 — 0.0.0.0/0
    • Do not open 8082 publicly.
  5. IAM instance profile: attach the SQS role.
  6. Elastic IP: assign and point api.codestream.npsolver.io at it.

Step 2 — Clone and bootstrap

sudo git clone https://github.com/YOUR_ORG/CodeStream.git /opt/codestream-src
cd /opt/codestream-src
sudo chmod +x deploy/scripts/bootstrap-ec2.sh
sudo REPO_DIR=/opt/codestream-src ./deploy/scripts/bootstrap-ec2.sh

Installs Java 17, Maven, Docker, nginx; builds api + worker JARs; builds the codestream-python-runner image; registers systemd units.

Step 3 — Configure environment

sudo nano /etc/codestream/env

Minimum required:

AWS_REGION=us-east-1
SQS_SUBMISSION_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/ACCOUNT/code-submissions
SQS_RESULT_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/ACCOUNT/execution-results

DATABASE_URL=postgresql://USER:PASSWORD@HOST/DBNAME?sslmode=require

# CORS — only required if the browser calls the API directly (NEXT_PUBLIC_API_BASE_URL).
# Not needed when Vercel rewrites /api/* via API_SERVICE_URL.
# CORS_ALLOWED_ORIGIN_PATTERNS=https://codestream.npsolver.io,https://*.vercel.app

Adjust the CORS origins to match your actual Vercel production domain. If using an EC2 IAM role, omit AWS access keys. Do not set AWS_ENDPOINT_URL in production.

sudo systemctl restart codestream-api codestream-worker

Step 4 — Start services

sudo systemctl enable --now codestream-api
sudo systemctl enable --now codestream-worker

Verify:

sudo systemctl status codestream-api codestream-worker
curl -s http://127.0.0.1:8082/result/test   # 404 or JSON, not connection refused
sudo -u codestream docker run --rm codestream-python-runner:latest python3 -c "print('ok')"

Logs: journalctl -u codestream-api -f / journalctl -u codestream-worker -f

Step 5 — nginx + HTTPS

Install certbot

Amazon Linux 2023: sudo dnf install -y certbot python3-certbot-nginx

Ubuntu: sudo apt-get install -y certbot python3-certbot-nginx

Configure nginx

Install the HTTP-only config first (SSL cert paths are added by certbot):

sudo sed 's/YOUR_DOMAIN/api.codestream.npsolver.io/g' \
  /opt/codestream-src/deploy/nginx/codestream.conf \
  | sudo tee /etc/nginx/conf.d/codestream.conf

sudo nginx -t
sudo systemctl enable --now nginx
sudo certbot --nginx -d api.codestream.npsolver.io

certbot obtains the certificate and updates the nginx config for HTTPS + HTTP redirect. Do not add ssl_certificate paths manually before running certbot — nginx will fail to start if those files do not exist yet.

Confirm HTTPS

curl -s -o /dev/null -w '%{http_code}\n' https://api.codestream.npsolver.io/result/test

Open your Vercel frontend, submit Python code, and confirm end-to-end execution.

Step 6 — Vercel frontend

In the Vercel project → Settings → Environment Variables:

Variable Value
API_SERVICE_URL https://api.codestream.npsolver.io

Redeploy after setting or changing this variable — Next.js reads it at build time for the /api/* rewrite rule.

The browser fetches /api/execute, /api/result/{id}, etc. on your Vercel domain. Vercel proxies those requests to the EC2 API, so you do not need CORS on api-service for normal use.

Step 7 — Deploy updates

cd /opt/codestream-src
git pull
sudo REPO_DIR=/opt/codestream-src ./deploy/scripts/bootstrap-ec2.sh --redeploy

Or restart manually after a full bootstrap:

sudo systemctl restart codestream-api codestream-worker

Automatic deploy: pushes to master/main run .github/workflows/ci-deploy.yml (tests, then SSH redeploy from GitHub-hosted runners). See GitHub Actions setup below.

Frontend updates deploy via Vercel (git push or Vercel dashboard).

GitHub Actions setup

CI/CD runs on every push/PR to master or main. Merging to the default branch deploys the backend to EC2 via SSH from GitHub-hosted runners (after tests pass).

Public repos: do not use self-hosted runners — forks can execute arbitrary code on them. This project uses SSH deploy instead.

1. GitHub secrets

In the repo: Settings → Secrets and variables → Actions → New repository secret

Secret Value
EC2_HOST EC2 public IP or hostname (e.g. Elastic IP)
EC2_USER SSH user (ec2-user on Amazon Linux, ubuntu on Ubuntu)
EC2_SSH_KEY Private key for SSH (PEM contents of the .pem file)

Optional: create a production environment under Settings → Environments to require manual approval before deploy.

2. EC2: allow GitHub Actions SSH

GitHub-hosted runners connect over SSH (port 22). Your security group must allow inbound SSH from GitHub Actions runners.

Practical option: allow SSH from 0.0.0.0/0 and rely on key-only auth (no password login). Restrict EC2_SSH_KEY to GitHub secrets and disable password auth in sshd_config.

Alternatively, skip automatic deploy and redeploy manually over SSH from your IP (see Deploy updates).

Add the deploy key's public half to ~/.ssh/authorized_keys for EC2_USER.

3. EC2: git pull access

The deploy job runs git pull on the server. Public repos work without extra setup.

4. First-time bootstrap

Run the full bootstrap once (without --redeploy) before relying on CI:

sudo REPO_DIR=/opt/codestream-src ./deploy/scripts/bootstrap-ec2.sh
sudo systemctl enable --now codestream-api codestream-worker

Subsequent pushes use --redeploy (build JARs, restart services — no package reinstall).

File reference

Path Purpose
deploy/env.production.example EC2 environment template
deploy/env.vercel.example Vercel environment template
deploy/nginx/codestream.conf nginx → api-service:8082
deploy/systemd/codestream-api.service API systemd unit
deploy/systemd/codestream-worker.service Worker systemd unit
deploy/scripts/bootstrap-ec2.sh EC2 setup script

Troubleshooting

Symptom Check
CORS error in browser Only if using NEXT_PUBLIC_API_BASE_URL; with API_SERVICE_URL rewrites, CORS should not apply
Worker can't run code journalctl -u codestream-worker; codestream user in docker group; image exists
API can't reach DB DATABASE_URL; DB firewall allows EC2 IP
SQS errors IAM role; queue URLs and region
502 from nginx systemctl status codestream-api; port 8082 listening on localhost
Frontend hits wrong host API_SERVICE_URL on Vercel; redeploy after changing (build-time rewrite)
Deploy SSH timeout Security group must allow port 22 from GitHub Actions (see GitHub Actions setup)