-
Notifications
You must be signed in to change notification settings - Fork 0
Feat/dockerize #87
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Feat/dockerize #87
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| # Running CrowdStream with Docker | ||
|
|
||
| Two top-level Compose files bring up the app plus its core dependencies: | ||
|
|
||
| | File | Purpose | | ||
| | --- | --- | | ||
| | `docker-compose.local.yml` | Local development — backend & frontend hot reload | | ||
| | `docker-compose.prod.yml` | Production — compiled backend, static frontend built into nginx | | ||
|
|
||
| Each stack runs **backend + frontend + a 6-node Redis cluster + MongoDB + an nginx ingress**. | ||
| The NAT-traversal layers (Coturn, Envoy, HAProxy) are **not** included here — start those from | ||
| `infra/` as before when you need TURN/relay. | ||
|
|
||
| ``` | ||
| browser ──▶ nginx :80 ──/──────────▶ frontend (local: Vite :5173 · prod: static in nginx) | ||
| ──/backend/──▶ backend :3000 (/backend/ prefix stripped → /api/v1, /db) | ||
| ──/socket.io/▶ backend :3000 (WebSocket) | ||
| backend :3000 ──▶ mongo 127.0.0.1:27017 + redis cluster 127.0.0.1:6379-6384 | ||
| ``` | ||
|
|
||
| > **Networking:** every service uses `network_mode: host`, matching the existing `infra/` setup. | ||
| > This is required so the Redis cluster can advertise `127.0.0.1:<port>` and mediasoup can bind | ||
| > the WebRTC/recording UDP ports directly. **Host networking is a Linux feature** — on Docker | ||
| > Desktop for macOS/Windows the port mapping semantics differ and this stack is not supported as-is. | ||
|
|
||
| ## 1. Prerequisites | ||
|
|
||
| - Docker Engine + Compose v2 on **Linux**. | ||
| - The RTC port range (`RTC_MIN_PORT`–`RTC_MAX_PORT`, default `40000-40100` UDP/TCP) free on the host. | ||
|
|
||
| ## 2. Environment | ||
|
|
||
| The backend requires ~25 env vars (all mandatory — it exits on any missing one). Seed them from the | ||
| template, then edit the secrets: | ||
|
|
||
| ```bash | ||
| cp deploy/env.docker.template backend/.env | ||
| # edit backend/.env: set JWT_SECRET, TURN_SECRET, DATABASE_NAME, etc. | ||
| ``` | ||
|
|
||
| You don't need to get the infra endpoints right in `backend/.env` — the Compose files force-override | ||
| `MONGO_DB_URL`, `REDIS_HOST`/`REDIS_PORT*`, `PORT`, `INSTANCE_ID`, and the `*_IP` vars so the | ||
| containerized Mongo/Redis are always used. | ||
|
|
||
| For **local** frontend dev, Vite reads `frontend/.env` at runtime (it's bind-mounted) — put any | ||
| `VITE_*` values there. For **production**, `VITE_*` are baked in at build time; provide them via your | ||
| shell or a root `.env` that `docker compose` reads (see the `args:` in `docker-compose.prod.yml`). | ||
|
|
||
| ## 3. Local development | ||
|
|
||
| ```bash | ||
| docker compose -f docker-compose.local.yml up --build | ||
| ``` | ||
|
|
||
| - Waits for Mongo to be healthy and `redis-init` to form the cluster, then starts the backend. | ||
| - Open **http://localhost/** — the nginx ingress serves the Vite app and proxies the API/socket. | ||
| - Editing `backend/src/**` → `nodemon` restart; editing `frontend/src/**` → Vite HMR. | ||
|
|
||
| Health checks: | ||
|
|
||
| ```bash | ||
| curl http://localhost/backend/db/__ping # -> PING OK | ||
| curl http://localhost/backend/health # -> HEALTH OK | ||
| ``` | ||
|
|
||
| ## 4. Production | ||
|
|
||
| ```bash | ||
| cp deploy/env.docker.template backend/.env # set real secrets | ||
| export HOST_PUBLIC_IP=<your server public IP> # so WebRTC reaches remote clients | ||
| docker compose -f docker-compose.prod.yml up --build -d | ||
| ``` | ||
|
|
||
| - Backend runs the compiled `node dist/index.js`. | ||
| - The frontend container is nginx serving the built SPA **and** proxying `/backend` + `/socket.io` | ||
| to the backend — it is the :80 ingress (no separate nginx service in prod). | ||
| - Verify the runtime asset copied by the build (the Lua rate-limit script): | ||
|
|
||
| ```bash | ||
| docker compose -f docker-compose.prod.yml exec backend ls dist/scripts # -> rateLimit.lua | ||
| ``` | ||
|
|
||
| ## 5. Notes & caveats | ||
|
|
||
| - **Redis cluster** is created once by the `redis-init` one-shot service; it's idempotent (re-runs | ||
| detect `cluster_state:ok` and exit). Data persists in the `redis-N-data` named volumes. Node configs | ||
| are reused read-only from `infra/redis/redis-N/redis.conf`. The image is `redis:7-alpine` (Redis ≥7 is | ||
| required for the sharded pub/sub the app uses; bump the tag if you want 8.x). | ||
| - **MongoDB** binds to `127.0.0.1` only (host networking) with data in the `mongo-data` volume. It has | ||
| no auth by default — fine because it isn't reachable off-host, but enable auth for a hardened deploy. | ||
| - **Recordings** are written relative to the working dir (`recording<room><ts>.mp4` and | ||
| `src/recording/<room>.sdp`). In local dev they land on the host via the bind mount. In production they | ||
| live inside the container and are **ephemeral** — bind-mount a host path onto `/app` (or change the | ||
| output path in `backend/src/recording/`) if you need them to persist. | ||
| - **TURN/relay:** for connectivity across restrictive NATs, run Coturn/Envoy/HAProxy from `infra/` and | ||
| point the frontend `VITE_TURN_*` values at them. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| node_modules | ||
| dist | ||
| logs | ||
| *.log | ||
| .env | ||
| .env.* | ||
| .git | ||
| .gitignore | ||
| Dockerfile | ||
| .dockerignore | ||
| Makefile | ||
| README.md | ||
| *.pem | ||
| recording*.mp4 | ||
| src/recording/*.sdp |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -1,20 +1,50 @@ | ||||||
| FROM node:20-slim | ||||||
| # syntax=docker/dockerfile:1 | ||||||
| # | ||||||
| # Multi-stage backend image for CrowdStream (mediasoup SFU, Node/TS). | ||||||
| # - target `dev` : nodemon + ts-node hot reload (source bind-mounted by compose) | ||||||
| # - target `prod` : TypeScript compiled to dist/, lean runtime, no build tools | ||||||
| # | ||||||
| # Build deps (python3/build-essential/pkg-config) are required to compile the | ||||||
| # mediasoup native worker and ffmpeg-static during `npm install`. | ||||||
|
|
||||||
| RUN apt-get update && apt-get install -y \ | ||||||
| # ---- base: toolchain shared by dev and build ---- | ||||||
| FROM node:20-slim AS base | ||||||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||||||
| python3 \ | ||||||
| build-essential \ | ||||||
| pkg-config \ | ||||||
|
Check warning on line 15 in backend/Dockerfile
|
||||||
| && rm -rf /var/lib/apt/lists/* | ||||||
| WORKDIR /app | ||||||
|
|
||||||
| WORKDIR /src | ||||||
|
|
||||||
| # ---- dev: hot reload; real source comes from a bind mount in compose ---- | ||||||
| FROM base AS dev | ||||||
| ENV NODE_ENV=development | ||||||
| COPY package*.json ./ | ||||||
|
|
||||||
| RUN npm install | ||||||
| COPY . . | ||||||
| EXPOSE 3000 | ||||||
| CMD ["npm", "run", "dev"] | ||||||
|
|
||||||
| # ---- build: compile TS -> dist, then drop dev dependencies ---- | ||||||
| FROM base AS build | ||||||
| ENV NODE_ENV=production | ||||||
| COPY package*.json ./ | ||||||
| RUN npm ci | ||||||
|
Check warning on line 32 in backend/Dockerfile
|
||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P1: The production build fails at Prompt for AI agents
Suggested change
|
||||||
| COPY . . | ||||||
| # `npm run build` = tsc + `cp -r src/scripts dist/scripts` (rateLimit.lua is read | ||||||
| # at runtime via __dirname and is NOT emitted by tsc). Then prune dev deps while | ||||||
| # keeping the already-built mediasoup worker + ffmpeg-static binaries. | ||||||
| RUN npm run build \ | ||||||
| && npm prune --omit=dev | ||||||
|
|
||||||
| # ---- prod: minimal runtime image ---- | ||||||
| FROM node:20-slim AS prod | ||||||
|
Check warning on line 41 in backend/Dockerfile
|
||||||
| ENV NODE_ENV=production | ||||||
| WORKDIR /app | ||||||
| # recording/sdp.ts writes `src/recording/<room>.sdp` relative to the working dir. | ||||||
| RUN mkdir -p src/recording | ||||||
| COPY --from=build /app/node_modules ./node_modules | ||||||
| COPY --from=build /app/dist ./dist | ||||||
| COPY --from=build /app/package.json ./package.json | ||||||
| EXPOSE 3000 | ||||||
|
|
||||||
| # Start app | ||||||
| CMD ["npm", "run","dev"] | ||||||
| CMD ["node", "dist/index.js"] | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,7 +4,9 @@ | |
| "description": "> Backend system for scalable HTTP Live Streaming (HLS) using FFmpeg, built with Node.js and TypeScript.", | ||
| "main": "index.js", | ||
| "scripts": { | ||
| "dev": "nodemon --watch src --ext ts,d.ts --exec ts-node --files src/index.ts" | ||
| "dev": "nodemon --watch src --ext ts,d.ts --exec ts-node --files src/index.ts", | ||
| "build": "tsc -p tsconfig.json && cp -r src/scripts dist/scripts", | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: Running Prompt for AI agents |
||
| "start": "node dist/index.js" | ||
| }, | ||
| "author": "Harshit Singh Parihar", | ||
| "license": "ISC", | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -140,8 +140,8 @@ io.on("connection", (socket) => { | |||||||||||||||||||
| } | ||||||||||||||||||||
| }) | ||||||||||||||||||||
|
|
||||||||||||||||||||
| socket.on("disconnect", async (reason) => { | ||||||||||||||||||||
| logger.info(`User disconnected ${socket.id} beacuse of ${reason}`) | ||||||||||||||||||||
| socket.on("disconnect", async (reason, details) => { | ||||||||||||||||||||
| logger.error(`User disconnected, ${socket.id} reason: ${reason} details: ${details}`) | ||||||||||||||||||||
| handleDisconnect(socket) | ||||||||||||||||||||
| await stopFfmpegRecording(socket.id) | ||||||||||||||||||||
|
Comment on lines
+143
to
146
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: Disconnect is a normal Socket.IO lifecycle event (tab close, navigation, network blip), not an error. Logging every disconnect with logger.error fills logs/error.log (the winston File transport at level 'error') and drowns genuine failures. Keep the previous logger.info level if you want these logs, or use logger.warn at most. Prompt for AI agents
Suggested change
|
||||||||||||||||||||
| }); | ||||||||||||||||||||
|
|
||||||||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,66 @@ | ||||||||
| # ============================================================================= | ||||||||
| # CrowdStream — Docker environment reference | ||||||||
| # | ||||||||
| # Copy the BACKEND section into backend/.env (the compose files load it via | ||||||||
| # env_file). The compose files then FORCE-OVERRIDE the infra endpoints | ||||||||
| # (MONGO_DB_URL, REDIS_HOST/PORT*, PORT, INSTANCE_ID, *_IP, NODE_ENV) via their | ||||||||
| # own `environment:` blocks, so the containerized Mongo/Redis are always used | ||||||||
| # regardless of what you put here — you only really need to set the secrets and | ||||||||
| # tuning values below. | ||||||||
| # | ||||||||
| # cp deploy/env.docker.template backend/.env # then edit the secrets | ||||||||
| # | ||||||||
| # The FRONTEND section is used two ways: | ||||||||
| # - local : Vite reads frontend/.env at runtime (bind-mounted) — put them there. | ||||||||
| # - prod : passed as build args (Vite inlines them at build time). Provide | ||||||||
| # them in your shell or a root .env consumed by docker compose. | ||||||||
| # ============================================================================= | ||||||||
|
|
||||||||
| # ------------------------------- BACKEND ------------------------------------- | ||||||||
| NODE_ENV=development | ||||||||
| PORT=3000 | ||||||||
| INSTANCE_ID=node-1 | ||||||||
|
|
||||||||
| # --- Mongo & Redis (overridden by compose to point at the containers) --- | ||||||||
| MONGO_DB_URL=mongodb://127.0.0.1:27017 | ||||||||
| DATABASE_NAME=crowdstream | ||||||||
| REDIS_HOST=127.0.0.1 | ||||||||
| REDIS_PORT1=6379 | ||||||||
| REDIS_PORT2=6380 | ||||||||
| REDIS_PORT3=6381 | ||||||||
| # Used by infra/redis/init-cluster.sh (the compose init uses fixed ports): | ||||||||
| REDIS_PORT4=6382 | ||||||||
| REDIS_PORT5=6383 | ||||||||
| REDIS_PORT6=6384 | ||||||||
|
|
||||||||
| # --- mediasoup / WebRTC --- | ||||||||
| # With host networking these bind directly to the host. For real remote clients | ||||||||
| # set PUBLIC_IP / ANNOUCED_IP to the server's public IP and open the RTC range. | ||||||||
| PUBLIC_IP=127.0.0.1 | ||||||||
| ANNOUCED_IP=127.0.0.1 | ||||||||
| RECORDING_IP=127.0.0.1 | ||||||||
| RTC_MIN_PORT=40000 | ||||||||
| RTC_MAX_PORT=40100 | ||||||||
| MEDIASOUP_WORKER=2 | ||||||||
| MEDIASOUP_MAX_WORKERS=4 | ||||||||
| WORKER_THRESHOLD=500 | ||||||||
| VIDEO_PORT=5004 | ||||||||
| AUDIO_PORT=5006 | ||||||||
|
|
||||||||
| # --- HTTP / auth / TURN secrets (CHANGE THESE) --- | ||||||||
| CORS_ORIGINS=* | ||||||||
| JWT_SECRET=change-me-to-a-long-random-string | ||||||||
| ACCESS_TOKEN_EXPIRY=1d | ||||||||
| TURN_SECRET=change-me-turn-shared-secret | ||||||||
| TURN_TTL=86400 | ||||||||
|
|
||||||||
| # ------------------------------- FRONTEND ------------------------------------ | ||||||||
| # API/signaling are same-origin relative paths in the app, so these are mostly | ||||||||
| # for TURN. Leave TURN blank to rely on STUN only. | ||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: The comment claims leaving TURN blank falls back to STUN only, but no STUN server is configured anywhere and the frontend builds its iceServers purely from the VITE_TURN_* URL vars, which are empty when blank. With the template values, remote clients behind NAT have no STUN or TURN, so only host candidates are gathered and they will not connect. Either set up a STUN server or correct the comment to state that blank TURN means no NAT traversal. Prompt for AI agents
Suggested change
|
||||||||
| VITE_API_URL=/backend/api/v1 | ||||||||
| VITE_SIGNALING_URL=/ | ||||||||
| VITE_TURN_USERNAME= | ||||||||
| VITE_TURN_CREDENTIAL= | ||||||||
| VITE_TURN_UDP_URL= | ||||||||
| VITE_TURN_TCP_URL= | ||||||||
| VITE_TURNS_TCP_URL= | ||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # Local ingress: single same-origin entrypoint (:80) for the dev stack. | ||
| # Runs with host networking, so upstreams are on 127.0.0.1. | ||
| # / -> Vite dev server (:5173), WebSocket upgrade enables HMR | ||
| # /backend/ -> backend (:3000), prefix stripped by the trailing slash | ||
| # /socket.io/ -> backend (:3000) Socket.IO | ||
| server { | ||
| listen 80; | ||
| server_name _; | ||
|
|
||
| location / { | ||
| proxy_pass http://127.0.0.1:5173; | ||
| proxy_http_version 1.1; | ||
| proxy_set_header Upgrade $http_upgrade; | ||
| proxy_set_header Connection "upgrade"; | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: The Prompt for AI agents |
||
| proxy_set_header Host $host; | ||
| proxy_set_header X-Real-IP $remote_addr; | ||
| proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | ||
| proxy_set_header X-Forwarded-Proto $scheme; | ||
| } | ||
|
|
||
| location /backend/ { | ||
| proxy_pass http://127.0.0.1:3000/; | ||
| proxy_set_header Host $host; | ||
| proxy_set_header X-Real-IP $remote_addr; | ||
| proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | ||
| proxy_set_header X-Forwarded-Proto $scheme; | ||
| } | ||
|
|
||
| location /socket.io/ { | ||
| proxy_pass http://127.0.0.1:3000; | ||
| proxy_http_version 1.1; | ||
| proxy_set_header Upgrade $http_upgrade; | ||
| proxy_set_header Connection "upgrade"; | ||
| proxy_set_header Host $host; | ||
| proxy_set_header X-Real-IP $remote_addr; | ||
| proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | ||
| proxy_set_header X-Forwarded-Proto $scheme; | ||
| proxy_read_timeout 3600s; | ||
| proxy_send_timeout 3600s; | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| #!/bin/sh | ||
| # Initialize the 6-node Redis cluster used by the backend for sharded pub/sub. | ||
| # | ||
| # Runs as a one-shot, host-networked init container (see the compose files), so | ||
| # the nodes are reachable on 127.0.0.1. Idempotent: exits early if the cluster | ||
| # is already formed, otherwise creates it (3 masters + 3 replicas). | ||
| set -e | ||
|
|
||
| HOST="127.0.0.1" | ||
| PORTS="6379 6380 6381 6382 6383 6384" | ||
|
|
||
| echo "Waiting for Redis nodes..." | ||
| for PORT in $PORTS; do | ||
| echo " waiting for $HOST:$PORT ..." | ||
| until redis-cli -h "$HOST" -p "$PORT" ping >/dev/null 2>&1; do | ||
| sleep 1 | ||
| done | ||
| echo " $HOST:$PORT is up." | ||
| done | ||
|
|
||
| if redis-cli -h "$HOST" -p 6379 cluster info 2>/dev/null | grep -q "cluster_state:ok"; then | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: When a persisted topology has slot coverage but is missing nodes or replicas, this check exits successfully without creating or repairing the promised 3-master/3-replica cluster. Validate Prompt for AI agents |
||
| echo "Redis cluster already initialized." | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "Creating Redis cluster (3 masters + 3 replicas)..." | ||
| redis-cli --cluster create \ | ||
| "$HOST:6379" "$HOST:6380" "$HOST:6381" \ | ||
| "$HOST:6382" "$HOST:6383" "$HOST:6384" \ | ||
| --cluster-replicas 1 \ | ||
| --cluster-yes | ||
|
|
||
| echo "Redis cluster created successfully." | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P3: The documented output for
/backend/db/__pingis wrong. After nginx strips the/backend/prefix the request hitsdbReadinessCheck, which returns JSON{success:true, message:"Database Up"}, not the literalPING OKshown. Update the comment so a user following the doc isn't confused by the different output.Prompt for AI agents