feat: modernize monorepo, vitest test suite, nextjs 15 dashboard rewrite, lavalink v4, and cloud docs - #829
Open
PhantomNimbi wants to merge 82 commits into
Open
feat: modernize monorepo, vitest test suite, nextjs 15 dashboard rewrite, lavalink v4, and cloud docs#829PhantomNimbi wants to merge 82 commits into
PhantomNimbi wants to merge 82 commits into
Conversation
…nd update now playing embed
…YouTube device flow auth to console
…s for interactive terminal view
… auth output and file log exclusion
… audit, and API key gating
…thout disk mutation
…d strip trailing JSON delimiters
…Redis keys in Queue.leave() unconditionally
…nce bot & dashboard - Upgrade Next.js to 15.2.0 and migrate App Router to async request APIs (await params, useParams) - Upgrade Auth.js/NextAuth to v5 beta with server action handlers and safe Discord avatar URL resolution - Upgrade @next/eslint-plugin-next to 15.2.0 and align environment parsers to @t3-oss/env-* 0.13.11 - Replace pure-ESM env wrapper in @master-bot/bot with native Zod schema parsing for 100% CJS compatibility - Wire dynamic feature flags (LAVA_ENABLED, GIFS_ENABLED, TWITCH_ENABLED, NEWS_ENABLED, IGDB_ENABLED) across bot preconditions - Connect automated cross-platform PostgreSQL and Redis service checks (connect-or-auto-launch) - Implement dynamic command help registry and standardized help tables across all 60 slash commands - Enhance web dashboard with active-tab sidebar navigation, server overview statistics, and Redis log streaming - Resolve next-themes hydration mismatch by adding suppressHydrationWarning to root layout
…sabled commands - Group slash commands into structured categories (GIFs & Anime, Twitch, News, Games & Entertainment, General & Utilities) - Filter out categories and individual commands disabled globally via environment feature flags (LAVA_ENABLED, GIFS_ENABLED, TWITCH_ENABLED, NEWS_ENABLED, IGDB_ENABLED) - Display server-specific enable/disable toggles and active status badges for all active commands
…ys monorepo-wide - Configure remoteCipher in application.yml with default endpoint (https://cipher.kikkia.dev/) and support custom YOUTUBE_CIPHER_URL / YOUTUBE_CIPHER_PASSWORD - Pass deterministic Java system properties (-D) for YouTube OAuth, skipInitialization, cipher, and Spotify credentials in launcher scripts - Wire YOUTUBE_CIPHER_URL and YOUTUBE_CIPHER_PASSWORD into @master-bot/bot, @master-bot/api, @master-bot/dashboard env schemas and .env.example - Display active cipher endpoint in dev and production console status banners
…ash command, and fix dual-domain auth
… remote cipher, and active music engine
…nds with fuzzy matching and scoring
…t, and add granular audit log dashboard
…on and ticket features
…, and README button
…eholders in app.json
…ckend, split session handlers, and rewrite docs
Migrate the entire stack away from PostgreSQL onto SQLite and restore the
dashboard API surface that was lost when packages/api was removed, while
retaining Lavalink + Redis for music/queue state.
\### Infrastructure \& dependencies
\- docker-compose.yml: drop the postgres service; keep Lavalink + Redis;
  add sqlite-data volume at /Master-Bot/packages/db/prisma; host logs now
  map to /Master-Bot/logs
\- docker.env: replace POSTGRES\_\* with DATABASE\_URL="file:./db.sqlite"
\- Dockerfile: remove stale POSTGRES\_HOST comment
\- scripts/{common,dev,start}.mjs: strip postgres service ensure/ports/status;
  show SQLite-backed status
\- pnpm-workspace.yaml: drop removed packages/api and packages/session
\- turbo.json: remove SHADOW\_DB\_URL (keep REDIS\_\* env)
\- apps/dashboard/next.config.mjs: transpilePackages -> @master-bot/auth, @master-bot/db
\- remove vitest.config.ts, tsconfig.test.json, and the tests/ tree
\### Database
\- packages/db/prisma/schema.prisma: Guild notifyList, disabledCommands,
  logEvents stored as JSON-encoded String columns; reminders are guild-scoped
  (guildId required)
\- packages/db/prisma/db.sqlite is the schema-relative SQLite database file
\### Bot: session layer
\- delete the dead @master-bot/session package and apps/bot/src/trpc.ts
\- split SessionManager.ts (1184 lines) into lib/session/:
  types.ts, SessionStore.ts (state + persistence + hydration), handlers/ with
  one factory per namespace (users, guildData, welcomeMessages, tickets,
  twitchConfig, hubChannels, playlists, songs, reminders, commands, members)
\- SessionManager is now a thin facade; public API unchanged
\- align all consumers (music playlists, reminders, twitch notify, tickets,
  temp channels, preconditions, listeners) with the split handlers and the
  JSON-encoded guild fields; add guildMemberRemove listener
\### Bot: gifs
\- lib/gifs/searchGif.ts: replace dead/mismatched fallback GIFs with 3 SFW,
  verified-working, query-matched GIFs per category (36 URLs, HTTP-verified)
\### Dashboard
\- add server-side tRPC backend at apps/dashboard/src/server: trpc.ts,
  context.ts (NextAuth session + prisma), root.ts, routers/ (guild, channel,
  welcome, tickets, command, music, broadcast, system), utils/axiosWithRefresh.ts
\- rewrite app/api/trpc/\[trpc]/route.ts and utils/api.ts (typed AppRouter);
  DISCORD\_CLIENT\_ID/SECRET placeholders added to env.mjs
\- guild list now shows every server the bot is in as card UI with Manage
  buttons; guild.getAll returns all bot guilds (no Discord OAuth ownership
  fetch); \[server\_id] layout no longer redirects non-owners
\- fix pre-existing schema mismatches: command disable/log-event consumers now
  JSON parse/serialize Scalar String columns; reminders require an owned guild
\- add axios dependency
\### Docs
\- rewrite root README, CONTRIBUTING, apps/bot + apps/dashboard READMEs
\- consolidate wiki/ into 14 updated pages (Architecture, Commands,
  Configuration, Dashboard, Deployment, FAQ, Getting-Started, Moderation,
  Music, Reminders-and-Twitch, Tickets, Welcome-and-Temp-Channels, \_Sidebar,
  Home); remove legacy cloud/Heroku/lavalink/API-key pages
…eep session/store access
Author
✅ Comprehensive Fork vs. Upstream Differences
📊 High-Level Comparison Matrix
🚀 Detailed Breakdown of Changes1. Consolidated Single-Process Architecture
2. Zero-Ops Database Migration (PostgreSQL ➔ SQLite)
3. Zero-Process In-Memory Cache (Redis Server ➔
|
…line envs & add one-click cloud deployments - 🚀 Consolidate Discord bot gateway and Next.js 15 web dashboard into a single Node.js runtime sharing PORT 3000 - ⚡ Switch to in-process ioredis-mock, eliminating external Redis dependencies and separate server processes - 🗄️ Streamline SQLite persistence via Prisma with zero-ops schema initialization - ☁️ Add one-click cloud deployment blueprints and manifests for Render, Railway, Heroku, and Fly.io - 🎵 Integrate one-click deployment buttons for HELIX Origin's external Lavalink v4 audio server - 🧹 Audit and clean up unused environment variables and correct INTERNAL_URL across workspace - 🛠️ Fix ESM/TypeScript compilation settings across Turborepo packages and emit declarations for @master-bot/db - 📖 Expand comprehensive multi-page wiki covering architecture, deployment, configuration, and music setup
Author
📅 Update: Today's Changes & EnhancementsHere is a summary of the updates and fixes implemented today: 1. 🧹 Environment Variable Audit & Cleanup
2. 🛠️ TypeScript, Tooling & ESM Compilation
3. ☁️ Cloud Deployment Defaults
4. 🎵 Dedicated Standalone Lavalink v4 Server Repository (
|
- Document the free public Lavalink server hosted by HELIX Origin in Music.md, Deployment.md, and Configuration.md - Provide ready-to-use .env configuration snippet for instant audio setup - Include direct links to the live Lavalink server dashboard (https://lavalink-server-4n9o.onrender.com/) - Update FAQ.md, Getting-Started.md, and _Sidebar.md with public server references
…Fly) - Switch Render blueprint (render.yaml) to runtime: docker targeting ./Dockerfile - Switch Railway configuration (railway.json) to builder: DOCKERFILE - Update Dockerfile with full native dependencies (libfontconfig1, openssl), build step, and auto-sync start command - Move turbo to root dependencies and add .npmrc to ensure build tools are always available - Update apps/bot build script to bundle Lua audio queue scripts into dist/ - Update deployment documentation with Dockerfile details
- Delete pnpm-lock.yaml, package-lock.json, and yarn.lock in Dockerfile before pnpm install - Use pnpm install --no-frozen-lockfile during container builds - Add lockfiles to .dockerignore so they are excluded from Docker context - Configure frozen-lockfile=false in .npmrc to prevent CI/cloud environments from locking rebuilds - Remove pnpm-lock.yaml from repository tracking
…ling
## Problem
Heroku builds were failing with `pnpm: not found` (exit 127). The
`heroku/nodejs` buildpack picks the package manager from the **lockfile**
present in the repo; since no `pnpm-lock.yaml` was committed it fell back
to npm, which ran the `postinstall` script
(`pnpm db:generate && pnpm db:push`) without pnpm on PATH.
## Changes
- 🔒 **Add `pnpm-lock.yaml`** — lets `heroku/nodejs` detect pnpm and use it
for install/build natively (no extra buildpack needed).
- 🔢 **Pin `engines.node` to `24.x`** in `package.json` — removes the "wide
range" warning and resolves to the Active LTS version deterministically.
- 🐳 **Dockerfile: reproducible installs** — drop the `rm -f pnpm-lock.yaml`
workaround and install with `pnpm install --frozen-lockfile`.
- 🚫 **`.dockerignore`:**
- stop baking `.env` & secrets into the image (previously copied in),
- ignore nested `node_modules`, `.turbo`, sqlite dbs, and build output,
- keep `pnpm-lock.yaml` in the build context so the frozen install works.
- 🧩 **`docker-compose.yml`:**
- load envs from `.env` **and** `docker.env` (docker overrides win) so
all runtime variables flow in without relying on a baked-in `.env`,
- run `pnpm start` (bot, which hosts the dashboard) instead of
`pnpm run -r start` (started a second Next.js server on port 3000).
## Required Heroku config vars (not committed here)
`PNPM_SKIP_PRUNING=true` (prisma CLI + dotenv are devDeps used at runtime),
plus `DATABASE_URL`, `DISCORD_TOKEN`, and all other app secrets as config
vars (see `.env.example`). Set once: `NODE_MODULES_CACHE=false` to evict old
npm-cached `node_modules`.
Replace the one-click deploy (app.json, deploy buttons) and the discontinued Render/Railway/Fly configs with a single supported cloud deployment: one Heroku eco dyno running bot + dashboard + Lavalink. - Add heroku-prebuild (scripts/heroku-setup-lavalink.sh): downloads the latest Lavalink v4 jar from lavalink-devs/Lavalink at build time and copies application.yml.example -> application.yml - Procfile: start embedded Lavalink on LAVA_PORT inside the dyno, then bot - wiki/Deployment.md full rewrite: heroku CLI steps (nodejs + jvm buildpacks), eco dyno $5/mo flat rate and 1,000h shared pool pricing, env push via heroku-config config:push, PNPM_SKIP_PRUNING, ephemeral filesystem warning, new Backups section, Docker/VPS self-hosting - README: drop one-click/discontinued platform rows, add pricing, refresh tree + system requirements (Java on Heroku via jvm buildpack) - Remove app.json and render.yaml (one-click manifest + Render blueprint) - Music/FAQ pages point to the embedded Heroku audio engine instead of the one-click external Lavalink server button
Replace the one-click deploy (app.json, deploy buttons) and the discontinued Render/Railway/Fly configs with a single supported cloud deployment: one Heroku eco dyno running bot + dashboard + Lavalink. - Add heroku-prebuild (scripts/heroku-setup-lavalink.sh): downloads the latest Lavalink v4 jar from lavalink-devs/Lavalink at build time and copies application.yml.example -> application.yml - Procfile: start embedded Lavalink on LAVA_PORT inside the dyno, then bot - wiki/Deployment.md full rewrite: Heroku CLI steps (nodejs + jvm buildpacks), eco dyno $5/mo flat rate and 1,000h shared pool pricing, env push via heroku-config config:push, PNPM_SKIP_PRUNING, ephemeral filesystem warning, new Backups section, Docker/VPS self-hosting - README: drop one-click/discontinued platform rows, add pricing, refresh tree + system requirements (Java on Heroku via jvm buildpack) - Remove app.json and render.yaml (one-click manifest + Render blueprint) - Music/FAQ pages point to the embedded Heroku audio engine instead of the one-click external Lavalink server button
application.yml.example is a customized config, not Lavalink stock: it fixes broken YouTube playback (youtube-plugin multi-client rotation + remoteCipher + optional OAuth) and Spotify resolution (lavasrc ISRC providers), with tuned streaming buffers. - Add missing wiki/Lavalink.md (fixes dangling reference in application.yml.example header): what the config changes vs stock, plugin pins, env vars, PORT/LAVA_PORT interplay, plugin upgrades - Prepare application.yml from our template in heroku-setup-lavalink.sh and warn against Lavalink default config in Deployment/Music/ Getting-Started docs
pnpm --filter runs the bot with cwd=apps/bot, so the Next.js dashboard was never found (/.next missing) and the internal web service failed to start on Heroku. Add candidates relative to the package dir and the compiled dist path so /app/apps/dashboard resolves correctly.
…; recommend custom domain
…self-hosted VPS & Docker - 🗑️ Removed Procfile cloud deployment file - 🌐 Added curated list of recommended low-cost compatible VPS services (Hetzner, OVH, DigitalOcean, Linode, Vultr) with hardware sizing tips - 🔗 Removed retired onrender.com public Lavalink endpoints across documentation and wiki guides - 📚 Updated README, Getting-Started, Music, Configuration, Deployment, and FAQ guides to standardize on self-hosted Docker and VPS infrastructure - ⚡ Preserved internal keep-alive service for custom environments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
✅ Comprehensive Fork vs. Upstream Differences
📊 High-Level Comparison Matrix
galnir/Master-Bot)PhantomNimbi/Master-Bot)3000) & Dashboard (3001) run separatelyPORT(3000)postgresql://...)packages/db/prisma/db.sqliteauto-createdredis://...)ioredis-mock: Zero external binaries or processesHELIX-Origin/Lavalink-Serverrender.yaml), Railway, Heroku, Fly.io/healthpings every 10 mins)🚀 Detailed Breakdown of Changes
1. Consolidated Single-Process Architecture
/dashboard) into a single Node.js process listening onPORT(default3000).apps/bot/src/lib/server/webServer.ts:/) and handles dashboard SSR routing./healthendpoint for container healthchecks./healthevery 10 minutes (keepAlive.ts) to prevent free-tier cloud containers (e.g. Render) from spinning down.pnpm devandpnpm startcommands.2. Zero-Ops Database Migration (PostgreSQL ➔ SQLite)
file:./db.sqlite), initialized and migrated automatically onpnpm install.StringJSON arrays with helper serialization forGuild.notifyList,Guild.disabledCommands, andGuild.logEvents.guildIdonReminderrecords for proper multi-tenant guild scoping.3. Zero-Process In-Memory Cache (Redis Server ➔
ioredis-mock)redis-serverbinary dependencies and background services.ioredis-mockdirectly into@master-bot/db, keeping state caching, rate limiting, and session sharing entirely in-memory within the main Node.js process.4. Modern Audio Engine (Lavalink v4 & Modular Plugins)
dev.lavalink.youtube:youtube-plugin):YOUTUBE_CIPHER_URL).TV,MUSIC,ANDROID_VR,IOS,WEB)./youtube-authdevice flow storing OAuth2 refresh tokens to prevent YouTube IP bans.com.github.topi314.lavasrc:lavasrc-plugin):apps/bot/src/lib/music).LAVA_EXTERNAL=trueandLAVA_ENABLED=falsedefaulted on cloud deployment templates to prevent Out-Of-Memory (OOM) crashes on 512 MB free containers.HELIX-Origin/Lavalink-Server(Render, Railway, Heroku, Fly.io).5. Web Dashboard Modernization (Next.js 15 & tRPC v11)
apps/dashboard/src/server)..txttranscript channel archiving.6. 1-Click Cloud Deployment Blueprints
render.yaml): Free-tier Docker/Node web service blueprint with automated keep-alive.railway.json): Nixpacks build specification with volume mount path for SQLite persistence (/packages/db/prisma).app.json&Procfile): Single-dyno Eco container manifest with environment variable defaults.fly.toml): MicroVM deployment targeting port 3000 with NVMe volume support.Dockerfile&docker-compose.yml): Production multi-stage single-container build.7. Cleaned Environment Variables
POSTGRES_*,REDIS_*, redundantNEXTAUTH_URLduplications).INTERNA_URLtypo toINTERNAL_URLacross all runtime schemas,.env.example,.env, andturbo.json.8. Monorepo & TypeScript Tooling Hardening
apps/bot/tsconfig.jsonto be self-contained (module: CommonJS,moduleResolution: Node), resolving package resolution issues.ignoreDeprecationscompiler options from all tsconfigs.@master-bot/dbto emit compiled JS and.d.tsdeclaration files viatscso workspace packages consume compiled outputs.9. Complete 14-Page Documentation Wiki
Home.md,Architecture.md,Getting-Started.md,Configuration.md,Deployment.md,Commands.md,Music.md,Dashboard.md,Moderation.md,Tickets.md,Reminders-and-Twitch.md,Welcome-and-Temp-Channels.md,FAQ.md, and_Sidebar.md.