How I run
AI bots in Discord

I run a Discord server where some of the members are bots I built. You type a message in a channel and a bot answers, the way a person would. Ask it a question and it replies. Ask it for a picture, a short video or a piece of music and it makes the file and drops it back in the channel. A few of the bots have their own personalities and run on their own. One of them works in the background on a timer without anyone prompting it.

The brain behind the talking is a language model, and the pictures and video come off a graphics card. The point of this page is that none of it has to cost money. You can run the whole thing on NVIDIA's free hosted models, or you can run it on your own machine with a local model and a local GPU, and the bots behave the same either way. I will show both.

I did not type most of the code. I told the Claude Code agent what I wanted and it wrote the scripts, which is also how you would rebuild any of this. So the page reads two ways. You can follow it to understand how each piece works, or you can copy the prompt at the end of each part and have Claude build your own version.

Nothing here leaks a real key or login. Anywhere a secret would go you will see a placeholder like <YOUR_TOKEN>, and the real values live in one file that never leaves my machine.

There is one main bot, the bridge, that listens in a channel and answers. It is a normal Discord bot built on discord.js: it logs in with a token, watches for messages, and posts replies. The difference from a toy bot is what sits behind it. Each message is handed to a language model, and the model's answer is sent back to the channel. The same bot also runs slash commands that generate images, video and music.

Around the bridge I run a few smaller bots with fixed personalities, named Pixel and Mochi, plus a background worker called the Planner that acts on a timer instead of waiting to be spoken to. They share the same plumbing but each is its own process.

The cloud-free way. NVIDIA gives away a generous free tier of hosted models through its NIM API. You make a key, point the bot at integrate.api.nvidia.com, and the brain, the image descriptions and the prompt rewriting all run there at no cost. This is the easiest start and needs no special hardware.

The fully-local way. If you have a GPU, you can run the language model on your own box behind a small local server, and run the image and video models in ComfyUI on the same card. Now nothing leaves your machine and there is no bill at all. The bot code points at a local address instead of the NVIDIA one, and everything else stays the same. You can also use the Claude command-line tool as the brain, which is how my live-chat bot on the Content Factory page works.

Both paths run the identical bot. Start on the free NVIDIA tier, and move the brain local whenever you want.

This is the shopping list for the whole stack. You will not need all of it for a plain chat bot, so skip what does not apply. Everything here has a free path.

Accounts and keys

Each row links to where you sign up. The Discord token and the model key are the only two you need for a talking bot.

What to install on your machine

On Windows I run all of this inside Ubuntu through WSL. On a Mac use Homebrew, on Linux use your package manager.

Node, for the bot itself

npm init -y
npm i discord.js
npm i -g pm2          # process manager to keep it running

ffmpeg, for stitching and captioning media

winget install Gyan.FFmpeg     # Windows
brew install ffmpeg            # macOS
sudo apt install ffmpeg        # Debian / Ubuntu

ComfyUI, only if you want local image and video on your own GPU

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI && pip install -r requirements.txt
python main.py                 # serves a local API on 127.0.0.1:8188

Or just point Claude at it

You do not have to wire any of this up by hand. Open Claude Code in an empty folder, paste a prompt like the one below, and answer the questions it asks. It will tell you which keys to make, write the bot, and run it with you.

For a specific piece, copy the matching prompt from the bottom of its section below.

discord-bridge.js is the front door of the stack: one Node process under PM2 holding a single gateway WebSocket to Discord. It decides which human messages deserve the agent and posts the reply back to the same channel, while the agent, media services, and memory store all sit behind it and never touch Discord directly.

It listens narrowly. DISCORD_CHANNELS names the guild-and-channel pairs it answers in, with an optional :mention flag so a channel stays quiet until the bot is @-mentioned. Most of the file is about not acting on a message twice, since Discord replays traffic on reconnect and PM2 restarts often. The pieces worth studying are the dedup that survives restarts, a per-user queue so one person cannot open two agent sessions at once (concurrent calls lock the sandbox session), and a small /health endpoint. It began as NVIDIA's NemoClaw reference bridge.

APIs & services

How it's built, step by step

1. Create a Discord app and bot in the Developer Portal, enable the Message Content intent, copy the token.
2. Install discord.js and load secrets from an env file at startup (PM2 has no shell env).
3. Build the Client with the Guilds/GuildMessages/MessageContent/GuildMessageReactions intents and Channel/Message/Reaction/User partials.
4. Parse DISCORD_CHANNELS into an allow-list of guildId/channelId/mentionOnly entries.
5. In messageCreate: drop bots, drop anything outside the allow-list, honor mentionOnly.
6. Run the persisted dedup layer before the agent runs, then forward through the per-user queue and reply.
7. Add health counters, /health via listenWithRetry, an unhandledRejection guard, then client.login under PM2.

Under the hood

Client and login

MessageContent is privileged: enable it in the portal or msg.content arrives empty. Partials cover reactions and messages discord.js has not cached.

const { Client, GatewayIntentBits, Partials } = require("discord.js");
const client = new Client({
  intents: [GatewayIntentBits.Guilds, GatewayIntentBits.GuildMessages,
    GatewayIntentBits.MessageContent, GatewayIntentBits.GuildMessageReactions],
  partials: [Partials.Channel, Partials.Message, Partials.Reaction, Partials.User],
});
client.login(process.env.DISCORD_BOT_TOKEN).catch(() => process.exit(1));

Allow-list and the messageCreate gate

DISCORD_CHANNELS is guild:channel with an optional :mention, comma-separated.

const ALLOWED = (process.env.DISCORD_CHANNELS || "").split(",").filter(Boolean)
  .map((s) => { const [guildId, channelId, flag] = s.trim().split(":");
    return { guildId, channelId, mentionOnly: flag === "mention" }; });

client.on("messageCreate", (msg) => {
  if (msg.author.bot) return;
  const entry = ALLOWED.find((e) => e.guildId === msg.guildId && e.channelId === msg.channelId);
  if (!entry) return;
  if (entry.mentionOnly && !msg.mentions.has(client.user.id)) return;
  handle(msg);
});

Dedup that survives restarts

An exact-id Set (reloaded from disk), a 120s age cutoff for reconnect replays, and a per-user content hash for edit-and-resend.

const processed = new Set();   // boot: add last ~500 ids from a disk log
const seen = new Map();         // "userId:first-100-chars" -> timestamp
// inside handle(msg):
if (processed.has(msg.id)) return;
if (Date.now() - msg.createdTimestamp > 120000) return;          // reconnect replay
const key = `${msg.author.id}:${(msg.content||"").slice(0,100).toLowerCase()}`;
if (seen.get(key) > Date.now() - 300000) return;                 // edit-and-resend
processed.add(msg.id); seen.set(key, Date.now());                // persist both to disk
setTimeout(() => processed.delete(msg.id), 600000);

Per-user queue, then reply

.then(fn, fn) keeps a rejected call from stalling the chain; one in-flight call per user prevents concurrent sandbox sessions.

const queue = new Map();
const enqueue = (uid, fn) => {
  const next = (queue.get(uid) || Promise.resolve()).then(fn, fn);
  queue.set(uid, next);
  next.finally(() => { if (queue.get(uid) === next) queue.delete(uid); });
  return next;
};
let reply = await enqueue(msg.author.id, () => runAgent(fullMessage));
await msg.reply(reply.replace(/\b\d{17,19}\b/g, "[user]")); // never echo raw IDs

Health and listenWithRetry

const health = { msgIn:0, msgOk:0, msgErr:0, agentCalls:0, agentFails:0, dedups:0, rejections:0 };
function listenWithRetry(server, port, host, label, n = 5) {
  server.on("error", (e) => {
    if (e.code === "EADDRINUSE" && n-- > 0) setTimeout(() => server.listen(port, host), 2000);
  });
  server.listen(port, host);
}
// GET /health returns { ok, uptime, queue: queue.size, counters: health,
//   discord: { connected: client.ws?.status === 0, ping: client.ws?.ping } }
process.on("unhandledRejection", () => { health.rejections++; });

const res = await fetch("https://integrate.api.nvidia.com/v1/chat/completions", {
  method: "POST",
  headers: { "Content-Type": "application/json",
    "Authorization": `Bearer ${process.env.NVIDIA_API_KEY}` },
  body: JSON.stringify({ model: "meta/llama-3.1-8b-instruct",
    messages: [{ role: "user", content: fullMessage }], max_tokens: 1024 }),
});

Gotchas & hard-won lessons

• MessageContent is privileged. Without the portal toggle, messages arrive with empty content and the bot looks deaf while throwing no errors.
• Dedup must survive restarts. Reconnects replay traffic and PM2 cycles often, so reload the id set and content map from disk on boot or the backlog replays every time.
• Key the queue by user id and chain with .then(fn, fn), not .then(fn). The two-arg form lets the next message run after a rejected call instead of stalling the chain.
• listenWithRetry exists because a fast PM2 restart can fire before the old process released the port; without the EADDRINUSE backoff the health server fails to bind.
• The bridge hard-exits if the bot token or NVIDIA_API_KEY is missing, and it masks any 17-to-19 digit run in replies so raw Discord IDs never leak into the channel.

Prompts to build it yourself

The kind of instructions you'd hand an AI coding agent (Claude Code) to build this from scratch.

Every reply this bot sends starts the same way: a message lands in an allowed Discord channel, my handler walks it through a few filters, glues some context onto it, and hands the result to a single function called runAgentInSandbox. That name is a small lie I kept on purpose. The original NemoClaw reference design forwarded each turn to an OpenClaw agent living inside an OpenShell sandbox, reached over SSH. In my build the chat turn does not SSH anywhere. It POSTs an OpenAI-shaped payload to a local model proxy on localhost:9340, and that proxy decides which model actually answers. I left the old name because the file header still describes the sandbox idea and I find it useful to remember where the wiring came from.

The proxy is the interesting part. When an NVIDIA_API_KEY is present it forwards the request straight to NVIDIA's hosted inference at integrate.api.nvidia.com, and the Nova persona answers with meta/llama-3.3-70b-instruct. With no key the very next branch hands the turn to a Grok proxy that runs grok-4.20-non-reasoning through Vertex AI's xAI endpoint. There is more code below that: a full OpenAI-to-Gemini generateContent translator with a 30 minute context cache on the system prompt and tools, plus a hook for a local OpenAI-compatible server. Both of those sit after the Grok block returns, so neither one fires in the shipped routing. And since the bridge process exits at startup when NVIDIA_API_KEY is missing, the NIM path is the route this bot takes every single time.

So what is the sandbox doing if the chat turn skips it? Media. The SSH plumbing is real and it earns its keep: resolveOpenshell finds the openshell binary, openshell sandbox ssh-config hands back a connection config, sshArgs bolts on an ephemeral known_hosts so a container restart does not wedge me, and pushImageToSandbox / pullImageFromSandbox base64-pipe images in and out of the sandbox's /tmp for img2img. The sandbox is where skills and image work run, not where the sentence comes from.

The persona itself is just a system prompt. I read SOUL.md off disk at startup and prepend it as the system message. Before the model ever sees the user's words I stack on crew memory, Mochi's past corrections, a knowledge-base search result, current trend data, and any crew pre-consult, then I serialize per user so two messages from the same person cannot trip over each other inside one session. After the model replies I scrub raw user IDs, run a hallucination filter, and intercept media tags like [ZTURBO:] or [GIF:] before the text reaches the channel.

APIs & services

How it's built, step by step

1. Stand up a discord.js client with the Guilds, GuildMessages, MessageContent, and GuildMessageReactions intents, and load DISCORD_BOT_TOKEN, NVIDIA_API_KEY, and the allowed guild:channel list from env. The process exits if the token or NVIDIA key is missing.
2. On messageCreate, drop anything outside the allowed channel, anything from a blocked user, and any message ID you have already processed (a persisted dedup set survives restarts).
3. If the message carries an image, describe it first by calling NVIDIA's vision model (meta/llama-3.2-90b-vision-instruct), so the brain gets text it can reason about.
4. Read SOUL.md once at startup and keep it as the system prompt for the persona, with a hardcoded fallback if the file is missing.
5. Assemble the contextual message: user prefix, crew memory, Mochi's past corrections, a knowledge-base search snippet, trend data, optional crew pre-consult, then the user's text.
6. Serialize per user with an enqueueAgent promise chain so one person's messages run one at a time, then call runAgentInSandbox.
7. runAgentInSandbox builds an OpenAI chat payload and POSTs it to the local proxy on localhost:9340 with an X-Agent-Id header and a 120s timeout.
8. The proxy routes by env: NVIDIA_API_KEY present forwards to integrate.api.nvidia.com (Llama-3.3-70B); with no key the next branch forwards to a Grok proxy (grok-4.20 via Vertex's xAI endpoint). A Gemini generateContent translator and a local-LLM branch also exist in the proxy but sit after the Grok return, so they stay dormant.
9. Stream a throttled progress preview back into Discord as an edited in-progress message while the model thinks.
10. Clean the reply: replace 17-to-19 digit user IDs with [user], run the hallucination filter, and unwrap any leaked JSON array.
11. Intercept media tags ([ZTURBO:], [GIF:], [SUNO:], [MAKE_GIF], [ACESTEP:] and friends) and run the matching generator, pushing or pulling images through the sandbox over SSH when img2img is involved.
12. Send the final text and any attachments to the channel.

Under the hood

The handler builds context, then calls one function

The message handler does the boring, important work: filter, describe images, stack context, serialize per user. The actual brain call is a single function with a deliberately misleading name.

// per-user serialization so two messages from one user can't collide
const agentQueue = new Map(); // userId -> promise chain
function enqueueAgent(userId, fn) {
  const prev = agentQueue.get(userId) || Promise.resolve();
  const next = prev.then(fn, fn);            // run after the previous one resolves OR rejects
  agentQueue.set(userId, next);
  next.finally(() => { if (agentQueue.get(userId) === next) agentQueue.delete(userId); });
  return next;
}

// inside the message handler, after context is assembled:
const contextualMessage =
  userPrefix + memoryContext + correctionsContext +
  vertexSearchContext + trendContext + crewPlanContext + fullMessage;

const reply = await enqueueAgent(msg.author.id, () =>
  runAgentInSandbox(contextualMessage, `dc-${msg.author.id}-${msg.id}`, onProgress)
);

The brain call: an HTTP POST to the local proxy

Despite the name, this never opens an SSH session. It loads the persona from SOUL.md and POSTs an OpenAI chat payload to the proxy on localhost:9340.

let systemPrompt = "You are Nova, lead of a small agent crew. Be concise and decisive.";
try { systemPrompt = fs.readFileSync(SOUL_PATH, "utf8"); } catch {}

function runAgentInSandbox(message, sessionId, onProgress) {
  return new Promise((resolve) => {
    const body = JSON.stringify({
      model: GEMINI_DEFAULT_MODEL,   // the proxy overrides this per route
      messages: [
        { role: "system", content: systemPrompt },
        { role: "user",   content: message },
      ],
      max_tokens: 4096,
      temperature: 0.8,
    });
    const req = http.request({
      hostname: "localhost",
      port: 9340,                    // the local model proxy
      path: "/v1/chat/completions",
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Content-Length": Buffer.byteLength(body),
        "X-Agent-Id": "nova",
      },
    }, (res) => {
      let buf = "";
      res.on("data", (c) => { buf += c; /* throttled onProgress preview elided */ });
      res.on("end", () => {
        try {
          const data = JSON.parse(buf);
          resolve(data.choices?.[0]?.message?.content || "Agent returned nothing");
        } catch (e) { resolve(`Agent error: ${e.message}`); }
      });
    });
    req.setTimeout(120000, () => { req.destroy(); resolve("Agent timed out after 120s"); });
    req.on("error", (e) => resolve(`Agent error: ${e.message}`));
    req.end(body);
  });
}

The proxy picks the model

The :9340 server is where the model choice happens. With an NVIDIA key it forwards straight to NIM. With no key the next branch is unconditional and hands the turn to the Grok proxy. The NVIDIA path is the one the bot always takes, because the bridge will not boot without the key.

const NIM_KEY         = process.env.NVIDIA_API_KEY || "";
const NIM_NOVA_MODEL = process.env.NIM_NOVA_MODEL || "meta/llama-3.3-70b-instruct";
const GROK_MODEL      = process.env.GROK_MODEL      || "xai/grok-4.20-non-reasoning";
const GROK_PROXY_PORT = 9342;

// inside the :9340 server, handling POST /v1/chat/completions:
if (NIM_KEY) {
  const payload = JSON.stringify({ ...oai, model: NIM_NOVA_MODEL, stream: false });
  const nimReq = https.request({
    hostname: "integrate.api.nvidia.com",
    path: "/v1/chat/completions",
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Content-Length": Buffer.byteLength(payload),
      "Authorization": `Bearer ${NIM_KEY}`,
    },
  }, nimRes => { res.writeHead(nimRes.statusCode, { "Content-Type": "application/json" }); nimRes.pipe(res); });
  nimReq.on("error", e => { res.writeHead(502); res.end(JSON.stringify({ error: { message: e.message } })); });
  nimReq.write(payload); nimReq.end();
  return;
}

// No NVIDIA key: this block is UNCONDITIONAL and returns. It forwards to the
// Grok proxy, which then hits Vertex AI's xAI endpoint with grok-4.20.
{
  const grokPayload = JSON.stringify({ ...oai, model: GROK_MODEL });
  const grokReq = http.request({
    hostname: "127.0.0.1", port: GROK_PROXY_PORT,
    path: "/v1/chat/completions", method: "POST",
    headers: { "Content-Type": "application/json", "Content-Length": Buffer.byteLength(grokPayload) },
  }, grokRes => { res.writeHead(grokRes.statusCode, { "Content-Type": "application/json" }); grokRes.pipe(res); });
  grokReq.on("error", e => { res.writeHead(502); res.end(JSON.stringify({ error: { message: e.message } })); });
  grokReq.write(grokPayload); grokReq.end();
  return;
}

// Everything below is dead in the current routing:
//   - a LOCAL_LLM_URL branch (forward to your own OpenAI-compatible server)
//   - a full OpenAI -> Vertex Gemini generateContent translator with a
//     30-minute context cache on the system prompt + tools
// To reach either, move it above the two returns above.

The sandbox over SSH carries media, not the sentence

The SSH helpers are genuinely used, just not for the chat turn. resolveOpenshell finds the binary, the CLI hands back an SSH config, and sshArgs adds an ephemeral known_hosts so a sandbox restart does not block me on a changed host key.

const { resolveOpenshell } = require("./lib/resolve-openshell");
const OPENSHELL = resolveOpenshell();
const SANDBOX   = process.env.SANDBOX_NAME || "my-assistant";

function sshArgs(confPath) {
  return ["-T", "-F", confPath,
    "-o", "StrictHostKeyChecking=accept-new",
    "-o", "UserKnownHostsFile=/dev/null",   // ephemeral: restarts rotate host keys
    "-o", "ConnectTimeout=10"];
}

// push the user's image in for img2img
function pushImageToSandbox(imageBuffer) {
  return new Promise((resolve) => {
    let sshConfig;
    try { sshConfig = execFileSync(OPENSHELL, ["sandbox", "ssh-config", SANDBOX], { encoding: "utf-8" }); }
    catch { return resolve(false); }
    const confDir   = fs.mkdtempSync("/tmp/img-push-");
    const confPath  = `${confDir}/config`;
    fs.writeFileSync(confPath, sshConfig, { mode: 0o600 });
    const proc = spawn("ssh", [...sshArgs(confPath), `openshell-${SANDBOX}`,
      "base64 -d > /tmp/input_image.png"], { timeout: 30000, stdio: ["pipe", "pipe", "pipe"] });
    proc.stdin.end(imageBuffer.toString("base64"));
    proc.on("close", (code) => resolve(code === 0));
    proc.on("error", () => resolve(false));
  });
}

SANDBOX_NAME is validated against RFC 1123 label rules before any of this runs, which keeps shell metacharacters and path traversal out of the openshell-<name> host string.

Cleaning the reply before it ships

The model's text is not trusted as-is. I scrub anything that looks like a raw Discord ID, then run a precompiled hallucination filter and a structural check before the message goes out.

// replace raw 17-19 digit user IDs in public replies
response = response.replace(/\b\d{17,19}\b/g, "[user]");

const hallucinationHit = HALLUCINATION_PATTERNS.find(p => p.test(response));
// structural check: only fire when several bold numbered items AND a dramatic
// closer AND an invented module name (that isn't a known-real one) all appear
const boldNumberedItems = (response.match(/^\d+\.\s+\*\*/gm) || []).length;
const structuralHit =
  boldNumberedItems >= 3 &&
  DRAMATIC_CLOSER_RE.test(response) &&
  INVENTED_MODULE_RE.test(response) && !KNOWN_REAL_RE.test(response);

if (hallucinationHit || structuralHit) {
  await msg.reply("My response got filtered, it looked like I was making stuff up. Try again with a bit more context.");
  return;
}

curl -s https://integrate.api.nvidia.com/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_NVIDIA_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"meta/llama-3.3-70b-instruct",
       "messages":[{"role":"user","content":"say hi in five words"}],
       "max_tokens":32}'

# option A: Ollama
ollama serve
ollama pull llama3.3:70b   # or a smaller model that fits your VRAM

# option B: vLLM, OpenAI-compatible server
python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Llama-3.3-70B-Instruct --port 8000

const LOCAL = process.env.LOCAL_LLM_URL; // e.g. http://localhost:8000
if (LOCAL) {
  const u = new URL(LOCAL.replace(/\/$/, "") + "/v1/chat/completions");
  const body = JSON.stringify({ ...oai, model: process.env.LOCAL_LLM_MODEL || "llama3.3", stream: false });
  const lReq = http.request({ hostname: u.hostname, port: u.port, path: u.pathname, method: "POST",
    headers: { "Content-Type": "application/json", "Content-Length": Buffer.byteLength(body) } },
    lRes => { res.writeHead(lRes.statusCode, { "Content-Type": "application/json" }); lRes.pipe(res); });
  lReq.on("error", e => { res.writeHead(502); res.end(JSON.stringify({ error: { message: e.message } })); });
  lReq.end(body);
  return;
}

Gotchas & hard-won lessons

• The function is named runAgentInSandbox but it does not SSH into the sandbox. It POSTs to the proxy on localhost:9340. Read the body, not the name, or you will waste an afternoon looking for an SSH call that handles chat.
• The brain default is meta/llama-3.3-70b-instruct, not Nemotron. Nemotron (nvidia/llama-3.1-nemotron-ultra-253b-v1) is only the default the hermes inference shim uses on its NIM path, and that shim is a separate process.
• With no NVIDIA key the proxy does not fall back to Gemini. The very next branch is unconditional and forwards to the Grok proxy (grok-4.20 via Vertex's xAI endpoint). The Gemini generateContent translator lives further down and never runs in the shipped order.
• The bridge calls process.exit(1) at startup if NVIDIA_API_KEY is unset, so you cannot run it on the Grok-only path without also satisfying that key check. NIM is the de-facto brain every time.
• The proxy also has a LOCAL_LLM_URL branch, but the NVIDIA block and the Grok block both return before it, so that branch never runs as written. Going fully local means promoting it above both returns, not just setting the env var.
• The hermes inference shim looks local because the container thinks it is talking to llama.cpp, but it proxies to NVIDIA NIM or Vertex Gemini. It is another cloud path, not local inference.
• The proxy on :9340 must be listening before the bridge handles its first message, otherwise every brain call returns Agent error: connect ECONNREFUSED. The proxy servers bind at require-time when services/vertex.js loads, so start the bridge process cleanly.
• Per-user serialization through enqueueAgent matters. Remove it and concurrent messages from one user can lock a session and produce empty or crossed replies.
• known_hosts is intentionally /dev/null with accept-new. Sandbox container restarts rotate host keys, and pinning them would block media transfer after every restart.
• The 120s timeout in runAgentInSandbox is the hard ceiling for a reply. A slow cold model start can hit it and the user gets Agent timed out after 120s instead of an answer.
• The hallucination structural check is a blunt instrument. A legitimate numbered roadmap with three or more bold items plus a flourish at the end can trip it, so keep the three conditions ANDed together (bold-numbered count, dramatic closer, invented-but-not-known module) as they are.

Prompts to build it yourself

The kind of instructions you'd hand an AI coding agent (Claude Code) to build this from scratch.

I run three personas as their own long-lived processes, kept apart from the main Nova bridge. Each one is a standalone Node script that reads its own bot token from its own .env file, carries a system prompt that defines its character, and points at whatever model I assign it. Pixel handles the social and creative side, Mochi plays the cat-flavored validator that checks logic and odds, and the Planner is the odd one out: a headless compute daemon with no Discord presence at all. That last difference is the one that matters, since Pixel and Mochi show up in the server and talk, while the Planner never touches Discord and only reads and writes files.

The reason I split them up is lane discipline. Pixel answers only when she is addressed by name or @mentioned, and she deliberately bails out of any message aimed at Nova so the two bots don't talk over each other. Mochi runs as a reaction layer that weighs in after Nova, and it also carries a second path for an open chatroom where it banters with other bot peers on a cooldown. Because each persona is its own PM2 process, I can restart one without disturbing the others, point each at a different model, and keep their personalities isolated in their own prompt files.

The Planner is the part worth studying. Mochi's command handlers (!odds, !drop, !validate, !result) compute nothing on their own. The compute ones append a task to a shared JSONL file and then poll a results file for the answer. The Planner daemon sits in a one-second loop reading that task file, sending each new task to an NVIDIA-hosted model under a strict JSON-only system prompt, and appending the parsed result. It is a file-backed work queue between two processes, so the Discord side and the compute side fail independently of each other.

One honest warning before you read the code: the function and comment names do not match what the code does. Mochi's LLM call is named callDeepSeek but actually posts to NVIDIA's llama-3.3-70b. Pixel's is named callNemotron but posts to a configurable OpenAI-compatible endpoint that defaults to a local model. I describe what each request really sends, not what the name suggests.

APIs & services

How it's built, step by step

1. Scaffold one Node script per persona (pixel.js, mochi.js, planner-daemon.js), each loading its own .env file and reading a persona-specific bot token with a generic fallback.
2. Create a discord.js Client with the Guilds, GuildMessages, MessageContent, and DirectMessages intents for Pixel and Mochi; give the Planner no client at all.
3. Write each persona's identity as a system prompt: Pixel's creative-director voice, Mochi's terse cat-validator plus a chattier chatroom voice, the Planner's strict JSON spec in a markdown file.
4. Gate the message handler so each persona replies only when addressed (mention or name) and stays out of other agents' lanes (Pixel ignores nova-addressed and her own messages).
5. Wire each persona's LLM caller as an OpenAI-compatible /v1/chat/completions POST and point it at NVIDIA NIM or a local endpoint.
6. Add shared Qdrant memory recall and store over the local :7338 wrapper, and parse the side-channel tokens each persona emits ([REMEMBER], [SEARCH], [ASK_NOVA], [ASK_MOCHI] for Pixel; [REMEMBER], [GIF], [MEME] for Mochi's chatroom) out of the model reply.
7. Build the file-backed task queue: Mochi's !odds and !drop append to tasks.jsonl and poll results.jsonl; the Planner daemon loops over tasks.jsonl, runs the model, and writes results.jsonl.
8. Run all three as separate PM2 processes so each can be restarted and configured on its own.

Under the hood

Three processes, three identities

Every persona loads its own env file and its own token, then either spins up a Discord client or doesn't. That single difference is what separates a chatty persona from the headless Planner.

const { Client, GatewayIntentBits, Partials } = require("discord.js");
const fs = require("fs"), path = require("path");

// Each persona loads its own env file so tokens never collide.
const envFile = path.join(__dirname, "..", ".env.pixel"); // .env.mochi / .env.deepseek
if (fs.existsSync(envFile)) {
  for (const line of fs.readFileSync(envFile, "utf8").split("\n")) {
    const m = line.match(/^([A-Z_][A-Z0-9_]*)=(.*)$/);
    if (m && !process.env[m[1]]) process.env[m[1]] = m[2];
  }
}

const DISCORD_BOT_TOKEN = process.env.PIXEL_BOT_TOKEN || process.env.DISCORD_BOT_TOKEN;

const client = new Client({
  intents: [
    GatewayIntentBits.DirectMessages,
    GatewayIntentBits.MessageContent,
    GatewayIntentBits.Guilds,
    GatewayIntentBits.GuildMessages,
  ],
  partials: [Partials.Channel, Partials.Message],
});
client.login(DISCORD_BOT_TOKEN); // the Planner daemon has none of this block

Lane gating

Pixel only speaks when spoken to, refuses to answer herself, and steps aside whenever Nova is the addressee. This is plain regex on the message content, so tune the names to your own bots.

const PIXEL_USER_ID = "<PIXEL_BOT_USER_ID>"; // her own snowflake, kept out of source

client.on("messageCreate", async (msg) => {
  if (msg.author.bot) return;

  // Stay in lane: ignore anything aimed at Nova, and never answer herself.
  if (/\b(nova|sportstwo|bignova)\b/i.test(msg.content)) return;
  if (msg.author.id === PIXEL_USER_ID) return;

  const isMentioned = msg.mentions.has(client.user.id);
  if (!isMentioned && !/\bpixel\b/i.test(msg.content)) return; // addressed-only

  // ...recall memory, call the model, strip side-channel tokens, reply
});

The persona LLM call

Pixel's caller is a generic OpenAI-compatible POST. The endpoint and model come from env, and it flips to HTTPS with a Bearer key the moment the URL points at NVIDIA. That is the whole cloud-or-local switch.

const LLM_URL = process.env.PIXEL_LLM_URL || "http://127.0.0.1:9342";
const MODEL   = process.env.PIXEL_MODEL   || "<LOCAL_MODEL_NAME>";

async function callModel(userMessage, systemPrompt) {
  const payload = JSON.stringify({
    model: MODEL,
    messages: [
      { role: "system", content: systemPrompt },
      { role: "user",   content: userMessage },
    ],
    temperature: 0.85, top_p: 0.95, max_tokens: 512, stream: false,
  });

  const url = new URL(LLM_URL);
  const isNvidia = LLM_URL.includes("nvidia.com");
  const lib = isNvidia ? require("https") : require("http");

  return new Promise((resolve, reject) => {
    const req = lib.request({
      host: url.hostname,
      port: url.port || (isNvidia ? 443 : 80),
      path: "/v1/chat/completions",
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Content-Length": Buffer.byteLength(payload),
        ...(isNvidia ? { Authorization: `Bearer ${process.env.NVIDIA_API_KEY}` } : {}),
      },
      timeout: 180000,
    }, (res) => {
      let body = ""; res.on("data", c => body += c);
      res.on("end", () => {
        const choice = JSON.parse(body).choices?.[0]?.message;
        // some instruct models route the answer to reasoning_content
        resolve((choice?.content || choice?.reasoning_content || "").trim());
      });
    });
    req.on("error", reject);
    req.write(payload); req.end();
  });
}

Mochi's caller, despite being named callDeepSeek, hardcodes NVIDIA and adds 429 backoff. Two souls drive it: a terse validator at temperature 0.15 and a looser chatroom voice at 0.85.

// Mochi: posts straight to NVIDIA NIM, llama-3.3-70b.
const payload = JSON.stringify({
  model: "meta/llama-3.3-70b-instruct",
  messages: [{ role: "system", content: soul }, { role: "user", content: userMessage }],
  temperature: opts.chatroom ? 0.85 : 0.15,
  max_tokens:  opts.chatroom ? 600  : 1500,
});
const req = https.request({
  hostname: "integrate.api.nvidia.com",
  path: "/v1/chat/completions",
  method: "POST",
  headers: { "Content-Type": "application/json",
             Authorization: `Bearer ${process.env.NVIDIA_API_KEY}` },
}, /* 429 → wait (attempt+1)*15s, up to 3 tries */);

Shared crew memory and side-channels

Both personas reach the same memory through a small HTTP wrapper in front of Qdrant on :7338. The model can also emit bracket tokens that the persona acts on and then deletes from the visible reply.

async function memorySearch(query, userId, limit = 2) {
  const r = await fetch("http://localhost:7338", {
    method: "POST", headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ cmd: "search", query, userId, limit }),
  }).then(r => r.json()).catch(() => null);
  return r?.results || [];
}

// Tokens the model emits; acted on, then stripped before sending.
for (const m of reply.matchAll(/\[REMEMBER:\s*([\s\S]*?)\]/gi)) memoryStore(m[1].trim(), userId);
for (const m of reply.matchAll(/\[ASK_NOVA:\s*([\s\S]*?)\]/gi)) swarmCall("ask_nova", { message: m[1].trim() });
reply = reply.replace(/\[(?:REMEMBER|ASK_NOVA|ASK_MOCHI|SEARCH):[\s\S]*?\]/gi, "").trim();

Note that pixel-memory.js is a different store: a social-media trend log written to ~/.nemoclaw/pixel-trends.jsonl with detectPatterns/compareTrends for velocity and clustering, plus an optional Google Drive backup. It ships with the director persona but is not what Pixel queries on each message.

The file-backed task queue and the Planner loop

Mochi's compute commands write a task line and then poll for a matching result line. The Planner is the other end of that file.

// Mochi side: append a task, then poll for its result.
function submitTask(type, payload) {
  const taskId = `task-${Date.now()}-${Math.random().toString(36).slice(2, 9)}`;
  fs.appendFileSync(TASKS_FILE,
    JSON.stringify({ taskId, type, payload, submittedAt: new Date().toISOString() }) + "\n");
  return taskId;
}
function waitForResult(taskId, timeoutMs = 30000) {
  const start = Date.now();
  return new Promise((resolve) => {
    const poll = setInterval(() => {
      const lines = fs.readFileSync(RESULTS_FILE, "utf8").split("\n").filter(Boolean);
      for (const line of lines) {
        const r = JSON.parse(line);
        if (r.taskId === taskId) { clearInterval(poll); return resolve(r); }
      }
      if (Date.now() - start > timeoutMs) { clearInterval(poll); resolve(null); }
    }, 500); // rescans the whole results file every half second
  });
}

// Planner daemon: no Discord, just files and the model.
const processed = new Set();
async function main() {
  while (true) {
    const lines = fs.existsSync(TASKS_FILE)
      ? fs.readFileSync(TASKS_FILE, "utf8").split("\n").filter(Boolean) : [];
    for (const line of lines) {
      const task = JSON.parse(line);
      if (processed.has(task.taskId)) continue;
      const text = await callModel(task);   // NVIDIA deepseek-v3.1-terminus, temp 0.1
      let result;
      try { result = JSON.parse(text); }    // the system prompt demands JSON-only
      catch { result = { taskId: task.taskId, type: task.type,
                         result: "ERROR", details: { error: "non-JSON output" } }; }
      fs.appendFileSync(RESULTS_FILE, JSON.stringify(result) + "\n");
      processed.add(task.taskId);
    }
    await new Promise(r => setTimeout(r, 1000)); // poll every second
  }
}
main();

The Planner's system prompt is a markdown file that names four task types (validateOutcome, calculateOdds, generateDrop, auditLogic), pins a deterministic xorshift32 RNG so anti-cheat re-simulation matches the client, and forbids any prose in the output. Every result line has to carry taskId, type, result, details, and completedAt.

Gotchas & hard-won lessons

• The Planner's processed-task tracking is an in-memory Set, and the loop re-reads the entire tasks.jsonl every cycle. Restart the daemon and it replays every historical task, appending duplicate result lines.
• tasks.jsonl and results.jsonl are append-only and never truncated. waitForResult rescans the whole results file every 500ms, so both files grow without bound and polling gets slower over time. Rotate or compact them.
• Function and comment names misname their backends. Mochi's callDeepSeek hits NVIDIA llama-3.3-70b, Pixel's callNemotron posts to a configurable endpoint that defaults to a local model, and only the Planner's deepseek name is accurate. Read the request, not the name.
• Pixel and Mochi recall the shared Qdrant store on :7338 on every message inside a try/catch. If that wrapper is down they answer with no memory and give the user no error, which can look like amnesia rather than an outage.
• pixel-memory.js (the trend store) is not the memory Pixel queries per message. pixel.js requires only the Brave helper, so the two memories are unrelated despite the shared persona name.
• Each persona runs an internal HTTP server so the bridge or swarm-mcp can post as them (Pixel on :7701, Mochi on :7702). Launch a second copy of either and you get EADDRINUSE.
• Lane gating is regex on raw text. If someone writes the word that matches another bot's name in an unrelated sentence, the persona stays silent by design, so adjust the patterns to your own bot names.
• Mochi's compute commands depend on the Planner being alive. With the daemon stopped, !odds and !drop time out after 30s with no computed answer, because nothing ever writes the matching result line. !validate only replies with a static prompt for game-state input and !result just reads the existing results file, so those two keep working regardless.

Prompts to build it yourself

The kind of instructions you'd hand an AI coding agent (Claude Code) to build this from scratch.

This is the part of the bot a user actually touches. Everything visible in Discord (the autocomplete list when you type /, the buttons under a generated image, the little popup form for a GIF clip) lives across five files. slash-commands.js declares the commands and pushes them to Discord over REST. Three handler modules under handlers/ carry the runtime logic: commands.js (slash commands), buttons.js (every button click), and modals.js (which holds both the select-menu handler and the modal-submit handler in one file because they share the /create flow). And ui/components.js builds the button and menu rows. I kept the wiring deliberately boring so I can add a command without re-reading the whole codebase.

The shape is one dispatcher and four handler functions. A single client.on("interactionCreate", ...) block in discord-bridge.js checks what kind of interaction came in and forwards it: isButton() goes to handleButton, isStringSelectMenu() to handleStringSelectMenu, isModalSubmit() to handleModalSubmit, and a plain isChatInputCommand() to handleCommand. Each handler takes (interaction, deps). The ComfyUI and Grok service clients are required directly at the top of each handler module, so deps is reserved for the higher-level helpers: the Instagram poster (postToBuffer), the prompt enhancer (enhanceVideoPrompt), the local ZTurbo image generator, the sandbox agent runner, the backup and segment helpers, and the owner id. One quirk worth knowing is that those deps objects start out empty when the dispatcher is wired, then get filled in with Object.assign further down the file once the rest of the bridge has defined the helpers.

The trick that makes the whole thing hold together is the customId. Discord hands you back nothing but a short string when a component is clicked, so I encode both the route and a context key into it: btn_<action>_<msgId>. The button handler parses that with one regex, then looks up the per-message generation context (the prompt, the image buffer, the video buffer) from an in-memory map keyed by the id baked into that string. That key is the originating interaction id for a fresh generation, and the handler falls back to interaction.message.id and the referenced message id when it does not find a hit. Select menus carry their state the same way with colon-delimited ids, like create_edit_style:short, so a multi-step wizard can carry its choices forward without any session storage. When the context map gets cleared (a restart, say), the handlers fall back to re-downloading the media straight off the Discord message attachment.

Modals are the one piece with a sharp edge. A modal has to be the very first reply to an interaction, so any button that needs text input opens the modal immediately and does the slow work later, on the modal's submit. That single rule explains most of the control flow you see. A "Make Video" button pops a duration modal (modal_i2v_dur_video_<msgId>), and the actual render runs in handleModalSubmit, where the duration field is read and the clip is generated through ComfyUI.</overview> <apis">[{"name":"discord.js","role":"Builders (SlashCommandBuilder, ActionRowBuilder, ButtonBuilder, StringSelectMenuBuilder, ModalBuilder, TextInputBuilder) plus the REST client and Routes for registering commands","doc_url":"https://discord.js.org/docs"},{"name":"Discord Application Commands API","role":"REST endpoint the bot PUTs its command definitions to (global registration via Routes.applicationCommands)","doc_url":"https://discord.com/developers/docs/interactions/application-commands"},{"name":"Discord Interactions (receiving & responding)","role":"The reply/defer/update/showModal lifecycle and the 3-second acknowledge window every handler obeys","doc_url":"https://discord.com/developers/docs/interactions/receiving-and-responding"},{"name":"NVIDIA NIM (integrate.api.nvidia.com)","role":"Free OpenAI-compatible chat completions for the prompt-enhance and caption-rewrite steps the handlers call into","doc_url":"https://build.nvidia.com"},{"name":"ComfyUI","role":"Local GPU backend the generation helpers (image/video/music) talk to over HTTP","doc_url":"https://docs.comfy.org"}]

APIs & services

How it's built, step by step

1. Declare each command with SlashCommandBuilder in slash-commands.js: setName, setDescription, then chain addStringOption / addIntegerOption / addBooleanOption / addAttachmentOption, with addChoices for dropdowns and addSubcommand for grouped actions (the /mc command does this).
2. On the client 'ready' event, call registerCommands(token, clientId): build a REST({ version: '10' }) client and PUT the JSON of all commands to Routes.applicationCommands(clientId) for a global register.
3. Wire one interactionCreate listener that branches on interaction type (isButton / isStringSelectMenu / isModalSubmit / isChatInputCommand) and forwards to the matching handler with a shared deps object.
4. In handleCommand, switch on interaction.commandName, read options with interaction.options.getString/getInteger/getAttachment, deferReply() for slow work, run the generation, then editReply with files plus component rows.
5. Build every button/menu row through a helper in ui/components.js so the customId convention (btn_<action>_<msgId>) and Discord's 5-buttons-per-row / 5-rows-per-message limits live in one place.
6. Store the per-message generation context (prompt, buffers, type) in the state map under the originating interaction id (the same id the buttons carry); button-driven follow-ups re-key the context under the new message id.
7. In handleButton, parse customId with /^btn_(\w+)_(.+)$/, resolve context by that key with fallbacks to interaction.message.id and the referenced message, then dispatch by action: reply, defer+editReply, update the message, or showModal for actions that need text input.
8. In handleStringSelectMenu, advance the wizard by reading interaction.values[0] and either interaction.update() with the next menu or showModal for the final input step, carrying state forward in colon-delimited customIds.
9. In handleModalSubmit, match the customId, read fields with interaction.fields.getTextInputValue(id), then run the deferred generation and reply with the result plus a fresh row of buttons.

Under the hood

Declaring commands

Commands are plain SlashCommandBuilder objects in an array. Options come in typed flavors, and addChoices turns a free-text option into a dropdown. Subcommands group related actions under one name, which is how the real /mc command exposes start, stop, and status.

const { SlashCommandBuilder, REST, Routes } = require("discord.js");

const commands = [
  new SlashCommandBuilder()
    .setName("imagine")
    .setDescription("Generate an image")
    .addStringOption(o => o.setName("prompt").setDescription("What to generate").setRequired(true))
    .addStringOption(o => o.setName("ratio").setDescription("Aspect ratio")
      .addChoices(
        { name: "1:1 (Square)",     value: "1:1" },
        { name: "16:9 (Landscape)", value: "16:9" },
      )),

  // Subcommands: /mc start | stop | status
  new SlashCommandBuilder()
    .setName("mc")
    .setDescription("Control the agent crew")
    .addSubcommand(s => s.setName("start").setDescription("Start the crew")
      .addIntegerOption(o => o.setName("agents").setDescription("1-4").setMinValue(1).setMaxValue(4)))
    .addSubcommand(s => s.setName("stop").setDescription("Stop the crew")),
];

Registering over REST

Registration is one PUT. Routes.applicationCommands(clientId) registers globally, which is what I want in production. Global commands can take up to an hour to propagate, so during development I register to a single guild instead (see gotchas).

async function registerCommands(token, clientId) {
  const rest = new REST({ version: "10" }).setToken(token);
  await rest.put(
    Routes.applicationCommands(clientId),
    { body: commands.map(c => c.toJSON()) },
  );
}
// called from the ready handler:
//   await registerCommands(process.env.DISCORD_BOT_TOKEN, client.user.id);

The one dispatcher

Every interaction lands here and is forwarded by type. The deps objects are how the higher-level helpers reach the handlers without the handlers importing them directly. They start empty and get filled in with Object.assign once the rest of the bridge has defined those helpers.

const { handleCommand }                             = require("./handlers/commands");
const { handleButton }                              = require("./handlers/buttons");
const { handleStringSelectMenu, handleModalSubmit } = require("./handlers/modals");

const commandsCtx = {}, buttonsCtx = {}, modalsCtx = {}; // filled later via Object.assign

client.on("interactionCreate", async (interaction) => {
  try {
    if (interaction.isButton())            { await handleButton(interaction, buttonsCtx); return; }
    if (interaction.isStringSelectMenu())  { await handleStringSelectMenu(interaction, modalsCtx); return; }
    if (interaction.isModalSubmit())       { await handleModalSubmit(interaction, modalsCtx); return; }
    if (!interaction.isChatInputCommand()) return;
    await handleCommand(interaction, commandsCtx);
  } catch (e) {
    const reply = interaction.deferred || interaction.replied
      ? (m) => interaction.editReply(m).catch(() => {})
      : (m) => interaction.reply({ content: m, ephemeral: true }).catch(() => {});
    await reply(`Error: ${e.message.slice(0, 200)}`);
  }
});

A command handler

handleCommand switches on interaction.commandName. The pattern is always the same: read options, deferReply() because generation is slow, do the work, then editReply with the file and a row of buttons. The originating interaction's id becomes the key for the stored context, and the same id is baked into the button customIds. generateImage below stands in for whatever backend you wire (in this stack that is a local ComfyUI workflow or the sandbox agent).

if (cmd === "imagine") {
  const prompt = interaction.options.getString("prompt");
  const ratio  = interaction.options.getString("ratio") || "1:1";
  await interaction.deferReply();

  const imgBuf = await generateImage(prompt, ratio);   // your backend
  const tmp = `/tmp/img-${Date.now()}.png`;
  fs.writeFileSync(tmp, imgBuf);

  await interaction.editReply({
    content: `🎨 *"${prompt.slice(0, 80)}"*`,
    files: [new AttachmentBuilder(tmp, { name: "image.png" })],
    components: imageButtons(interaction.id),           // builder from ui/components.js
  });
  state.setGenerationContext(interaction.id, { prompt, ratio, imageBuf: imgBuf, type: "image" });
  fs.unlink(tmp, () => {});
}

Component builders and the customId convention

ui/components.js is the only place that knows the id format and Discord's layout limits (5 buttons per row, 5 rows per message). Encoding the context key into every id is what lets a click hours later still find its data.

function imageButtons(msgId) {
  return [
    new ActionRowBuilder().addComponents(
      new ButtonBuilder().setCustomId(`btn_video_${msgId}`).setLabel("🎬 Make Video").setStyle(ButtonStyle.Primary),
      new ButtonBuilder().setCustomId(`btn_enhance_${msgId}`).setLabel("✨ Enhance & Video").setStyle(ButtonStyle.Primary),
      new ButtonBuilder().setCustomId(`btn_groki2v_${msgId}`).setLabel("🤖 Grok i2v").setStyle(ButtonStyle.Primary),
    ),
    new ActionRowBuilder().addComponents(
      new ButtonBuilder().setCustomId(`btn_post_${msgId}`).setLabel("📱 Post to IG").setStyle(ButtonStyle.Success),
      new ButtonBuilder().setCustomId(`btn_regen_${msgId}`).setLabel("🔄 Regenerate").setStyle(ButtonStyle.Secondary),
    ),
  ];
}

// Rows can be built conditionally. A "Stitch All" button only appears
// once two or more video segments have accumulated for this message:
function videoButtons(msgId) {
  const items = [ new ButtonBuilder().setCustomId(`btn_gif_${msgId}`).setLabel("🎞 Make GIF").setStyle(ButtonStyle.Secondary) ];
  const segs = state.getStorySegments(msgId);
  if (segs && segs.length >= 2) {
    items.push(new ButtonBuilder().setCustomId(`btn_stitch_${msgId}`).setLabel(`🎬 Stitch All (${segs.length})`).setStyle(ButtonStyle.Danger));
  }
  return [ new ActionRowBuilder().addComponents(...items) ];
}

The grid builder for Grok's four-image output is where the per-row cap actually bites: it packs the selection buttons plus a Regen button into one or two rows depending on count, so it never overflows a single ActionRow.

The button handler

Parse the id, resolve the context, dispatch on the action. Note the fallback chain for context, and that some actions open a modal rather than acting immediately.

async function handleButton(interaction, deps) {
  const [, action, msgId] = interaction.customId.match(/^btn_(\w+)_(.+)$/) || [];
  if (!action) return;

  const ctx = state.getGenerationContext(msgId)
    || state.getGenerationContext(interaction.message?.id)
    || state.getGenerationContext(interaction.message?.reference?.messageId)
    || {};

  // Action that needs text input -> open a modal as the FIRST response:
  if (action === "video") {
    const modal = new ModalBuilder()
      .setCustomId(`modal_i2v_dur_video_${msgId}`)
      .setTitle("🎬 Make Video")
      .addComponents(new ActionRowBuilder().addComponents(
        new TextInputBuilder()
          .setCustomId("i2v_duration")
          .setLabel("Duration in seconds (2-30, default 10)")
          .setStyle(TextInputStyle.Short)
          .setRequired(false),
      ));
    return interaction.showModal(modal);
  }

  // Action that just acts -> defer, work, edit:
  if (action === "regen") {
    await interaction.deferReply();
    if (!ctx.prompt) return interaction.editReply("⚠️ Lost the prompt for this image.");
    const imgBuf = await deps.generateImageWithZTurbo(ctx.prompt, seed, ctx.style || "none");
    /* write temp, editReply with imageButtons(interaction.message.id), store ctx */
  }

  // Owner-only action:
  if (action === "post") {
    if (interaction.user.id !== deps.OWNER_ID_GLOBAL) {
      return interaction.reply({ content: "⚠️ Owner only.", ephemeral: true });
    }
    /* ... */
  }
}

Select menus as a wizard, ending in a modal

/create opens a type menu; each pick edits the same message into the next menu with interaction.update(); the final pick opens a modal. State rides along in the customId.

async function handleStringSelectMenu(interaction, deps) {
  if (interaction.customId === "create_type") {
    const type = interaction.values[0];                  // "image" | "video" | ...
    if (type === "image") {
      const row = new ActionRowBuilder().addComponents(
        new StringSelectMenuBuilder()
          .setCustomId("create_image_model")
          .setPlaceholder("Choose image model…")
          .addOptions([{ label: "ZImage Turbo", value: "zturbo", emoji: "⚡" }]),
      );
      return interaction.update({ content: "🎨 Pick a model:", components: [row] });
    }
  }

  if (interaction.customId === "create_image_model") {
    const model = interaction.values[0];
    const modal = new ModalBuilder()
      .setCustomId(`modal_create_img_${model}`)          // carries the model choice
      .setTitle("Describe the image")
      .addComponents(new ActionRowBuilder().addComponents(
        new TextInputBuilder().setCustomId("create_prompt").setLabel("Prompt").setStyle(TextInputStyle.Paragraph).setRequired(true),
      ));
    return interaction.showModal(modal);
  }
}

The modal submit

Match the id, pull fields by their input ids, then run the deferred work.

async function handleModalSubmit(interaction, deps) {
  const m = interaction.customId.match(/^modal_create_img_(\w+)$/);
  if (m) {
    const model  = m[1];
    const prompt = interaction.fields.getTextInputValue("create_prompt").trim();
    await interaction.deferReply();
    const imgBuf = await deps.generateImageWithZTurbo(prompt, seed, "none");
    /* write temp, editReply with imageButtons(interaction.id), store ctx */
  }
}

Adding one command end to end

Say I want /remix <prompt> that generates and shows a Regenerate button.

1. In slash-commands.js, add the builder to the commands array and let registerCommands push it:

new SlashCommandBuilder()
  .setName("remix")
  .setDescription("Generate a remix from a prompt")
  .addStringOption(o => o.setName("prompt").setDescription("What to remix").setRequired(true)),

1. In handlers/commands.js, add a branch:

if (cmd === "remix") {
  const prompt = interaction.options.getString("prompt");
  await interaction.deferReply();
  const buf = await generateImage(prompt, "1:1");
  const tmp = `/tmp/remix-${Date.now()}.png`; fs.writeFileSync(tmp, buf);
  await interaction.editReply({
    content: `🎛 *"${prompt.slice(0, 80)}"*`,
    files: [new AttachmentBuilder(tmp, { name: "remix.png" })],
    components: remixButtons(interaction.id),
  });
  state.setGenerationContext(interaction.id, { prompt, imageBuf: buf, type: "remix" });
  fs.unlink(tmp, () => {});
  return;
}

1. In ui/components.js, add and export the builder:

function remixButtons(msgId) {
  return [ new ActionRowBuilder().addComponents(
    new ButtonBuilder().setCustomId(`btn_remixregen_${msgId}`).setLabel("🔄 Remix again").setStyle(ButtonStyle.Secondary),
  )];
}

1. In handlers/buttons.js, handle the remixregen action (the dispatcher already routes every button here, so no change there):

if (action === "remixregen") {
  await interaction.deferReply();
  const buf = await deps.generateImage(ctx.prompt, "1:1");
  /* write temp, editReply with remixButtons(interaction.message.id), re-store ctx */
}

Restart, and because the command was re-registered on boot, /remix shows up in Discord with its button working.

const res = await fetch("https://integrate.api.nvidia.com/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.NVIDIA_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "meta/llama-3.1-8b-instruct",       // any catalog model from build.nvidia.com
    messages: [{ role: "user", content: `Rewrite this prompt with rich cinematic detail: ${prompt}` }],
    max_tokens: 300,
    temperature: 0.8,
  }),
});
const enhanced = (await res.json()).choices?.[0]?.message?.content?.trim();

Gotchas & hard-won lessons

• Every interaction must be acknowledged within 3 seconds. If the work is slow, deferReply() (or deferUpdate() for a menu) right away and editReply later. Skip the defer and Discord shows 'This interaction failed'.
• showModal must be the FIRST response to an interaction. You cannot deferReply and then showModal. That is why buttons that need input open the modal immediately and do the heavy work on the modal's submit instead.
• Global command registration via Routes.applicationCommands can take up to an hour to appear. For fast iteration register to one guild with Routes.applicationGuildCommands(clientId, guildId), which updates instantly, then switch to global for release.
• customId is capped at 100 characters. The btn_<action>_<msgId> scheme fits, but do not try to stuff a whole prompt in there. Keep ids to a route plus a lookup key and hold the real payload in your state map.
• Discord enforces 5 buttons per ActionRow and 5 rows per message. The grid builder splits rows to respect this. A 6th button in one row throws at send time, not at definition time.
• Action names are parsed out of customId by a single regex. If two actions would collide on a numeric suffix (grokvid0 versus grokvid), encode the index inside the action name so the parser does not confuse them.
• Use reply for a brand-new message, update to edit the message the component lives on (menus do this to advance a wizard), and editReply only after you have deferred. Mixing these up double-replies and errors out.
• In-memory generation context is lost on restart. The handlers recover by re-downloading the image or video straight off the Discord message attachments, so design any new action to tolerate an empty ctx.
• Owner-only actions compare interaction.user.id against your own user id. Read that id from an env var rather than pasting it inline, or the check becomes a published secret and an awkward thing to rotate.

Prompts to build it yourself

The kind of instructions you'd hand an AI coding agent (Claude Code) to build this from scratch.

The media side is my favorite part to run locally, because once a GPU is in the box the marginal cost of a generated image drops to zero. Video and music ride the same hardware. The shape of every media command is the same: a user types something like /video a neon river at night in Discord, the bot loads a ComfyUI workflow graph, patches my prompt into the right node, submits it to the GPU, waits for the render, and posts the finished file straight back into the channel as an attachment. No per-call billing sits anywhere on that path.

Three kinds of output run through ComfyUI on my own card. Stills come from a ZImage Turbo workflow behind /zturbo. Clips run through LTX Video 2.3, with /video for plain text-to-video and /combi for a first-frame-plus-last-frame blend, while /music writes full songs with ACE-Step. The cloud lane is there when I want it: Grok/Aurora for images through a Playwright-driven browser on a local grok-server, and Suno for songs. Those cost money or ride fragile auth, so I treat them as optional and lead with the local path.

Once I have raw clips and a track, two compositors turn them into something postable. One is a plain ffmpeg engine (video-editor.js) that does Ken Burns on stills, crossfades or beat-synced jumpcuts, lyric overlays, an audio mix, and a final size pass to fit Discord's upload cap. The other (capcut-compose.js plus capcut-client.js) builds CapCut drafts over a local API so I can finish by hand in the desktop app. The deeper timeline math and renderer internals are shared with the Content Factory page, so I only cover the bot-specific glue here and point you at /factory.html for the rest.

If you do not have an NVIDIA card, none of this is out of reach. NVIDIA's NIM endpoints at integrate.api.nvidia.com give you a free hosted lane for the inference, and the bot glue stays the same: a command in, a file out.</overview> <apis">[]

APIs & services

How it's built, step by step

1. Register a slash command (for example /video, /zturbo, or /music) with a prompt string option in discord.js.
2. On invoke, call deferReply() right away, since a GPU render takes far longer than Discord's 3-second ack window.
3. Load the matching ComfyUI workflow JSON from disk at call time, not at boot, so a missing or stale graph surfaces per request.
4. Patch the input nodes by id: the positive-prompt text, a random seed, the duration, and any uploaded image names.
5. Call freeComfyMemory() so the GPU unloads the previous model and can load this one.
6. POST the graph to ComfyUI /prompt and capture the returned prompt_id.
7. Poll /history/<prompt_id> every few seconds until status.completed, handling the error and timeout cases.
8. Download the finished file from /view as a Buffer.
9. Optionally compose: feed stills, clips, and the track through video-editor.js (ffmpeg) or capcut-compose.js, then size-guard the result under Discord's 25MB cap.
10. Write the Buffer to a temp file and editReply with it wrapped in an AttachmentBuilder, so the file lands back in the channel.

Under the hood

The command-to-ComfyUI-to-Discord loop

Every media command has the same skeleton in handlers/commands.js. Defer first, render, write the Buffer to a temp file, attach it, then clean up. This is the /video text-to-video path, trimmed to the load-bearing lines:

// handlers/commands.js
if (cmd === "video") {
  const prompt = interaction.options.getString("prompt");
  await interaction.deferReply(); // a render outlasts Discord's 3s ack window

  const queue = await comfy.getComfyQueueStatus();
  await interaction.editReply(`Rendering: "${prompt.slice(0, 60)}"` +
    (queue.total ? ` (${queue.total} in queue)` : ""));

  const videoBuf = await comfy.generateVideoWithComfyUI(prompt, null); // T2V, returns a Buffer
  const tmp = `/tmp/video-${Date.now()}.mp4`;
  fs.writeFileSync(tmp, videoBuf);
  await interaction.editReply({
    content: `"${prompt.slice(0, 60)}" via LTX Video 2.3`,
    files: [new AttachmentBuilder(tmp, { name: "video.mp4" })],
  });
  fs.unlinkSync(tmp);
}

The image command /zturbo is the same idea with generateImageWithZTurbo(prompt, seed, style) returning a PNG Buffer, and /music swaps in generateMusicWithAceStep(tags, lyrics, duration). Only the generator changes between commands; the defer-render-attach wrapper stays put.

Patching a workflow graph by node id

A ComfyUI workflow is a JSON object keyed by node id. The bot reads the exported graph, overwrites the inputs it cares about, and submits. Node numbers are specific to the saved file, so the code addresses them directly. This is the fast text-to-video workflow:

// services/comfy.js
async function generateFastT2V(prompt, durationSec = 10) {
  await freeComfyMemory();                 // unload whatever model is resident
  const seed = Math.floor(Math.random() * 2147483647);
  const dur  = Math.max(2, Math.min(30, durationSec || 10));

  const workflow = JSON.parse(fs.readFileSync(COMFY_T2V_FAST_WORKFLOW, "utf-8"));
  workflow["18"].inputs.text       = prompt; // positive prompt node
  workflow["33"].inputs.noise_seed = seed;   // pass 1 seed
  workflow["61"].inputs.noise_seed = seed;   // pass 2 seed
  workflow["48"].inputs.value      = dur;    // duration in seconds

  const promptId = await submitComfyWorkflow(workflow);
  const fileInfo = await waitForComfyResult(promptId, 900000);
  return await downloadComfyFile(fileInfo);  // Buffer of the finished mp4
}

For image-to-video the bot uploads the source frame first with uploadImageToComfyUI() and writes the returned name into the LoadImage node, then runs the same submit-and-poll loop.

Talking to ComfyUI

One small HTTP helper handles every call. The interesting part is host discovery: from WSL, ComfyUI running on the Windows host is reachable at whatever nameserver shows up in /etc/resolv.conf, not localhost. The code also prefers a queue proxy on port 5002 when its health check reports redis ok, and falls back to direct ComfyUI on 8188 otherwise.

function getComfyHost() {
  if (_comfyQueueAvailable === true) return "localhost"; // queue proxy on :5002
  if (process.env.COMFYUI_HOST) return process.env.COMFYUI_HOST;
  // WSL: ComfyUI on the Windows host is the resolv.conf nameserver
  try {
    const m = fs.readFileSync("/etc/resolv.conf", "utf-8").match(/^nameserver\s+(\S+)/m);
    return m ? m[1] : "<WINDOWS_HOST_IP>";
  } catch { return "<WINDOWS_HOST_IP>"; }
}
function getComfyPort() {
  return _comfyQueueAvailable === true ? 5002 : 8188; // proxy vs direct
}

freeComfyMemory() is what lets a single card do stills, video, and music in one session. It POSTs /free with unload_models before each job so the next model has room:

async function freeComfyMemory() {
  await comfyRequest("POST", "/free",
    JSON.stringify({ unload_models: true, free_memory: true }));
}

Music locally with ACE-Step

Same pattern, audio output. Tags go in the caption node, lyrics in the lyrics node, and an empty lyric string flips the workflow to instrumental so the model does not sing gibberish over silence:

async function generateMusicWithAceStep(tags, lyrics, durationSec = 60) {
  await freeComfyMemory();
  const workflow = JSON.parse(fs.readFileSync(ACEMUSIC_WORKFLOW, "utf-8"));
  workflow["4"].inputs.caption      = tags;            // genre / mood prompt
  workflow["3"].inputs.lyrics       = lyrics;
  workflow["2"].inputs.duration     = durationSec;
  workflow["2"].inputs.seed         = Math.floor(Math.random() * 2147483647);
  workflow["2"].inputs.instrumental = !lyrics.trim();  // blank lyrics => instrumental

  const promptId = await submitComfyWorkflow(workflow);
  const fileInfo = await waitForComfyAudio(promptId);  // prefers the SaveAudioMP3 output
  return await downloadComfyFile(fileInfo);            // Buffer of the mp3
}

Composing the result

After generation I usually want a real edit, not a bare clip. video-editor.js is the ffmpeg engine: it builds a timeline from stills and clips, applies a color style, optionally syncs cuts to detected beats (the brainslop and ludicrous styles use a jumpcut timeline), burns lyric captions, mixes the song, and ends with a size guard that re-encodes anything over 24MB so it clears Discord's 25MB upload limit. The CapCut path in capcut-compose.js and capcut-client.js instead builds a draft over a local API (all times in microseconds, and the *_infos endpoints hand back JSON strings you feed straight into the add_* calls) for finishing in the desktop app. The shared timeline and renderer internals live with the Content Factory, so I keep those out of this section and send you to /factory.html.

curl https://integrate.api.nvidia.com/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_NVIDIA_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"meta/llama-3.1-70b-instruct",
       "messages":[{"role":"user",
         "content":"Rewrite into a vivid LTX video prompt: neon rain over a river"}]}'

Gotchas & hard-won lessons

• A GPU render blows past Discord's 3-second interaction ack window. You must deferReply() immediately and editReply with the file later, or the interaction dies before the render finishes.
• freeComfyMemory() before every job is what lets one card swap between LTX video, ZImage stills, and ACE-Step music. Skip it and the second model out-of-memories on a consumer card.
• Workflows are patched by node id (for example 18 = positive prompt and 48 = duration in the T2V graph; 2/3/4 in the ACE-Step graph). If you re-export the graph from ComfyUI the ids renumber and the patch silently writes to the wrong nodes. Pin the workflow JSON.
• From WSL, ComfyUI on the Windows host is the nameserver in /etc/resolv.conf, not localhost. Set COMFYUI_HOST to override when you run ComfyUI somewhere else.
• The bot checks a queue proxy on port 5002 at startup; if its health endpoint reports redis ok it routes jobs there, otherwise it falls back to direct ComfyUI on 8188. Know which one is actually serving when a job stalls.
• ACE-Step with an empty lyric string produces gibberish vocals. The code forces instrumental when the lyrics are blank, so keep that guard if you reimplement it.
• Discord caps uploads near 25MB. video-editor.js re-encodes anything over 24MB to fit, so render at high quality and let the size guard shrink it on the way out.
• The LTX low-VRAM path leans on Q4 GGUF weights, a distilled LoRA, and 2-pass sampling to fit a consumer card. That is what makes the node map dense and fragile to re-export.
• CapCut Mate takes all times in microseconds, and its *_infos endpoints return JSON strings you pass directly into add_videos / add_images / add_captions. It also needs a local file server so the CapCut side can fetch your media by URL.
• Suno's direct path renews short-lived access tokens about every 60 seconds, but the real fragility is the refresh JWT you pull from the browser plus a captcha solver. When that refresh token expires the whole path breaks. Local ACE-Step has none of that overhead.
• A second ComfyUI instance bound to a second GPU (cuda:1) can render stills in parallel while the main instance renders video, via the withComfyPort port-override (set _COMFY_PORT_OVERRIDE for the still job).

Prompts to build it yourself

The kind of instructions you'd hand an AI coding agent (Claude Code) to build this from scratch.

Once a bot can make images, music, and video inside Discord, the next problem is getting that media out into the world and keeping the whole stack running while nobody is watching. This section covers the last mile: pushing a finished render to Instagram, parking the file on a public URL first because Instagram refuses to read a Discord CDN link or a raw buffer, backing every generated file up to Google Drive, and running the bot under pm2 so a crash or a reboot does not take it down for the night.

The piece that surprises most people is that you cannot hand Instagram a file. The Graph API wants a publicly reachable URL that it can fetch on its own, so my bridge uploads the bytes somewhere public, gets back a link, and only then starts the Instagram post. I run a Cloudflare R2 worker as the primary host (no OAuth, just a static upload secret, and a 7-day object lifecycle that is plenty since Instagram fetches within seconds), with Catbox Litterbox for video and Imgur for images as the fallback when R2 is not configured or is down. After the URL exists, Instagram takes two write calls with a wait wedged between them: create a media container, poll it until it reports FINISHED, then publish.

On the ops side, the single most important fact is that pm2 does not source your shell profile. If your secrets live in .bashrc or .profile, a pm2-managed process starts blind. My fix is one plain env file, ~/.nemoclaw_env, that the bridge parses by hand on startup, and that the smoke test reads to confirm the critical keys are present before anything restarts. Everything else about keeping the lights on follows a few hard rules I learned the painful way: restart processes, never delete them, run the smoke test as a gate before every restart, and pm2 save after adding a process so it survives a reboot.

I also auto-back-up every render to Drive. Generation is cheap to trigger and slow to redo, so the moment a buffer comes back larger than a kilobyte I drop it to a temp file and fire it off to a Drive folder in the background. A separate cron tarball bundles the scripts, the env files, and the pm2 state to Drive every thirty minutes on top of that, keeping the last 48 copies.

APIs & services

How it's built, step by step

1. Put all secrets in one flat env file (~/.nemoclaw_env) as KEY=value lines, because pm2 will not load your shell profile.
2. At the top of the bridge, parse that env file line by line into process.env before any other module reads a key.
3. When a render finishes, back the buffer up to Google Drive in the background if it is larger than ~1KB.
4. To post, first upload the bytes to a public host: try the Cloudflare R2 worker (PUT with a secret header), fall back to Catbox for video or Imgur for images.
5. Create an Instagram media container with the public URL and the caption (image_url for stills, media_type=REELS + video_url + share_to_feed for video).
6. Poll the container's status_code every few seconds until it reads FINISHED (bail on ERROR), giving video a longer interval than images.
7. Call media_publish with the container's creation id to push it live.
8. Separately, run a token-expiry check that warns when the Page token is within 7 days of expiring so you re-mint it before it dies.
9. For ops, gate every restart: node --check the entrypoint and run the bridge unit tests (pre-restart-bridge.sh) or run node scripts/smoke-test.js, and only pm2 restart (never delete) if the gate exits 0.
10. Run pm2 save after adding any process so the dump includes it and it comes back on reboot.

Under the hood

One env file that pm2 can actually read

pm2 launches processes without sourcing ~/.bashrc or ~/.profile, so a key you export-ed in your shell is invisible to a pm2-managed bot. The bridge solves this by reading one flat file at startup, before any service module touches process.env:

const fs   = require("fs");
const os   = require("os");
const path = require("path");

// Load ~/.nemoclaw_env if present (pm2 doesn't source shell profiles)
const envFile = path.join(os.homedir(), ".nemoclaw_env");
if (fs.existsSync(envFile)) {
  for (const line of fs.readFileSync(envFile, "utf8").split("\n")) {
    const m = line.match(/^([A-Z_][A-Z0-9_]*)=(.*)$/);
    if (m) process.env[m[1]] = m[2];
  }
}

The env file is just KEY=value lines, never committed:

DISCORD_BOT_TOKEN=<YOUR_DISCORD_BOT_TOKEN>
NVIDIA_API_KEY=<YOUR_NVIDIA_API_KEY>
IG_USER_ID=<YOUR_IG_BUSINESS_USER_ID>
FB_PAGE_TOKEN=<YOUR_LONG_LIVED_PAGE_TOKEN>
FB_APP_ID=<YOUR_FB_APP_ID>
FB_APP_SECRET=<YOUR_FB_APP_SECRET>
R2_WORKER_URL=https://<your-worker>.workers.dev
R2_UPLOAD_SECRET=<YOUR_R2_UPLOAD_SECRET>
GDRIVE_FOLDER_ID=<YOUR_DRIVE_FOLDER_ID>
GDRIVE_SA_KEY=/home/you/secrets/gdrive-service-account.json

Get the media onto a public URL

Instagram fetches the file itself, so I need a link, not bytes. R2 is the primary host through a tiny Cloudflare Worker that takes a PUT /upload with a secret header and returns { ok, url }:

function uploadToR2(fileBuffer, mimeType) {
  if (!R2_WORKER_URL || !R2_UPLOAD_SECRET) {
    return Promise.reject(new Error("R2 not configured"));
  }
  const url = new URL("/upload", R2_WORKER_URL);
  return new Promise((resolve, reject) => {
    const req = https.request({
      hostname: url.hostname, path: url.pathname, method: "PUT",
      headers: {
        "Content-Type":   mimeType || "application/octet-stream",
        "Content-Length": fileBuffer.length,
        "X-Upload-Secret": R2_UPLOAD_SECRET,
      },
    }, res => {
      let d = ""; res.on("data", c => d += c);
      res.on("end", () => {
        if (res.statusCode !== 200) return reject(new Error(`R2 ${res.statusCode}`));
        const j = JSON.parse(d);
        if (!j.ok || !j.url) return reject(new Error("R2 bad response"));
        resolve(j.url);
      });
    });
    req.on("error", reject);
    req.write(fileBuffer);
    req.end();
  });
}

The fallback uses free anonymous hosts: Catbox Litterbox for video (multipart form, 72h TTL) and Imgur for images (base64 with a Client-ID). The Imgur client id is yours, never hardcode mine:

async function getPublicMediaUrl(fileBuffer, mimeType) {
  if (mimeType?.startsWith("video/")) {
    // Catbox Litterbox: multipart POST to litterbox.catbox.moe with
    // reqtype=fileupload and time=72h; returns a plain-text https:// URL
    /* ... build the multipart body, POST, resolve(text.trim()) ... */
  }
  // Images -> Imgur anonymous upload
  const body = `image=${encodeURIComponent(fileBuffer.toString("base64"))}&type=base64`;
  return new Promise((resolve, reject) => {
    const req = https.request({
      hostname: "api.imgur.com", path: "/3/image", method: "POST",
      headers: {
        "Authorization":  "Client-ID <YOUR_IMGUR_CLIENT_ID>",
        "Content-Type":   "application/x-www-form-urlencoded",
        "Content-Length": Buffer.byteLength(body),
      },
    }, res => {
      let d = ""; res.on("data", c => d += c);
      res.on("end", () => {
        const j = JSON.parse(d);
        if (!j.success) return reject(new Error("Imgur upload failed"));
        resolve(j.data.link);
      });
    });
    req.on("error", reject); req.write(body); req.end();
  });
}

The Instagram container-then-publish dance

Every Graph call is a POST to graph.facebook.com/v21.0 with the Page token in the query. One small helper covers them all:

function graphApiRequest(apiPath, params = {}) {
  return new Promise((resolve, reject) => {
    const qs = new URLSearchParams({ ...params, access_token: FB_PAGE_TOKEN }).toString();
    const req = https.request({
      hostname: "graph.facebook.com",
      path: `/v21.0${apiPath}?${qs}`,
      method: "POST",
    }, res => {
      let d = ""; res.on("data", c => d += c);
      res.on("end", () => { try { resolve(JSON.parse(d)); } catch { resolve({ raw: d }); } });
    });
    req.on("error", reject);
    req.end();
  });
}

In the bridge this lives in one function called postToBuffer: it takes the raw buffer, gets it onto a public URL, then runs the create/poll/publish cycle. Video needs media_type: REELS plus share_to_feed, while a still just needs image_url:

// The real bridge calls this postToBuffer; it uploads the buffer, then posts.
async function postToBuffer({ text, mediaBuffer, mimeType, channels = ["instagram"] }) {
  if (!FB_PAGE_TOKEN) throw new Error("FB_PAGE_TOKEN not set");
  const isVideo = mimeType?.startsWith("video/");

  // 1. Get the bytes onto a public URL: R2 first, Catbox/Imgur fallback
  let mediaUrl;
  try {
    mediaUrl = await uploadToR2(mediaBuffer, mimeType || (isVideo ? "video/mp4" : "image/png"));
  } catch {
    mediaUrl = await getPublicMediaUrl(mediaBuffer, mimeType || (isVideo ? "video/mp4" : "image/png"));
  }
  if (!channels.includes("instagram")) return [{ mediaUrl }];

  // 2. Create the media container
  const containerParams = {
    caption: text || "",
    ...(isVideo
      ? { media_type: "REELS", video_url: mediaUrl, share_to_feed: "true" }
      : { image_url: mediaUrl }),
  };
  const container = await graphApiRequest(`/${IG_USER_ID}/media`, containerParams);
  if (container.error) throw new Error(container.error.message);
  const creationId = container.id;

  // 3. Poll until the container finishes processing (video gets a longer interval)
  for (let i = 0; i < 24; i++) {
    await new Promise(r => setTimeout(r, isVideo ? 5000 : 3000));
    const qs = new URLSearchParams({
      fields: "status_code,status,error_message",
      access_token: FB_PAGE_TOKEN,
    }).toString();
    const status = await new Promise((resolve, reject) => {
      https.get(`https://graph.facebook.com/v21.0/${creationId}?${qs}`, res => {
        let d = ""; res.on("data", c => d += c);
        res.on("end", () => resolve(JSON.parse(d)));
      }).on("error", reject);
    });
    if (status.status_code === "FINISHED") break;
    if (status.status_code === "ERROR")
      throw new Error(`container failed: ${status.error_message || "unknown"}`);
  }

  // 4. Publish
  const publish = await graphApiRequest(`/${IG_USER_ID}/media_publish`, { creation_id: creationId });
  if (publish.error) throw new Error(publish.error.message);
  return publish.id;
}

The real function returns a per-channel results array and catches each channel's error instead of throwing, so one failed post does not take down the others. The shape above is the teaching version.

Watch the Page token expiry

A long-lived Page token lasts about 60 days. The bridge does not auto-rotate it. The function is named refreshPageTokenIfNeeded, but despite the name it only checks debug_token and logs a warning when there are fewer than 7 days left, so I re-mint it by hand before it dies.

// Despite the name, this only warns; it does not re-mint the token.
async function refreshPageTokenIfNeeded() {
  if (!FB_APP_ID || !FB_APP_SECRET || !FB_PAGE_TOKEN) return;
  const qs = new URLSearchParams({
    input_token: FB_PAGE_TOKEN,
    access_token: `${FB_APP_ID}|${FB_APP_SECRET}`,
  }).toString();
  const res = await new Promise((resolve, reject) =>
    https.get(`https://graph.facebook.com/v21.0/debug_token?${qs}`, r => {
      let d = ""; r.on("data", c => d += c); r.on("end", () => resolve(JSON.parse(d)));
    }).on("error", reject));
  const exp = res?.data?.expires_at;
  if (exp && exp > 0) {
    const daysLeft = (exp - Date.now() / 1000) / 86400;
    if (daysLeft < 7) console.warn(`[ig] Page token expires in ${daysLeft.toFixed(1)} days, refresh soon`);
  }
}

Back every render up to Drive

The moment a render comes back, I write it to a temp file and push it to a Drive folder in the background. Anything under ~1KB is skipped as junk.

const MEDIA_FOLDER_ID = process.env.GDRIVE_MEDIA_FOLDER_ID || process.env.GDRIVE_FOLDER_ID || "";
function backupMedia(buf, fileName, mimeType) {
  if (!MEDIA_FOLDER_ID || !buf || buf.length < 1024) return;
  const tmp = `/tmp/gdrive-upload-${Date.now()}-${fileName}`;
  fs.writeFileSync(tmp, buf);
  gdrive.uploadToDrive(tmp, mimeType, fileName, MEDIA_FOLDER_ID)
    .then(r => { console.log(`[gdrive] backed up ${fileName}`); try { fs.unlinkSync(tmp); } catch {} })
    .catch(e => { console.warn(`[gdrive] backup failed: ${e.message}`); try { fs.unlinkSync(tmp); } catch {} });
}

The Drive module prefers a personal OAuth refresh token when one is set, and falls back to a service-account JWT (signed RS256, no user consent screen). It caches the resulting access token for 55 minutes. The folder you upload to must be shared with the service-account email:

// Service-account JWT -> Google token endpoint -> access_token
async function getServiceAccountToken(scopes) {
  const key = JSON.parse(fs.readFileSync(SA_KEY_PATH, "utf8"));
  const now = Math.floor(Date.now() / 1000);
  const header  = base64url(Buffer.from(JSON.stringify({ alg: "RS256", typ: "JWT" })));
  const payload = base64url(Buffer.from(JSON.stringify({
    iss: key.client_email,
    scope: scopes.join(" "),
    aud: "https://oauth2.googleapis.com/token",
    exp: now + 3600, iat: now,
  })));
  const sign = require("crypto").createSign("RSA-SHA256");
  sign.update(`${header}.${payload}`);
  const jwt = `${header}.${payload}.${base64url(sign.sign(key.private_key))}`;
  // POST grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion=<jwt>
  // to oauth2.googleapis.com/token, returns access_token
  /* ... */
}

Share the target folder with the address printed by the setup helper, which looks like <SA_NAME>@<YOUR_PROJECT>.iam.gserviceaccount.com, then set GDRIVE_FOLDER_ID in the env file.

Keep it alive: gate the restart, never delete

Before I bounce the bridge I run a pre-restart gate. The real script syntax-checks the entrypoint and runs the bridge unit tests, and aborts if either fails:

#!/usr/bin/env bash
# pre-restart-bridge.sh
set -euo pipefail
cd "$(dirname "$0")/.."

echo "[pre-restart] syntax check"
node --check scripts/discord-bridge.js || { echo "SYNTAX ERROR, aborting restart"; exit 1; }

echo "[pre-restart] running bridge unit tests"
npx vitest run --project bridge --reporter=dot || { echo "TESTS FAILED, aborting restart"; exit 1; }

echo "All checks passed. Safe to restart."

The deeper gate is scripts/smoke-test.js. It node -c syntax-checks every critical script, require()s each sibling module to catch broken exports, asserts DISCORD_BOT_TOKEN and NVIDIA_API_KEY actually loaded from the env file (those two hard-fail, everything else only warns), and curls a few live endpoints including https://integrate.api.nvidia.com/v1/models and the ComfyUI /queue on the LAN host. After the restart, a tiny smoke-bridge.sh curls the bridge's own /health on port 9341 to confirm it came back and Discord reconnected. The operational rules around all this are short and non-negotiable:

node scripts/smoke-test.js && pm2 restart discord-bridge --update-env
pm2 save        # AFTER adding a process, so the dump includes it on reboot
# NEVER: pm2 delete <name>   (wipes env vars and tokens loaded into the process)

curl https://integrate.api.nvidia.com/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_NVIDIA_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"model":"meta/llama-3.2-90b-vision-instruct","messages":[{"role":"user","content":"hello"}],"max_tokens":64}'

Gotchas & hard-won lessons

• pm2 does not source your shell profile. A key you export-ed in bash is invisible to a pm2 process. Put every secret in one flat env file and parse it yourself on startup.
• Instagram will not accept a raw buffer or a Discord CDN link. It fetches the URL you give it, so you must host the bytes on a publicly reachable URL first.
• The Instagram container is processed asynchronously. If you call media_publish before status_code is FINISHED you get an error. Poll the container until it finishes, and give video a longer interval than images.
• Video posts need media_type=REELS plus video_url and share_to_feed=true. Stills use image_url. Mixing these up returns a confusing container error.
• The Page-token check is named refreshPageTokenIfNeeded but only warns when expiry is under 7 days; it does not refresh the token for you. A long-lived Page token still dies around 60 days, so re-mint it before then or posting goes silent.
• Drive uploads with a service account fail unless the target folder is explicitly shared with the service-account email. The bot can authenticate fine and still see nothing.
• R2 objects expire on a 7-day lifecycle in my setup, which is fine only because Instagram fetches within seconds. Do not rely on those URLs as permanent storage; the Drive copy is the archive.
• Catbox Litterbox links last 72 hours and Imgur anonymous uploads are rate-limited, so treat both as last-resort fallbacks behind R2.
• Never pm2 delete a process. Delete-then-start wipes the env vars and tokens that were loaded into it, and if you forget pm2 save afterward the process vanishes on reboot. Always pm2 restart.
• Run pm2 save only when pm2 list looks healthy. Saving during a half-started state writes a broken dump that resurrect will replay.
• The gdrive-backup cron uses pm2 via an absolute path because cron has no PATH, and it does not silence stderr. The earlier version hid a failure for a day before anyone noticed the backups had stopped.

Prompts to build it yourself

The kind of instructions you'd hand an AI coding agent (Claude Code) to build this from scratch.