durable background jobs

stop running fragile multi-step async work inline in a request. uses the verified inngest module for durable, retryable steps.

intent

take a builder from "this api call sometimes fails and the user sees an error" to durable background work that retries each step independently. the failure mode this prevents: a single try/catch around three external calls, where one transient failure loses all the prior work. use this when you have multi-step async workflows (email + provisioning, webhooks + database updates, etc.) that need to survive process crashes, network timeouts, and partial failures without losing intermediate progress.

inputs

• inngest account and signing key (get at inngest.com/sign-up)
• INNGEST_EVENT_KEY environment variable (event key from inngest dashboard)
• INNGEST_SIGNING_KEY environment variable (signing key from inngest dashboard)
• a serve route mounted for inngest to call back into your app (e.g., POST /api/inngest)
• external apis you're calling (sendgrid for email, stripe for billing, etc.) with their own env keys if needed

edge cases to handle:

• if INNGEST_EVENT_KEY or INNGEST_SIGNING_KEY are missing, inngest client will throw at init time. fail fast, do not proceed.
• if the inngest callback route is not mounted or reachable, jobs will hang indefinitely waiting for a response. test the route is live before deploying.
• rate limits on downstream apis (sendgrid, stripe, etc.) will cause step failures. configure concurrency limits in inngest to avoid slamming them.
• if a step is not idempotent (e.g., charging a card without checking for duplicate), replays will duplicate side effects. idempotency is non-negotiable.

procedure

step 1: define the function with discrete steps

each step is independently retried and its result memoized across replays. one external side effect per step, always.

const { Inngest } = require('inngest');
const inngest = new Inngest({ id: 'app' });

const onSignup = inngest.createFunction(
  { id: 'on-signup', retries: 4 },
  { event: 'user/signup' },
  async ({ event, step }) => {
    await step.run('send-welcome', () => sendEmail(event.data.email));
    await step.run('provision', () => provisionWorkspace(event.data.userId));
  },
);

input: event name (e.g., 'user/signup'), event payload (userId, email), step definitions (send-welcome, provision). output: a durable function object registered with inngest.

step 2: send the event instead of doing the work inline

the request handler just emits and returns fast. inngest runs the steps durably in the background.

app.post('/signup', async (req, res) => {
  const { userId, email } = req.body;
  await inngest.send({ name: 'user/signup', data: { userId, email } });
  res.status(202).json({ status: 'processing' });
});

input: event name, event data (userId, email). output: event enqueued to inngest, request returns 202 accepted immediately.

step 3: mount the inngest serve route

inngest calls back into your app to run the function steps.

import { serve } from 'inngest/express';
const { onSignup } = require('./functions');

app.use('/api/inngest', serve({ client: inngest, functions: [onSignup] }));

input: inngest client, list of functions. output: POST /api/inngest route ready to receive inngest requests.

step 4: tune retries and concurrency

set retries per function and concurrency limits to protect downstream apis from rate limits and cascading failures.

const onSignup = inngest.createFunction(
  {
    id: 'on-signup',
    retries: 4,
    concurrency: {
      key: 'event.data.email',
      limit: 1,  // serialize by email, prevent duplicate signups
    },
  },
  { event: 'user/signup' },
  async ({ event, step }) => {
    // ...
  },
);

input: retry count, concurrency key and limit. output: function with backoff and rate limiting configured.

decision points

• step granularity: one external side effect per step, always. if you have "call api, check response, call another api", split into two steps so the first can be retried without redoing the second.
• idempotency: steps can replay on failure, so make each side effect safe to repeat. charge a card only if you haven't already (check idempotency key). send an email only if it wasn't sent (check sent_at timestamp). if a side effect is not idempotent, wrap it in a lookup (e.g., "has this user already been provisioned?") before executing.
• concurrency: if a downstream api rate-limits at 10 requests/sec, set concurrency.limit to 10 or lower. if you're doing per-user work, use concurrency.key to serialize by user id so you don't swamp one user's workflows.
• if inngest is unreachable (network outage, inngest service down), the event will be retried by your app's inngest client for a few minutes, then dropped. if durability is critical, add a fallback: log the event to a database and replay it manually if inngest goes dark. for non-critical work (analytics, notifications), dropping is fine.
• if a step throws an error, inngest will retry the entire function (from the start) up to retries times. already-completed steps are memoized and skipped, so you don't re-execute them. if you need finer control, use step.waitForEvent to pause and wait for a webhook callback, or step.sleep to delay.

output contract

request handlers emit events and return fast (202 accepted). durable functions are defined with one side effect per step, retries configured per function, and concurrency limits set to protect downstreams. on success, each step is marked done and its result is cached. on failure, inngest retries the function (skipping completed steps) up to the retry limit, then logs a failure event you can hook into for alerting.

data format:

• event payload: json object with userId, email, etc.
• step result: the return value of the side effect (e.g., sendEmail returns a messageId).
• inngest logs: viewable in the inngest dashboard, includes step execution times, retries, and errors.

outcome signal

success means killing the process mid-job (e.g., kill -9 the node process while a step is running), waiting a few seconds, and watching inngest automatically resume the function from the last completed step on replay. if you see the send-welcome step re-executed and the same email sent twice, step memoization is not working (check inngest logs). if a transient failure restarts the whole function and duplicates all side effects, the steps are too coarse (merge them or make them idempotent). if inngest hangs waiting for a callback, the serve route is not reachable (check firewall, logs, and that the route is mounted).