Your App Needs Background Jobs. Here's the Simplest Way to Add Them.

A user signs up. Your app creates their account, sends a welcome email, generates a thumbnail of their photo, notifies you on Slack, and adds them to your mailing list — all before showing them the "Welcome!" page. It takes six seconds. If the email provider is slow, it takes thirty. If Slack is down, sign-up fails.

Most of that work didn't need to happen while the user waited. It needed to happen soon. That's what background jobs are for.

Two kinds of work that don't belong in a request

Background jobs — work triggered by something that happened, done shortly afterwards:

Sending emails and notifications
Processing uploads: resizing images, transcoding video, extracting text from PDFs
Calling slow external services, including AI models that take many seconds to respond
Syncing data to other systems
Anything that might fail and should be retried

Scheduled tasks (often called cron jobs) — work that happens on a timetable:

Nightly reports and digest emails
Cleaning up expired sessions, abandoned uploads, old logs
Charging subscriptions, sending renewal reminders
Backups

How background jobs work

The pattern is always the same:

The web request records the work — "send welcome email to user 4812" — in a queue, and responds to the user immediately.
A separate process, the worker, takes jobs from the queue and does them.
If a job fails, it's retried later, usually with increasing delays. If it keeps failing, it's set aside for a human to look at.

The user gets a fast response. A slow or broken external service delays a job instead of breaking sign-up.

The simplest reliable queue: your existing database

Many guides jump straight to a dedicated queue system — Redis-based libraries, or a managed message queue. Those are good tools. But for most small and medium apps, the Postgres database you already have makes an excellent job queue, and it means one less service to run, secure, and back up.

Postgres has a feature built for this (SELECT … FOR UPDATE SKIP LOCKED) that lets several workers take jobs from a table without ever grabbing the same one. You don't need to write it yourself — mature libraries use it for you:

Node.js: pg-boss, Graphile Worker
Python: Procrastinate, and several others
Ruby: GoodJob, Solid Queue (the Rails default)
Elixir: Oban

A bonus you only get with a database queue: you can create a job in the same transaction as the data it's about. If creating the user fails, the welcome-email job is never created either. With a separate queue service, getting that right takes real care.

When to reach for something else: very high job volumes (thousands per second), or when you already run Redis and your framework's standard tool uses it (Sidekiq for Ruby, BullMQ for Node, Celery with Redis for Python). Those are well-trodden paths too.

Scheduled tasks

There are three common ways to run something on a schedule:

Your job library's scheduler. Most of the libraries above can schedule recurring jobs — "every day at 3am" — using the same workers. Usually the best choice: one system, one place to look.
Your host's scheduler. Many platforms offer scheduled jobs or cron triggers that hit a URL or run a command.
Classic cron on a server you control.

Whichever you use, make sure a scheduled task runs once, not once per server. If your app runs on three instances and each has the same cron entry, your nightly email goes out three times.

Where workers run — the serverless catch

A worker is a long-running process that waits for jobs. On a traditional server or container host, you simply run it alongside your web app.

On serverless platforms, where your code only runs in response to requests and each invocation has a time limit, there's no long-running process to host a worker. You either use the platform's own queue and cron products, or an external job service that calls your app over HTTP. That works, but it's a design constraint that surprises people when their app first needs background work. It's one of the trade-offs in VPS vs PaaS: Where Should a Small App Live?.

The mistakes that make jobs fail silently

1. No retries. A network blip means a customer never gets their receipt. Use a library that retries with backoff.

2. Jobs that aren't safe to run twice. Retries mean a job can run more than once — for example, if it succeeds but the worker crashes before recording that. A job that charges a card or sends an email must check whether it already did. (Same principle as webhooks — see How to Add Stripe Payments to an AI-Built App.)

3. Huge jobs. One job that processes all ten thousand users will time out halfway. Have it create ten thousand small jobs instead — or process in batches.

4. Passing whole objects instead of IDs. Put user_id: 4812 in the job, not a copy of the user. By the time the job runs, the data may have changed.

5. Nobody watching the failures. Jobs that exhaust their retries land in a "failed" list. If nobody looks at it, you'll find out from customers. Alert on failed jobs, and put a heartbeat check on scheduled tasks so you know when one didn't run. (How to Know When Your App Is Down covers heartbeats.)

6. Forgetting the worker in production. A classic: the web app is deployed, the worker isn't, and jobs pile up in the queue unprocessed. Make the worker part of your deployment, and check it's running.

Asking an AI agent to add background jobs

A prompt that gets a good result:

Move the welcome email and thumbnail generation out of the sign-up request into background jobs. Use [pg-boss / Graphile Worker / your framework's standard] with our existing Postgres database. Jobs must be safe to retry. Add a worker process to our deployment configuration, and show me how failed jobs are surfaced.

Then check that the worker actually runs where your app is deployed, and try making a job fail on purpose to see what happens.

The checklist

Slow or failure-prone work moved out of web requests
A job library with retries and backoff (Postgres-backed is fine to start)
Jobs safe to run twice
Jobs carry IDs, not copies of data
Scheduled tasks run exactly once, not once per instance
The worker is part of the production deployment
Alerts on failed jobs; heartbeats on scheduled tasks

EasySpawn runs long-lived processes — your web app, workers, and scheduled tasks — side by side in a persistent workspace, with a managed Postgres database ready to serve as the job queue. See how it works or join the waitlist.