Queue fundamentals — why defer work, driver options (database, Redis, SQS)
Concept
Queue fundamentals — understanding why queuing exists and the trade-offs between queue drivers is essential before writing any job.
The problem: Some operations take too long for a synchronous HTTP request. Sending 1000 emails, resizing 20 image variants, calling a slow third-party API, generating a PDF — these block the user's request for seconds. Users shouldn't wait.
The solution: Instead of doing the work immediately, the request writes a "job" (a serialized PHP class) to a fast queue (database table, Redis list, SQS queue) and returns immediately. A separate "worker" process reads from the queue and processes jobs one by one.
Queue drivers and their trade-offs:
| Driver | Speed | Durability | Setup | Best for |
|---|---|---|---|---|
sync | Instant | N/A | None | Development/testing |
database | Slow | High (SQL) | Migrate | Low volume, simple infra |
redis | Fast | Medium (AOF) | Redis | Most production apps |
sqs | Fast | Very high | AWS | AWS-hosted apps, serverless |
beanstalkd | Fast | Medium | Beanstalkd | Dedicated job server |
Durability: The queue must survive crashes. Redis by default is in-memory — data is lost if the server restarts without persistence configured. Enable Redis AOF (append-only file) or use database queue for critical jobs.
Workers: Long-running PHP processes php artisan queue:work. The worker loops: fetch a job, execute it, fetch the next job. Workers must be restarted after code deploys (new code doesn't hot-reload).
dispatchAfterResponse(): A middle ground — defers work until after the HTTP response is sent, within the same PHP process. No worker needed. Limited to short operations (timeout still applies).
Code Example
// .env — choose queue driver
QUEUE_CONNECTION=redis # redis, database, sqs, sync, beanstalkd
// What a job looks like in the database queue
// jobs table (when using database driver):
// id | queue | payload (JSON) | attempts | available_at | created_at
// What a job looks like in Redis
// Redis key: queues:default
// LPUSH queues:default '{"uuid":"...","displayName":"App\\Jobs\\SendEmail","job":"...","maxTries":3,"backoff":null,"timeout":null,"data":{"commandName":"App\\Jobs\\SendEmail","command":"..."}}'
// Worker flow
// 1. php artisan queue:work
// 2. Worker fetches next job from queue (blocks/polls if empty)
// 3. Unserializes job payload back to PHP class
// 4. Calls $job->handle()
// 5. On success: deletes job from queue
// 6. On exception: releases job back to queue (if retries remain) or moves to failed_jobs
// Worker commands
// php artisan queue:work — run default queue
// php artisan queue:work redis — specific connection
// php artisan queue:work --queue=high,default — priority queues
// php artisan queue:work --sleep=3 — sleep 3s between polls if queue empty
// php artisan queue:work --max-jobs=1000 — restart after 1000 jobs (memory leak prevention)
// php artisan queue:work --max-time=3600 — restart after 1 hour
// php artisan queue:listen — restarts worker after EACH job (dev only, slow)
// Always restart workers after deploy!
// php artisan queue:restart — signals all workers to gracefully restart
// In CI/CD pipeline, run this after new code is deployed