Message queues vs event streaming — RabbitMQ vs Kafka context

Advanced5 min read·eng-11-010

interviewcompare

Concept

Message queues vs event streaming — two different paradigms for async communication between services.

Message Queue (RabbitMQ, SQS, Laravel Queue):

Consumer pops the message: Once consumed, it's gone from the queue.
Point-to-point or pub/sub: One consumer gets each message (work queue) OR all subscribers get a copy (fan-out/exchange).
Designed for tasks: "Do this work exactly once." The consumer acknowledges the message — if it crashes before ACK, the broker re-delivers.
Retention: Messages are stored only until consumed. No historical replay.
Use case: Job queues, notifications, email sending, order processing. "I need this work done."

Event Streaming (Apache Kafka, AWS Kinesis):

Consumers read events from a log: Events are NOT removed after consumption. They're appended to a persistent log.
Multiple independent consumers: Each consumer has its own offset (position). Consumer A is at event 1000; Consumer B is at event 5000. Both read the same events independently.
Log retention: Events retained for days/weeks (configurable). New services can replay all historical events from the beginning.
High throughput: Kafka handles millions of events per second. Sequential disk writes are fast.
Use case: Audit logs, event sourcing, multiple services consuming the same event, analytics pipelines, replaying history.

Key difference: In a message queue, a message is consumed and gone. In event streaming, the event is persistent — many consumers read it independently, and new consumers can read from the past.

PHP connection: Laravel's queue system supports database, Redis, SQS (message queues). For Kafka, use php-rdkafka or enqueue/rdkafka. Or produce Kafka events via an HTTP sidecar.

When to choose:

Tasks with exactly-once processing → queue.
Multiple consumers, historical replay, audit trail → event streaming.
Most PHP apps → queue (simpler, already in Laravel).

Code Example

php

<?php
// ============================================================
// MESSAGE QUEUE — Laravel / Redis (work queue pattern)
// ============================================================
// Producer — order service dispatches a job
class OrderController extends Controller
{
    public function store(): \Illuminate\Http\JsonResponse
    {
        $order = Order::create([...]);
        SendOrderConfirmationEmail::dispatch($order); // "do this work once"
        return response()->json($order, 201);
    }
}

// One consumer picks this up and sends the email
// If it fails: retried up to $tries times, then → failed_jobs
// Other jobs in the queue are not affected

// Fan-out with multiple queues (simulated pub/sub in Laravel)
class OrderPlacedEventJob
{
    public function handle(): void
    {
        // Dispatch to multiple specific queues
        SendOrderEmail::dispatch($this->order)->onQueue('notifications');
        UpdateInventory::dispatch($this->order)->onQueue('inventory');
        NotifyAnalytics::dispatch($this->order)->onQueue('analytics');
    }
}

// ============================================================
// EVENT STREAMING — Kafka conceptual PHP example
// ============================================================
// Using php-rdkafka (not Laravel-native, but the concept)

// Producer — append event to Kafka topic
$producer = new \RdKafka\Producer();
$producer->addBrokers('kafka:9092');
$topic = $producer->newTopic('order-events');
$topic->produce(RD_KAFKA_PARTITION_UA, 0, json_encode([
    'type'     => 'OrderPlaced',
    'order_id' => $order->id,
    'at'       => now()->toIso8601String(),
    'data'     => $order->toArray(),
]));
$producer->flush(1000);

// Consumer A — inventory service, starts at its own offset
$consumer = new \RdKafka\KafkaConsumer($conf);
$consumer->subscribe(['order-events']);
while (true) {
    $message = $consumer->consume(10_000);
    $event   = json_decode($message->payload, true);
    if ($event['type'] === 'OrderPlaced') {
        decrementStock($event['data']['items']);
    }
}

// Consumer B — analytics service, reads the SAME events independently
// Both consumers read from the same Kafka topic, each maintaining their own offset
// Consumer A at offset 5000, Consumer B at offset 4800 — both reading independently

// NEW: replay all events from beginning for a new analytics service
$consumer->assign([new \RdKafka\TopicPartition('order-events', 0, RD_KAFKA_OFFSET_BEGINNING)]);
// Processes ALL historical events — impossible with a message queue (they're consumed and gone)

PreviousMonolith to microservices — when it's worth it, how to startSenior Interview Prep — System Design NextHydration & dehydration — mapping raw data into typed objects and backVocabulary — PHP & OOP