Messaging & Event Brokers

A message broker is a middleman post office: systems drop messages off and pick them up, so the sender never needs to know who (or how many) will read them.

At a glance

• What it is: Infrastructure that sits between systems and delivers messages on their behalf. A queue delivers each message to exactly one consumer; a topic (pub/sub — publish/subscribe) broadcasts each message to every subscriber.
• Why it matters: Direct calls require both systems to be available at the same time; a broker decouples them in time — the sender can publish a message even if every consumer is temporarily down.
• For developers: if you’re tempted to call another team’s system directly just to tell it “something happened,” check whether an event on a topic already exists (or should exist) instead — it avoids coupling your service’s uptime to theirs.
• When to use it: Whenever one system needs to notify one or more others that something happened, without waiting for a response.
• When not to use it: When the caller genuinely needs an immediate answer to continue (that’s a request/response case — see API-Led Connectivity).
• Prerequisites: Integration Patterns & Styles.

Mental model

graph LR
    P[Publisher: Warehouse] --> T{{Topic: InventoryAdjusted}}
    T --> S1[Subscriber: Storefront]
    T --> S2[Subscriber: Fulfillment Partner]
    T --> S3[Subscriber: Analytics]

The publisher doesn’t know how many subscribers exist, or whether a new one is added next month — it just publishes to the topic and moves on.

Step-by-step walkthrough

1. Decide whether exactly one consumer should handle each message (queue) or every subscriber should see it (topic/pub-sub).
2. Define the message/event schema and version it explicitly — consumers depend on this contract as much as an API’s; AsyncAPI is the standard specification for documenting this (the async equivalent of OpenAPI for REST), covering channels, message payloads, and bindings for a given broker technology.
3. Choose a broker technology appropriate to the landscape (e.g. a managed queue/topic service, or a log-based event broker for high-throughput streaming).
4. Decide what happens to a message that repeatedly fails to process (see the dead-letter queue pattern in Resilience & Error Handling in Integration).
5. Add monitoring for queue depth / consumer lag so a stuck or slow consumer is visible before it causes a backlog.

Worked example

The warehouse system needs to tell the storefront (to update “in stock” labels) and a new fulfillment partner (to plan shipments) whenever inventory changes — without calling either of them directly.

• Input: a warehouse system that currently has no way to notify other systems except a nightly batch export.
• Process: the warehouse publishes an InventoryAdjusted event to a topic whenever stock changes; the storefront and the fulfillment partner each subscribe independently, with no code change required on the warehouse side to add the fulfillment partner later.
• Output: near-real-time inventory visibility for both consumers, and the warehouse team never needs to know (or care) how many systems are listening. This same event stream can be consumed by a workflow engine for stricter sequencing — see Orchestration vs. Choreography.

// Warehouse: publish once, to an Azure Service Bus topic — subscriber count is irrelevant
await using var client = new ServiceBusClient(connectionString);
ServiceBusSender sender = client.CreateSender("inventory-adjusted");
await sender.SendMessageAsync(new ServiceBusMessage(JsonSerializer.SerializeToUtf8Bytes(evt)));

Storefront and the fulfillment partner each get their own Azure Service Bus subscription on the same topic — adding the fulfillment partner later is a new subscription, not a change to the warehouse’s publishing code.

resource sbNamespace 'Microsoft.ServiceBus/namespaces@2022-10-01-preview' = {
  name: 'sb-inventory'
  location: location
  sku: { name: 'Standard', tier: 'Standard' }
}

resource topic 'Microsoft.ServiceBus/namespaces/topics@2022-10-01-preview' = {
  parent: sbNamespace
  name: 'inventory-adjusted'
  properties: { defaultMessageTimeToLive: 'P14D' }
}

resource storefrontSub 'Microsoft.ServiceBus/namespaces/topics/subscriptions@2022-10-01-preview' = {
  parent: topic
  name: 'storefront'
}

Adding the fulfillment partner later, as described above, is literally just one more subscriptions resource on this topic — no redeploy of the warehouse’s publishing code or the namespace itself.

Common use cases

Use case	Reach for	Notes
One task should be handled by exactly one worker	Queue	Classic task-distribution/load-leveling pattern
Many systems need to know “X happened,” count unknown or growing	Topic / pub-sub	New subscribers can be added without touching the publisher
High-volume event streams needing replay or multiple independent consumer groups	Log-based event broker	Retains events for a window, letting new consumers “catch up” from history

Advanced details

• At-least-once delivery: most brokers guarantee a message is delivered at least once, not exactly once — consumers must handle duplicates (see Data Synchronization & Consistency).
• Consumer groups: log-based brokers let multiple independent consumer groups each read the full event stream at their own pace, useful for adding a new consumer without disrupting existing ones.
• Schema evolution: treat event schemas like public API contracts — add fields additively, and version breaking changes rather than mutating an existing event type in place.

Quick reference

Term	Meaning
Queue	Delivers each message to exactly one consumer
Topic / pub-sub	Broadcasts each message to every subscriber
Broker	The middleman infrastructure that stores and routes messages
Consumer lag	How far behind a consumer is from the latest published message
At-least-once delivery	A broker guarantee that a message will be delivered, possibly more than once
AsyncAPI	The standard specification for documenting event/message contracts — the async equivalent of OpenAPI

Troubleshooting

Symptom	Likely cause	Fix
Same event processed twice causes duplicate side effects	At-least-once delivery, non-idempotent consumer	Make the consumer idempotent (see Data Synchronization & Consistency)
New subscriber needs a publisher-side code change	Direct call disguised as “messaging”	Confirm the publisher is actually publishing to a topic, not calling subscribers directly
Growing backlog / rising consumer lag	Consumer processing slower than publish rate	Scale out consumers, or investigate a slow downstream dependency

Check your understanding

• Why might the warehouse’s InventoryAdjusted event use a topic instead of a queue?
• What has to be true of a consumer for “at-least-once delivery” to be safe in practice?

Related pages

Part of Introduction to Integration Architecture. See also Orchestration vs. Choreography for coordinating multi-step processes over events, Data Synchronization & Consistency for handling duplicates, and the Glossary for term definitions.