Event-Driven Architecture: When the Complexity Actually Pays Off

8 min read

Last updated: May 17, 2026

Event-driven architecture has been the default recommendation in conference talks for so long that most teams reach for Kafka before they have asked whether they need a broker at all. The result is a generation of CRUD applications carrying the operational weight of a distributed log they will never use to its potential. The honest framing is that event-driven is a powerful pattern with a high fixed cost, and the question is whether your workload pays that cost back.

This post is a decision tool. It assumes you understand the basic mechanics of Kafka, NATS, Pulsar, and equivalent systems. What it gives you is a way to tell whether your workload is one of the ones where EDA earns its keep, or one where you have just bought yourself a second on-call rotation.

What Event-Driven Actually Buys You

Strip away the slideware and the real benefits of an event broker between services come down to four things. Producers and consumers decouple in time, so a slow downstream cannot stall an upstream. Multiple consumers can read the same stream independently, which makes fan-out and replay possible. The log itself becomes an audit trail you can rewind. And throughput tends to scale linearly with partitions in a way that point-to-point HTTP rarely does.

Each of these benefits has a counterpart cost. Decoupling in time means you have to design for eventual consistency. Independent consumers mean schema governance becomes a real discipline rather than a per-call concern. The audit trail is only useful if you can replay it without breaking idempotency. And linear throughput requires you to actually run the brokers at scale, which is its own engineering project.

The Real Triggers for EDA

Three signals reliably indicate that an event-driven pattern will pay back its cost. When your workload checks all three, EDA is the right call. When it checks one, you might be better served by a queue. When it checks none, you are looking for a problem to fit a tool.

Three or More Producers Writing to the Same Logical Stream

If a single user action produces an event that ten downstream systems need to react to, point-to-point integration becomes an N-times-M problem within a year. Order placement is the canonical example. The order goes to fulfillment, accounting, fraud, the loyalty program, the recommendation engine, the notification service, the analytics warehouse, the search index, the customer-facing dashboard, and the partner API. Building 10 HTTP integrations works. Building the 11th breaks down. A broker turns this into a single publish and 10 subscriptions.

Asynchronous Workflows With Long Tails

If the work that follows a user action takes more than a few hundred milliseconds, you are already in async territory. Document processing, ML inference pipelines, batch reconciliation, video transcoding, and large-scale notifications all benefit from a broker because the producer should not block on the consumer. The alternative, a synchronous fan-out across multiple HTTP services, creates cascading failure modes that are hard to test and worse to debug at 3am.

Audit Trail or Replay as a First-Class Requiremen

t

If regulators, compliance teams, or product analysts need to reconstruct what happened in your system at a point in time, the log itself becomes the system of record for those events. Financial services, healthcare claims, fraud investigations, and any workflow where state transitions are reviewable months later all benefit from this property. A database with audit triggers can fake it for a while. A real event log scales further and supports replaying events into new consumers without rebuilding history from scratch.

The CRUD Anti-Pattern

The most common misuse of EDA is putting Kafka in front of an internal admin tool that updates a table and emits a webhook. The producer is one service. The consumer is one service. The throughput is fewer than ten events per second. The audit requirement is satisfied by the database itself. There is no replay scenario beyond “check what happened yesterday” which a SELECT covers in 50 milliseconds.

What you have built in that scenario is an HTTP call with extra steps and a new failure mode. The broker can be down. The schema can drift. The consumer can fall behind. None of these problems existed before, and you will spend the next year building tooling to make them visible. The honest design for that workload is a synchronous call, or a database trigger if you must, or at most a managed queue like SQS or Cloud Tasks. Save Kafka for when the shape of the workload actually demands it.

The Operational Cost Nobody Models Up Front

The reason EDA decisions go badly is that the operational cost is invisible during the architecture review and overwhelming during the second year of production. A realistic cost model includes the items below.

• Broker operations: Kafka self-hosted needs a dedicated platform team, ZooKeeper or KRaft expertise, capacity planning, partition rebalancing under load, and an upgrade cadence that touches every consumer. Confluent Cloud, MSK, NATS Cloud, and StreamNative remove the worst of this but introduce a five to seven figure annual line item.
• Schema governance: Avro or Protobuf with a schema registry is not optional past three producers. Without it, every consumer breaks every time a producer adds a field. With it, you have a new system to operate, version, and migrate.
• Idempotency and exactly-once semantics: at-least-once is the default. Building consumers that are safe to replay is a nontrivial discipline that touches every team. Most teams underestimate this and ship duplicate-side-effect bugs for years.
• Replay tooling: the ability to reset offsets and replay a topic into a consumer sounds simple until you discover the consumer cannot tell the difference between a replay and a real event. Solving this requires deduplication keys, idempotency tables, or content-addressed events.
• Observability: tracing across an event boundary breaks most APM tools unless you propagate context manually. End-to-end latency becomes harder to measure. Lag monitoring becomes its own dashboard.
• Disaster recovery: cross-region replication for Kafka is a real engineering project. The default assumption that the broker is durable does not survive the first regional outage.

A reasonable rule of thumb is that the fully loaded cost of running a production-grade Kafka or Pulsar deployment with the surrounding to

oling is at least one senior engineer’s time on a continuing basis. Below that level of investment, you do not have an event-driven platform. You have a single point of failure that nobody understands.

Choosing the Broker

Once you have decided EDA is justified, the broker choice is mostly about operational fit. Kafka remains the default for high-throughput log-based workloads, with Confluent Cloud or MSK removing most of the operational burden at a real cost. NATS JetStream is the right choice for low-latency request-reply and lightweight pub-sub, particularly when you need a footprint that runs comfortably on a single node and grows from there. Pulsar wins when you need true multi-tenancy, geo-replication, and tiered storage out of the box, and lose-out where the operator ecosystem is thinner. Redpanda is a credible Kafka-compatible alternative when you want lower latency and simpler operations on the same client APIs. Cloud-native managed queues like SQS, Cloud Tasks, and Service Bus are not event logs but cover most async workflow needs without the operational tax.

The Migration Path That Actually Works

The teams that succeed with EDA almost never start there. They migrate into it deliberately, one workflow at a time, after a synchronous version has revealed the actual integration shape. The pattern looks like this: build the first version with HTTP calls and a managed queue (SQS, Cloud Tasks, or Service Bus) for any async work. Operate it for at least six months. When you find yourself adding the third or fourth consumer to the same business event, that is the signal to introduce a real broker for that specific stream. Not for the whole system. Not as a platform initiative. For the one workflow that has earned it.

The opposite path, building on Kafka from day one because the architecture deck said so, fails predictably. The team spends the first quarter on broker operations instead of product. The schemas are wrong because the workflows have not stabilized. The consumers ship duplicate-handling bugs because they were written before anyone understood the real failure modes. By the time the platform is mature, half the original use cases have changed and the broker is in the wrong shape for the actual business.

When This Pattern Applies

EDA pays off when you have multiple independent consumers of the same business event, when async workflows dominate the user experience, when audit and replay are first-class requirements, and when you have the operational maturity to run brokers at production grade. Order processing, financial transactions, IoT ingestion, ML feature pipelines, change data capture, and large-scale notification systems are the natural fits.

When It Does Not

EDA does not pay off for internal CRUD tools, for two-service integrations where a synchronous call works, for low-throughput workflows where a managed queue covers the async case, or for teams without the headcount to own broker operations. It does not pay off when the team is still learning distributed systems and would benefit from a year of monolith discipline first. And it does not pay off when the only argument for it is that a vendor or a conference talk said it was best practice. Best practice is whatever survives your post-mortems.