The Number of Requests Isn't the Problem

Why a queue sits between EventBridge and the external RMV check, and how batch size plus reserved concurrency turn a flood into a trickle a third-party system can survive.

August 29, 20265 min read8 / 12

There's a simpler version of the vehicle registration design that skips SQS entirely: EventBridge matches the VehicleRegistered event and invokes a Lambda function directly, and that Lambda calls the Department of Motor Traffic to verify the vehicle. Fewer moving parts, one less service to configure.

It also breaks the moment real traffic shows up, and the reason why is the actual lesson here.

Requests aren't the problem. Concurrent requests are.

A system built to handle a large number of requests over time is a different problem than a system built to handle a large number of requests at the same instant. The first is a matter of throughput. The second is a matter of concurrency, and concurrency is what actually breaks things.

Picture a thousand people submitting vehicle registrations within the same few seconds, entirely plausible for a launch-day spike like the one from the very first post in this series. Wire EventBridge straight to a Lambda function, and that's roughly a thousand Lambda invocations firing in parallel, each one making its own call to the DMT system at the same moment.

Nobody on this project knows how much concurrent load that external system can actually take, and a thousand simultaneous calls is a bad way to find out.

EventBridge wired straight to Lambda fans out to roughly a thousand parallel calls against an external system of unknown capacity. Routing through a queue with capped concurrency turns that into a controlled trickle ExpandEventBridge wired straight to Lambda fans out to roughly a thousand parallel calls against an external system of unknown capacity. Routing through a queue with capped concurrency turns that into a controlled trickle

What a queue actually buys here

A queue between EventBridge and that Lambda function draws a hard line: everything on one side of the queue is under this design's control. Everything on the other side isn't.

SQS absorbs the full burst instantly, a thousand messages land in the queue without breaking a sweat, since that's exactly what a managed queue is built to do. What happens next, how fast those messages get pulled off the queue, is a separate decision, made entirely on this side of the line.

Two dials, tuned to what the other system can take

SQS connects to Lambda through what AWS calls an event source mapping, a piece of configuration, not code, that tells Lambda which queue to poll and how to turn its messages into invocations. Two settings on that connection decide how aggressively messages get processed.

  • Batch size. This controls how many messages one Lambda invocation receives at a time. Set to 1, each invocation handles exactly one registration.
  • Reserved concurrency. This caps how many invocations of that function can run at once, regardless of how many messages are waiting.

Batch size alone doesn't solve the concurrency problem. Even at a batch size of 1, Lambda will happily scale out to dozens of simultaneous single-message invocations if the queue keeps growing, which puts the original problem right back on the table.

Reserved concurrency is the actual lever. Cap it at 5, and no matter how many thousands of messages are waiting in the queue, at most 5 calls to the DMT system happen at once. The rest simply wait their turn. Both numbers, batch size and concurrency cap, get tuned to whatever the external system can actually absorb, not to whatever AWS lets this design scale to.

The tradeoff this creates

Capping concurrency at 5 against a queue of a thousand messages means most of those messages sit "in flight" for a while. Users aren't served instantly.

That's not a flaw introduced by this post. It's the exact reason QR code delivery was made asynchronous all the way back in the requirements post. The design was built around this constraint from the start, not surprised by it later.

The actual user flow reflects that. A click on Register calls API Gateway, which puts the event onto EventBridge and immediately responds. The frontend shows something like "your registration is being processed, you'll get an SMS when it's ready." Nobody's waiting on a spinner for a queue to drain.

What happens after a successful check

Once the rate-limited Lambda gets a positive response from the DMT system, it publishes a new event back onto EventBridge, call it EventMatched, carrying whether the check succeeded. A new event rule listens for that event type and routes it to a Step Functions Express Workflow.

That workflow, and why Step Functions specifically is the right tool for what happens after a match, deserves its own explanation rather than a rushed mention here. What deserves attention right now is the other branch: what happens when the DMT system doesn't answer at all, and how a queue handles a downstream failure without losing a single registration.

The Essentials

  1. Design for concurrent requests, not just total requests. A thousand requests over an hour is easy. A thousand requests in the same second is the actual test.
  2. A queue is a boundary between what you control and what you don't. Tune what's on your side to protect what isn't.
  3. Batch size and reserved concurrency are two separate dials. Batch size alone doesn't cap concurrency, since Lambda will still scale out across many single-message invocations.
  4. Reserved concurrency is what actually throttles calls to a downstream system. Set it to what that system can survive, not to what AWS allows.
  5. A rate-limited queue means slower processing, not immediate delivery. Design the user experience around that from the start, an async acknowledgment, not a synchronous wait.

Further Reading and Watching