Amazon DynamoDB

A managed NoSQL database with a single-digit millisecond latency guarantee at any scale. DAX (DynamoDB Accelerator) sits in front as an optional read-through cache, for when even single-digit milliseconds isn't fast enough.

That latency guarantee only holds if the table is designed around how the application actually queries data, not designed first and queried however happens to come up later.

Two databases optimizing for different things

Relational databases optimize for storage: normalize the data across tables, eliminate duplicate copies, pay the cost back in joins at query time. That tradeoff made sense when storage was the expensive resource.

DynamoDB optimizes for the opposite: duplicate data across items so a query never needs a join, because compute per request is what actually costs money at scale, not storage. Designing a DynamoDB table like a relational schema is the single most common way newcomers end up with a surprise bill, paying for joins a document database was never built to do cheaply.

Start from the queries, not the entities

The single most important step in modeling a DynamoDB table: list out almost every access pattern the application needs before deciding on a single key or index. Not the entities, the actual queries. Get one item. List all of a parent's children. Filter by status. Whatever the application genuinely does.

Most applications, per AWS's own guidance, only need one table. Every entity type, organizations, projects, employees, whatever a given app has, lives in the same table as differently-shaped items, instead of a separate table per entity, each with its own throughput to provision and pay for.

A repeatable process

1. Draw an entity diagram. Map the real-world entities before touching a key.
2. Identify the relationships between them. One-to-many, many-to-many. A many-to-many is usually worth splitting into two one-to-many relationships through a junction entity (a project-employee item, say), so each side can be queried directly off a key.
3. List every access pattern, per entity. The step worth spending the most time on.
4. Pick the primary key for each entity. A good key alone should satisfy most of the access patterns from step 3, no index required.
5. Add secondary indexes for whatever the primary key still can't answer. Covered below.

One table, told apart by prefixed keys

A single table holding multiple entity types needs a way to tell them apart and to fetch related ones together. The common pattern: prefix key values by entity type, ORG#<id> as a partition key, PROJECT#<id> or EMPLOYEE#<id> as a sort key.

ExpandA query filtering by sort-key prefix returns only project items; dropping that condition returns every item under the same organization

That prefixing does two jobs at once.

• A begins_with condition on the sort key filters to one entity type. Query PK = ORG#1234, SK begins_with PROJECT# and get only that organization's projects. Drop the sort-key condition entirely, and the same query returns every child item, projects and employees together, in one request.
• Child entities inherit the parent's ID as their partition key. A project's partition key is its organization's ID, not its own. That keeps every query naturally scoped to the right organization without an extra filter, and it mirrors the real hierarchy: a project can't exist without an organization, so it doesn't get a partition of its own.

Two shapes for a primary key

• A hash key alone (a simple partition key), when one attribute uniquely identifies an item.
• A hash key plus a range key (a composite key), when items need to be grouped under a partition and then queried within it, sorted by the range key.

Getting this right matters because DynamoDB doesn't let arbitrary attributes be queried efficiently by default, only the key.

Querying an attribute that isn't part of the key

An attribute outside the key can still be read with a Scan and a filter expression, but a scan reads the whole table and filters after the fact. That's a correctness path, not a performance one.

The efficient path is an index.

• Global Secondary Index (GSI): a different primary key layout over the same data, queryable independently of the base table's key. Can be added any time, even after the table already has data in it.
• Local Secondary Index (LSI): shares the base table's hash key but defines a different range key. Can only be created when the table itself is created, not added later, worth deciding on upfront rather than discovering the need for one after the fact.

Inverted index: the other side of a many-to-many relationship

A table keyed to answer "which employees are on this project" can't also answer "which projects is this employee on" using that same key. A GSI that swaps the table's partition key and sort key answers the reverse question, same underlying data, opposite lookup direction.

ExpandThe base table answers "who's on this project," keyed by project. A GSI with the keys swapped answers "what's this employee on," keyed by employee, same data, opposite direction

DynamoDB keeps the GSI in sync automatically as items change, there's no manual duplication to maintain. One relationship, two lookup directions, because one key was only ever built to answer one of them.

GSI overloading: one index, several entity types

A GSI's own partition and sort key don't have to be the table's key attributes at all, they can be entirely different attributes. One nameIndex GSI, keyed on an organization-scoped partition key plus a generic filterName attribute (ORG#hi, PROJECT#name, EMPLOYEE#name), can answer "find by name" for every entity type through a single index instead of building one per entity.

Sparse index: cheap filtering on a rarely-set attribute

An index only contains items that actually have the attribute it's keyed on. Add an isOnHold GSI, and only the handful of items where that flag is actually set get written into the index, not every item in the table. Querying that GSI returns just the on-hold projects directly, and cheaply, because the index itself stays small.

For search that goes beyond exact-match and prefix lookups, text search, fuzzy matching, the better tool is usually outside DynamoDB entirely: enable Streams (below), catch every change with a Lambda function, and index it into a service built for that, like Elasticsearch. Writes still go straight to DynamoDB; only the search-shaped reads go to the search index.

The real cost tradeoff: filter expression vs index

A filter expression looks like it narrows a query, but it runs after DynamoDB has already read every item matching the key condition. Querying 100 items and filtering down to 3 still consumes the read capacity for all 100, the filter reduces what gets returned, not what gets read.

• A secondary index costs its own capacity to maintain, worth it when that access pattern gets queried often.
• A filter expression costs nothing extra to set up but pays the full read cost of the underlying query every time, which can actually be cheaper for a pattern that's queried rarely.

Check the query's actual frequency before assuming an index is automatically the right answer.

Streams

Enabling Streams means every write, insert, update, delete, gets emitted as an event, configurable as key attributes only, the full new item, the full old item, or both. A Lambda function can listen directly, which is what makes DynamoDB a natural event-driven trigger, not just a place data ends up.

Capacity modes

• On-demand: no capacity planning, pay per request. Absorbs a large volume of writes without pre-provisioning, the right default when traffic is spiky or unpredictable.
• Provisioned: set read/write capacity ahead of time, cheaper at steady, predictable volume.

Conditional writes

attribute_not_exists makes an insert idempotent for free: write only if this record doesn't already exist, and a duplicate insert attempt fails cleanly instead of creating a second copy. It only covers the insert case, an action with a side effect (charging a card) needs an explicit idempotency key and lookup table instead. Lambda Powertools ships a utility for that rather than hand-rolling it.

TTL (time to live)

An attribute that tells DynamoDB when to delete an item automatically, no scheduled cleanup job required. Used for a circuit breaker heartbeat (a status: down record with a short TTL, checked before calling an external system) and for idempotency table entries.

Other built-in features

• Encryption at rest, on by default.
• Point-in-time recovery, restore to any point in the retention window.
• Global Tables, multi-region replication, for disaster recovery and multi-region architectures.

The Essentials

1. DynamoDB optimizes for compute, relational databases optimize for storage. Modeling one like the other produces the wrong bill.
2. List access patterns before picking a key. Most applications need exactly one table, with entities told apart by prefixed keys.
3. A begins_with condition and inheriting the parent's ID as partition key are what make a single table queryable per entity type or all together.
4. Scan is a correctness path, not a performance one. GSIs can be added any time; LSIs only at table creation.
5. Inverted indexes, GSI overloading, and sparse indexes cover the access patterns a good primary key alone can't reach.
6. A filter expression still pays full read cost for everything scanned. Weigh index cost against actual query frequency.
7. attribute_not_exists makes inserts idempotent for free; TTL deletes items automatically, useful for heartbeat and idempotency tables alike.

Where this showed up

• The token gets checked first: the user table write.
• Check the fuse before you try the call: the heartbeat table and TTL.
• Same request twice should mean nothing changes: conditional writes and idempotency keys.

Further Reading and Watching

• Video: AWS re:Invent 2021 - DynamoDB deep dive: Advanced design patterns
• Video: AWS re:Invent 2018 - Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401)
• Docs: Best practices for designing and using partition keys effectively (AWS docs)
• Docs: Global secondary indexes (AWS DynamoDB docs)