DynamoDB: Indexes & Cost Tradeoffs
Inverted indexes, GSI overloading, sparse indexes, and the real cost tradeoff between a filter expression and an index.
A good primary key answers most of a table's access patterns. This page covers what to do with the ones it can't. See DynamoDB: Data Modeling for single-table design and keys first.
Querying an attribute that isn't part of the key
An attribute outside the key can still be read with a Scan and a filter expression, but a scan reads the whole table and filters after the fact. That's a correctness path, not a performance one.
The efficient path is an index.
- Global Secondary Index (GSI): a different primary key layout over the same data, queryable independently of the base table's key. Can be added any time, even after the table already has data in it.
- Local Secondary Index (LSI): shares the base table's hash key but defines a different range key. Can only be created when the table itself is created, not added later, worth deciding on upfront rather than discovering the need for one after the fact.
Inverted index: the other side of a many-to-many relationship
A table keyed to answer "which employees are on this project" can't also answer "which projects is this employee on" using that same key. A GSI that swaps the table's partition key and sort key answers the reverse question, same underlying data, opposite lookup direction.
ExpandThe base table answers "who's on this project," keyed by project. A GSI with the keys swapped answers "what's this employee on," keyed by employee, same data, opposite direction
DynamoDB keeps the GSI in sync automatically as items change. There's no manual duplication to maintain. One relationship, two lookup directions, because one key was only ever built to answer one of them.
GSI overloading: one index, several entity types
A GSI's own partition and sort key don't have to be the table's key attributes at all. They can be entirely different attributes. One nameIndex GSI, keyed on an organization-scoped partition key plus a generic filterName attribute (ORG#hi, PROJECT#name, EMPLOYEE#name), can answer "find by name" for every entity type through a single index instead of building one per entity.
Sparse index: cheap filtering on a rarely-set attribute
An index only contains items that actually have the attribute it's keyed on. Add an isOnHold GSI, and only the handful of items where that flag is actually set get written into the index, not every item in the table. Querying that GSI returns just the on-hold projects directly, and cheaply, because the index itself stays small.
For search that goes beyond exact-match and prefix lookups, text search, fuzzy matching, the better tool is usually outside DynamoDB entirely: enable Streams (see DynamoDB: Querying, Pagination & Capacity), catch every change with a Lambda function, and index it into a service built for that, like Elasticsearch. Writes still go straight to DynamoDB; only the search-shaped reads go to the search index.
The real cost tradeoff: filter expression vs index
A filter expression looks like it narrows a query, but it runs after DynamoDB has already read every item matching the key condition. Querying 100 items and filtering down to 3 still consumes the read capacity for all 100. The filter reduces what gets returned, not what gets read.
- A secondary index costs its own capacity to maintain, worth it when that access pattern gets queried often.
- A filter expression costs nothing extra to set up but pays the full read cost of the underlying query every time, which can actually be cheaper for a pattern that's queried rarely.
Check the query's actual frequency before assuming an index is automatically the right answer.
The Essentials
- Scan is a correctness path, not a performance one. GSIs can be added any time; LSIs only at table creation.
- Inverted indexes, GSI overloading, and sparse indexes cover the access patterns a good primary key alone can't reach.
- A filter expression still pays full read cost for everything scanned. Weigh index cost against actual query frequency.
Next: using it in code
DynamoDB: Reading & Writing in Code covers the SDK, conditional writes, and update expressions.
Further Reading and Watching
Keep reading