Advanced Features in AWS DynamoDB
DynamoDB Secondary Indices
Refer to Blog Post @ DynamoDB Secondary Indices
DynamoDB TTL
DynamoDB Time to Live allows you to set a per-item time stamp to determine when an item is not needed.
DynamoDB removes the item from the table after the specified date and time without any write throughput.
DynamoDB TTL can be provided at no additional cost and can help reduce storage by retaining only the required data.
Items that are removed from the table are also removed in any local secondary index or global secondary index in exactly the same way as a DeleteItem operation.
DynamoDB Stream records the delete operation as a system deletion and not a regular one.
If the items are no longer relevant after a certain time, TTL can be useful. For example, to remove user or sensor data after one year of inactivity in an app
Archive expired items via DynamoDB Streams or AWS Lambda to an S3 Data Lake
Protect sensitive data for a specified time, according to contractual or regulatory obligations. DynamoDB Cross-region Replication
DynamoDB cross region replication allows identical copies (called replicates) of a DynamoDB master table to be maintained in one or several AWS regions.
All replicas will automatically receive the write to the table.
Cross-region replication currently supports one master mode. A single master can have one master table and one to three replica tables.
DynamoDB acknowledges that a write operation is successful when it is accepted by the master tables. Read replicas are then updated asynchronously. The write will then be sent to each replica with a slight delay.
Cross-region replication is an option for scenariosEfficient disaster Recovery, in the event of a data center crash.
Customers in multiple locations can get faster reads by receiving data faster from AWS by reading a DynamoDB Table from the nearest AWS data center.
Traffic management is easier, as the read workload is distributed across tables and the master table consumes less capacity.
Facilitate regional migration by promoting a replica to master
Live data migration: To replicate data and switch the application to write in the destination region when the tables are in sync
Costing of cross-region replication depends on Provisioned throughput (Writes, Reads).
Storage for replica tables
Data Transfer between Regions
To keep the tables in sync, DynamoDB streams can be used to read data.
Cost of provisioning EC2 instances depends on the type and location of the instances.
NOTE: Cross-region replication on DynamoDB was done by defining AWS Data Pipeline job that used EMR internally to transfer data prior to the DynamoDB streams.
DynamoDB Global Tables, a new multi-master, cross region replication capability from DynamoDB, is designed to support data access locality as well as regional fault tolerance for database workloads.
Applications can now read and write to DynamoDB from AWS regions around world. Changes in any region will be propagated to all regions where a table has been replicated.
Global Tables are useful in building applications that take advantage of data locality to reduce overall latency.
Global Tables ensure eventual consistency.
Global Tables is a replication of data between regions within a single AWS account. It currently does not support cross-account accessibility.
DynamoDB streams are required to enable Global Tables with both New and Old image settings.
DynamoDB Streams allows you to view a time-ordered list of item-level changes to data in a table.
DynamoDB Streams stores data for the last 24hrs, and then they are deleted
DynamoDB Streams maintains an orderly sequence of events per item, but not the sequence across items.
Example: Let’s say you have a DynamoDB Table that tracks high scores in a particular game. Each item represents a player. If you do the following:
