An Intro to Amazon DynamoDB Pricing: Challenges and Best Practices | Finout
NoSQL databases were developed to tackle the growing needs of dynamic applications that process both unstructured and semi-structured data. These non-tabular databases store data differently than SQL databases and support a broad range of data formats, including document, key-value, and columnar.
Amazon DynamoDB is an AWS-native managed NoSQL database that scales automatically with changing workloads. While it offers high throughput, low latency, and flexible scalability, it also introduces multiple challenges related to cost management. In this article, we’ll delve into some of these challenges, as well as best practices to follow if you want to achieve optimized pricing.
DynamoDB offers a seamless NoSQL service for microservices, mobile backends, operations, and serverless web apps that require database access with low latency. While the platform provides reliable database services so you can run dynamic applications at scale, it’s important to ensure that the database costs match your projected budget. To control and optimize your DynamoDB costs, your first point of reference should be the classification of read/write operations and the fundamentals of capacity modes.
DynamoDB costs are factored on a number of granular operational metrics. The metrics that primarily influence the overall pricing are read capacity units (RCUs), write capacity units (WCUs), and stream read request units.
You can find additional details about resource consumption metrics and their corresponding prices here.
AWS DynamoDB offers pricing based on two capacity modes: provisioned capacity and on-demand capacity.
Provisioned capacity mode allows you to specify the number of read/write operations your application requires per second, ahead of time. It’s recommended to base the specified provisioned capacity on the maximum traffic capacity forecast. Once provisioned, capacity levels can then be auto-scaled to meet spikes.
This mode is typically suitable for use cases with predictable traffic patterns or if an organization plans to increase traffic gradually. By allowing teams to provision instances by the hour, provisioned capacity mode makes it easy to predict and control costs based on traffic forecasts. For high-performance applications that experience regular spikes, teams can also develop smart schedules to accommodate the expected rise in demand, then scale down to a normal provisioning level after the spike.
With on-demand capacity mode, DynamoDB resources scale automatically when the workloads fluctuate. This mode helps simplify operations, since DynamoDB instantaneously accommodates changes in workloads, without requiring administrators to specify read/write operations.
As a pay-as-you-use model, on-demand capacity is typically suitable for applications with unpredictable traffic or unknown workloads. While this mode enables FinOps teams to keep DynamoDB operational costs in check, it also works for applications with a small number of requests, with a high throughput variation.
AWS DynamoDB is a fully managed NoSQL database system that simplifies data access by abstracting away the complexity of setting up key-value stores. While the solution is simple to interact with, it can be challenging to design and implement functional architectures for large and rapidly scaling applications. This is mostly because scalable architectures introduce challenges related to controlling data access costs.
Here are some of DynamoDB’s inherent challenges, which, if not handled well, can inflate costs substantially.
While DynamoDB is highly scalable, large data sets cause spike scaling, a situation where growing workloads require disproportionately larger resource capacities. As the number of DynamoDB nodes provisioned depends on the data volume, large data volume requires a larger proportion of nodes. Distribution of requested throughput capacity among this higher number of nodes reduces resource efficiency, causes throughput exceptions, and inflates operational costs.
DynamoDB is offered as a fully-managed software-as-a-service (SaaS) model. While this makes it easy to get started with data modeling and integration, the platform does not provide different environments for development, testing, and staging. As a result, you end up paying for usage, even when developing and testing table prototypes.
In the absence of optimally designed partition keys, DynamoDB tables result in hot partitions (partitions that receive more requests than the maximum allowed throughput). In this case, the requested capacity has to be increased so that every partition is provisioned with enough resources needed by the hottest partition. This leads to over-provisioning, which increases DynamoDB costs considerably.
For high-performing applications, the innate latency of AWS DynamoDB may not be low enough. In this case, developers often have to augment the database with a caching solution, such as ElastiCache or DAX, for enhanced performance. This cache tier adds expenses to the already growing costs of storing and accessing data on DynamoDB.
DynamoDB allows local and global secondary indexing for applications that need to query data on attributes not included in the primary key. Though local secondary indexes are free, global indexes require additional provisioning of read/write capacity, resulting in higher operational costs.
Starting at $0.00065 per WCU, applications with numerous writes can incur significant database costs. Additionally, strongly consistent reads cost twice as much as eventually consistent reads. Since RCUs consider the number of bytes for applications that perform database scans, they tend to rapidly become cost prohibitive.
Despite the challenges we just discussed, there are many best practices you can follow to optimize DynamoDB costs. They include:
While RIs allow you to lock down future resource capacity and eliminate overhead in managing sudden consumption spikes, it is crucial to identify the resource capacity consumption before purchasing reserved capacity for DynamoDB.
You should also use tools that allow your team to monitor metrics such as ConsumedReadCapacityUnits and ConsumedWriteCapacityUnits so that FinOps teams can assess DynamoDB usage patterns in real time. This enables the teams to accurately predict how DynamoDB tables consume RCUs and WCUs, so RIs can cover realistic resource budgets.
In addition to monitoring usage projection, you should regularly monitor the expiry date of RIs and the instances they cover to avoid budget overrun.
AWS DynamoDB offers automatic multi-region replication and unlimited throughput, enabling teams to handle rapidly scaling applications easily. While the platform provides single-digit latency and flexible scalability, it also presents several challenges when it comes to cost control. AWS offers reserved instance (RI) plans to limit rising costs, but you’ll still need to spend considerable time and effort to regularly monitor coverage levels, which is often considered an additional challenge.
With Finout, you can monitor your RI coverage on the highest level of granularity (write/read), while assessing the impact of a configuration change on the price of your unit cost.
Click here to reach out to get a full walkthrough of the product
Asaf is the CPO and co-founder of Finout. He has more than 12 years of experience in software engineering, QA and product management at companies like Taboola and Intel. In his last position at Logz.io, he met Roi, and together they decided to embark on the Finout journey. Asaf is a die-hard fan of Manchester United, and Fantasy Football.