Using DynamoDB For Your Next Application

Build powerful cloud-native applications backed by DynamoDB

Lakindu Hewawasam
Enlear Academy
Published in
6 min readJul 11, 2022

--

If you’re a seasoned AWS developer, you’ve deployed quite a few applications on it already. In addition, many of us developers have become accustomed to building applications driven by SQL Databases due to its capability to avoid data redundancy, filter all sorts of data on many columns, and even execute complex queries across multiple tables. Hence, many prefer to utilize Amazon RDS (Amazon Aurora).

But, what many of us fail to factor in is its capability to scale. SQL databases in nature don’t scale well. SQL Databases create extremely resource-intensive operations.

For example:

  1. The SQL Join is an expensive operation that can sometimes cause latencies of up to 25 seconds (depending on your dataset's size).
  2. SQL databases don’t scale well. They can only be scaled vertically. Therefore, you’re bound to hit a performance bottleneck in a production workflow.
  3. If you try horizontal scaling, you’d have to implement complex network operations so that the single SQL query can communicate across a network and retrieve data stored across the array of drives.

These are many reasons why people tend to drive away from SQL-driven databases when building complex and highly active applications.

Enter, DynamoDB

DynamoDB is a fully managed, serverless key-value and document-based NoSQL database that was introduced back in 2012 as a way to handle the millions of requests Amazon receives on its platform per second. It stores your data across an array of high-speed SSDs and offers single-digit millisecond latency. In addition, users can opt to implement the DynamoDB Accelerator (DAX) and obtain a microsecond latency.

The best part of it is that AWS manages everything. All you have to do is start using it.

Apart from that, DynamoDB can scale up and down in demand. This means that even if your application serves ten users today and a million users tomorrow, your application will not experience any performance bottleneck and will still get the single-digit millisecond latency regardless of the user base.

How Does DynamoDB Scale?

Based on the previous section, we’ve understood that DynamoDB is fast. But why is it fast?

Simply put, DynamoDB provides a strictly defined query scheme consisting of only a partition (hash) key and a range (sort) key. Think of this as a primary and secondary key in your SQL table. Unlike SQL, DynamoDB only allows you to query data using these two keys.

Additionally, whenever an item gets persisted on DynamoDB, it passes through a hash function. The hash function determines the partition (in the SSD) to store the data. This way, even if you have 1000TB of data in your application, you’d need to provide the partition key, and DynamoDB would use the hash function to locate the partition your item is stored in and would only query the required partition. Therefore, your 1000TB of data never gets queried at once (unless you implicitly Scan).

Optimizing DynamoDB

If I’ve managed to convince you that DynamoDB is the best way to go for your next application, let us take a look at some ways that you can optimize DynamoDB performance and costs.

Selecting the Billing Plan

Initially, it’s essential to understand that DynamoDB has two billing modes.

  1. Pay Per Request — This is where you do not provision RCUs or WCUs and let DynamoDB scale up and down based on demand. This is recommended if your application has varying workloads.
  2. Provisioned — This is where you provision WCUs and RCUs from the start. Therefore, in the end, you pay for the units you’ve provisioned. This is beneficial if your application has predictable workloads.

If you’re getting started, I highly recommend using the “PAY PER REQUEST” billing model, as it would help you understand how much throughput your application needs.

Designing the Schema

Traditionally, we’ve become accustomed to designing the database and discussing and implementing user stories. This is because SQL offers highly flexible access patterns.

Since DynamoDB offers only a partition/range key for queries, you must try to identify all the access patterns for a database table before designing it. This allows you to select the best-suited keys for your table.

Utilize Intuitive Keys & Secondary Indexes

You may wonder, if there are only two keys, how can you create proper access patterns? Well, Amazon recommends a concept known as the Single Table Design. This is essentially where you use one DynamoDB table to store your application data and use structured partition and range keys to query data.

For example, let’s design a user management system for an organization. Our access patterns would be:

  1. Find all departments in an organization
  2. Find users belonging to a single department
  3. Find information about a single user in an organization.

The ERD for this possible use case is shown below.

Figure — ERD of a user management system for an organization

In the ERD shown above, we can identify the following relationships.

  1. An organization has many departments
  2. A department belongs to one organization
  3. A department has many employees
  4. An employee can belong in one department.

With SQL, this would be three individual tables. However, with DynamoDB, all of this can be modeled in one table. For example:

  1. All records would have a partition key of ORG#ORG_ID . This will help us scope all the records to an organization.
  2. All our employee records would have a sort key of DEPT#DEPT_ID EMPLOYEE#EMPLOYEE_ID. This will query users belonging to an organization, a department, or even query information of a single user.
  3. All our department records would have a sort key of DEPT#DEPT_ID.
  4. All our organization records would have a partition key and sort key of ORG#ORG_ID.

With the help of the “#” in the keys, users are able to construct complex keys for their items and use complex query functions such as — begins_with , > , < , >= , <= or =.

The sample table would look as shown below.

Figure — Designing the Single Table Design

If we observe the 4 points discussed above, we’ve managed to implement a fully-functionality user management system for an organization where we’ve created the access patterns discussed below.

  1. View all departments within an organization
  2. View a single department in an organization
  3. View all users belonging to an organization
  4. View all users belonging to a department
  5. View a single user information

As you can see, with a perfectly modeled table, we can create all the required access patterns to implement our system. The essential advantage is that this table will not experience any bottlenecks and will scale as needed.

Using Indexes

However, if you feel like the combination of the keys isn’t enough, DynamoDB offers both Local & Global Secondary Indexes to provide additional keys to query.

For example, a Local Secondary Index allows you to specify a separate range key for your item. However, you can use a Global Secondary Index to support a different partition and range key for your item! Therefore, try using indexes when working with DynamoDB rather than filter expressions or scans.

What Should I Avoid?

I’ve spoken about Filter Expressions and Scans in the previous paragraph. You must always avoid using Scans or Filter Expressions for your data. A Scan will cause DynamoDB to scan all the partitions of your table to search for data. This is extremely expensive and will create high latency. On the other hand, always avoid Filter Expressions as filters aren’t executed during a query. They are always executed after data has been fetched before it has been returned to you.

Key Takeaways

DynamoDB is a compelling database that will effectively scale up and down and serve millions of customers with no performance loss. However, always remember to plan first before designing, use optimal partition and range keys, and use indexes effectively while avoiding scans and filtering.

Doing so will help you build robust applications with millisecond latency and enhance UX drastically.

Do use DynamoDB in your next project, and let me know how your experience with it was!

Thank you for reading!

--

--

Published in Enlear Academy

We provide high quality content on web development and cloud technologies for developers.

No responses yet

What are your thoughts?