Using Amazon Dynamo DB from Node

Submitted by admin on Friday, December 16, 2016 - 20:46.

In my previous article I showed a tutorial on "Uploading data to Amazon S3 from Node.js". This is a good fist step to move your project to a cloud, but there's another important aspect that you have to consider. Your data. Luckily Amazon offers a range of services that supports the needs of many kinds of applications. AWS offers RDS, DynamoDB, Aurora, Redshift and ElastiCache: everything to support data management from fast in-memory caches to petabyte-scale data warehouses. In this post I will show you how to use DynamoDB - a NoSQL database from Amazon. We will create a simple interface to perform CRUD operations over the array of Users. Start from signing in to your Amazon account and go to the DynamoDB control panel. Once you're there, create a new table called Users. You will have to select the hash key and sort key. The first is mandatory while the second one is optional (and allows faster sort queries over users). Since we will not sort users, we'll create just hash key called "id". Notice that DynamoDB is schema-less, much like MongoDB or other document databases. When you create a new table, you don't need to think about the fields that you want to store or their datatypes, you can define and extend your documents later. In this article I assume that you have already set aws-sdk as a dependency and also created aws.config.json file with your API keys. If you haven't, do it now because you will need it to connect to Amazon services. Here's how aws.config.json might look for you:

{
  "accessKeyId": "[your key id]",
  "secretAccessKey": "[your access key]",
  "region": "[your AWS region]"
}

Once everything is set, let's check if we can connect to the database and run a simple query. We'll try to find user by ID. Since there are no users in the database yet, the result will be empty. This is a great way to check if we have the connection to the DB, since if there's any problems with either connectivity or configuration, you will instantly see an error message in a console.

const AWS = require('aws-sdk');
 
AWS.config.loadFromPath('./aws.config.json');
const docClient = new AWS.DynamoDB.DocumentClient();
 
function findUserById(uid) {
  const params = {
    TableName: 'Users',
    Key:{
      id: uid
    }
  };
 
  return new Promise((resolve, reject) => {
    docClient.get(params, (err, data) => {
      if (err) {
        reject(err);
        return;
      }
      resolve(data.Item);
    });
  });
}
 
findUserById('abcd')
	.then((u) => console.log('User is ', u))
	.catch((err) => console.log(err));

In this code I decided to use Promise API. Run this code. If you see message like "User is undefined", great, this code works for you. If you see a stack trace - check your config once again. Now let's look at the code. We created an object called docClient. It is used to access DynamoDB. Under the hood, all this object does is calling REST services with proper parameters. You could use this "lower-level" method too, but it is much more convenient to use ready-made APIs. NOTE: It is important to understand that most AWS services are backed up by REST APIs. This means that even if there's no library for your platform of choice, you can always fall back to plain HTTP calls. We know how to find a user. Let's see how we can add one!

  function createOrUpdateUser(name, pass) {
    const params = {
      TableName: 'Users',
      Item: {
	    id: name,
	    password: pass
	  }
    };
 
    return new Promise((resolve, reject) => {
      docClient.put(params, (err, data) => {
        if (err) {
          reject(err);
          return;
        }
        resolve(params.Item);
      });
    });
  }

One difference that you will instantly spot compared to the "usual" databases: you must take care of ID generation yourself. Most datatabse will take care of it for you, but not DynamoDB. In this example I user user name as ID, but if you want to generate random IDs, here's a good code snippet for you:

  function randomUUID() {
    const pattern = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx';
    return pattern.replace(/[xy]/g, function(c) {
      const r = Math.random()*16|0;
      const v = c == 'x' ? r : (r & 0x3 | 0x8);
      return v.toString(16);
    });
  }

To finish our CRUD methods we need to learn how to delete a user. With DynamoDB it is very simple task:

function deleteUser(id) {
  const params = {
    TableName: 'Users',
    Key: { id },
    ReturnValues: 'ALL_OLD'
  };
 
  return new Promise((resolve, reject) => {
    docClient.delete(params, (err, data) => {
      if (err) {
        reject(err);
        return;
      }
      resolve(data.Attributes);
    });
  });
}

All you need to do is provide a Key of the document that you want to delete and a table name. One more parameter - ReturnValues tells DynamoDB what exactly do you want to receive as the result of DELETE operation. In our case we are returning the last values of the document *before* we deleted it. Now let's perform the whole chain of operations:

createOrUpdateUser('juriy', 'securepass')
  .then((u) => {
    console.log('User Created');
    return createOrUpdateUser('juriy', 'verysecurepass')
  })
  .then((u) => {
    console.log('User Updated!');
    return findUserById('juriy');
  })
  .then((u) => {
    console.log('Now password is', u.pass);
    return deleteUser('juriy');
  })
  .then((u) => {
    console.log('User is deleted');
  })
  .catch((e) => console.log);

So now you know how to perform basic and most important operations with DynamoDB - creating or updating documents (in DynamoDB it is the same operation), finding a document by ID and deleting a document. If you consider moving to DynamoDB make sure you know its limitations. Even though it is extremely powerful in terms of raw performance and scalability, the number of supported operations is very limited. I would suggest to use it for big volumes of immutable data without relations between tables. This is a use case where it truely shines.

Add new comment

28 views

Using Amazon Dynamo DB from Node

Tags

Add new comment