DynamoDB Best practices in PHP

Moe Far — Sat, 25 Apr 2020 09:16:15 +0000

Introduction to DynamoDB

Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale. It's a fully managed, multi-region, Multimaster, durable database with built-in security, backup and restores, and in-memory caching for internet-scale applications. DynamoDB can handle more than 10 trillion requests per day and can support peaks of more than 20 million requests per second.

I've been working with DynamoDB for one year and I really keen to share my experiences, pain points, and best practices I learned.

Best practices

Amazon has prepared SDK for its services. Working with these SDKs reduce lots of complexities and bring tons of advantages. However, it would be great to consider the best practices for each service. For example, DynamoDB works on HTTP protocol and there's no permanent connection(socket or pipeline) with AWS servers, so, it's important to make sure the client has tuned properly. Here are some best practices about DynamoDB client:

Use PHP > 5.5

By default, PHP(the prior versions of 5.5) must load a file from disk, parse the PHP code into opcodes, and finally execute the opcodes. Installing an opcode cache allows the parsed opcodes to be cached in memory so that you do not need to parse the script on every web server request, and in ideal circumstances, these opcodes can be served directly from memory.

Use classmap autoloader

Class map generation essentially converts PSR-4/PSR-0 rules into classmap rules. This makes everything quite a bit faster as for known classes the classmap returns instantly the path, and Composer can guarantee the class is in there so there is no filesystem check needed. You can easily add one of these switches to your composer command to gain this feature:

--optimize-autoloader
-o

Like this:

composer install --optimize-autoloader

Note: You should not enable any of these optimizations in development as they all will cause various problems when adding/removing classes. The performance gains are not worth the trouble in a development setting.

Turn off parameter validation

DynamoDB has No data validation per se and the validation must be handled in the application layer. The good news is AWS SDK for PHP handles the parameter validation. However, it affects performance and we might need to disable this feature in our prod servers. There's a config in Client that handle this feature and you may want to disable it in your live servers:

    new Aws\DynamoDb\DynamoDbClient([
        'region'     => 'us-west-2',
        'validate' => false
    ]);

Note: The default value for this config is true and you might need to leave it at it is in your dev environment to make sure your queries are correct.

Cache AWS credentials

The SDK will attempt to utilize IAM instance profile credentials by contacting the Amazon EC2 instance metadata service (IMDS). Contacting the IMDS requires an HTTP request to retrieve credentials from the IMDS. You can use a provider to cache your credential instance and it will handle refreshing the token per se.

    $provider = CredentialProvider::defaultProvider();
    $provider = CredentialProvider::cache(
        $provider,
        new DoctrineCacheAdapter(new FilesystemCache('/tmp/cache'))
    );

    new Aws\DynamoDb\DynamoDbClient([
        'region'      => 'us-west-2',
        'credentials' => $provider
    ]);

Note: You will need to install Doctrine Cache in order for the SDK to cache credentials using the provider.

     "doctrine/cache": "*"

Debug requests

Sometimes we need to track the requests and make sure our service and connection works fine. There's a config for DynamoDB client in SDK that provides all the metadata for each request and these data not only help us to debug the data flow but also show us the unique request ID that we can follow it with AWS support/tech team. Here is some useful info we can get from this config:

Collecting statistics from lower-level HTTP adapters (e.g., values returned in GuzzleHttpTransferStats). HTTP stats are returned as an indexed array of associative arrays; each associative array contains the transfer stats returned for a request by the client’s HTTP handler.
Reporting on retries attempted. Retry statistics are collected by default and returned.
Reporting the total wall clock time spent on an operation in seconds.

We can enable this config and log them all by enabling it in the client config:

    $client = new  Aws\DynamoDb\DynamoDbClient([
        'region' => 'us-west-2',
        'version' => 'latest',
        'stats' => true
    ]);

You can access the statistics with @metadata attribute in client's result:

    $result = $client->query($params);
    $result['@metadata']; // <--- bunch of useful information

As the metadata looks like:

[
  {
    "statusCode": 200,
    "effectiveUri": "https:\/\/dynamodb.ap-southeast-2.amazonaws.com",
    "headers": {
      "server": "Server",
      "date": "Sat, 25 Apr 2020 09:10:19 GMT",
      "content-type": "application\/x-amz-json-1.0",
      "content-length": "8028",
      "connection": "keep-alive",
      "x-amzn-requestid": "A UNIQUE REQUEST ID", # traceable ID in AWS support panel
      "x-amz-crc32": "-----"
    },
    "transferStats": {
      "http": [
        {
          "total_time": 0.280828,
          "url": "https:\/\/dynamodb.ap-southeast-2.amazonaws.com\/",
          "content_type": "application\/x-amz-json-1.0",
          "http_code": 200,
          "header_size": 256,
          "request_size": 721,
          "filetime": -1,
          "ssl_verify_result": 0,
          "redirect_count": 0,
          "namelookup_time": 0.193096,
          "connect_time": 0.204818,
          "pretransfer_time": 0.264914,
          "size_upload": 144,
          "size_download": 8028,
          "speed_download": 28671,
          "speed_upload": 514,
          "download_content_length": 8028,
          "upload_content_length": 144,
          "starttransfer_time": 0.279284,
          "redirect_time": 0,
          "redirect_url": "",
          "primary_ip": "1.1.1.1",
          "certinfo": [],
          "primary_port": 443,
          "local_ip": "1.1.1.1",
          "appconnect_time": 0.264852
        }
      ],
      "total_time": 0.3063068389892578
    }
  }
]

Forem: Moe Far