dynamodb-bnr

DynamoDB backup'n'restore python script with tarfile management

Description

dynamodb-bnr, or DynamoDB backup'n'restore, is a Python script that allows to backup and restore DynamoDB tables to and from directories or tar archives (with or without compression, supported formats are tar, tar.gz and tar.bz2). The backups can also be automatically sent to S3 buckets.

A Dockerfile is provided to run dynamodb-bnr from a docker container. The dynamodb-bnr docker container can also be obtained from the docker hub using docker pull bhvrops/dynamodb-bnr

Requirements

dynamodb-bnr requirements are currently as follow:

• Python 2.7 or 3.5
• boto3
• pyopenssl

AWS Errors management

When making calls to AWS API, some errors can happen. Here is a list of the errors reported on the AWS API Reference for DynamoDB, and a description of how they are handled by dynamodb-bnr.

ListTables

• InternalServerError: script fails and exit != 0

CreateTable

• InternalServerError: script fails and exit != 0
• LimitExceededException: wait for 15*(current_retry+1) seconds before retrying, max retry = 5 (default values), if not solved after max retry, script fails and exit != 0
• ResourceInUseException: wait for 10*(current_retry+1) seconds before retrying, max retry = 5 (default values), if not solved after max retry, script fails and exit != 0

DeleteTable

• InternalServerError: script fails and exit != 0
• LimitExceededException: wait for 15*(current_retry+1) seconds before retrying, max retry = 5 (default values), if not solved after max retry, script fails and exit != 0
• ResourceInUseException: wait for 10*(current_retry+1) seconds before retrying, max retry = 5 (default values), if not solved after max retry, script fails and exit != 0
• ResourceNotFoundException: already deleted, ignore

UpdateTable

• InternalServerError: script fails and exit != 0
• LimitExceededException: wait for 15*(current_retry+1) seconds before retrying, max retry = 5 (default values), if not solved after max retry, script fails and exit != 0
• ResourceInUseException: wait for 10*(current_retry+1) seconds before retrying, max retry = 5 (default values), if not solved after max retry, script fails and exit != 0
• ResourceNotFoundException: script fails and exit != 0

DescribeTable

• InternalServerError: script fails and exit != 0
• ResourceNotFoundException
  • for backup: should not happen (describe is used to get the table structure on a table that has been found with ListTables ), script fails and exit != 0
  • when creating table: should not happen (describe is used to verify that the table is not anymore in 'CREATING' mode and becomes 'ACTIVE'), script fails and exit != 0
  • when deleting table: should happen (describe is used to verify that the table is not anymore in 'DELETING' mode and becomes actually deleted), ignore

Scan

• InternalServerError: script fails and exit != 0
• ProvisionedThroughputExceededException: wait for 10*(current_retry+1) seconds before retrying, max retry = 5 (default values), if not solved after max retry, script fails and exit != 0
• ResourceNotFoundException: should not happen (scan only tables found with ListTables), script fails and exit != 0

BatchWriteItem

• InternalServerError: script fails and exit != 0
• ItemCollectionSizeLimitExceededException: should not happen as the maximum batch write is set to the current AWS API maximum of 25, script fails and exit != 0
• ProvisionedThroughputExceededException: wait for 10*(current_retry+1) seconds before retrying, max retry = 5 (default values)
• ResourceNotFoundException: should not happen (Table is deleted and created just before running the batch write), script fails and exit != 0