AWS Glue CI/CD Blueprint

This is the companion repository for the Streamlining AWS Glue CI/CD — A Comprehensive Blueprint blog post. It contains a reference CI/CD pipeline for building, testing, and deploying data processing jobs in AWS Glue.

Coverage Report

File	Stmts	Miss	Cover	Missing
src
s3_json_to_parquet_job.py	21	1	95%	72
s3_to_s3_job.py	21	1	95%	60
TOTAL	42	2	95%

1. Overview

1.1. Continuous Integration / Continuous Delivery

• GitHub Actions

1.2. Infrastructure as Code

• Terraform

1.3. Quality Assurance

• Python unit tests: unittest + pytest
• Python formatter: Black
• Python linter: Flake8

2. How to contribute

Please make sure to take a moment and read the Code of Conduct.

2.1. Report issues

Please report bugs and suggest features via the GitHub Issues.

Before opening an issue, search the tracker for possible duplicates. If you find a duplicate, please add a comment saying that you encountered the problem as well.

2.2. Contribute code

Please make sure to read the Contributing Guide before making a pull request.