Welcome to terraform-compliance Discussions!

[eerkunt]

👋 Welcome!

We are using Github Discussions in order to provide quick responses on some of the elements that you are curious about. This section can be used for anything really. Please note that we can convert some of the discussions into Issues.

[mdesmarest]

Greetings!

I wanted to provide context to give a detailed write up of how we are trying to deploy terraform compliance and what lead us to your tool. Thanks to your attention to our requests (creating more robust filtering for silent mode, and fixing the "known after apply default values), our potential deployment to our full prod environment is now very close to reality. PLEASE feel free to ask questions for further context or clarity

Currently we are in the middle of deploying Atlantis to automate Terraform deployments via Github Pull Request, and are integrating Terraform-Compliance into Atlantis to run in tandem with it. We are running in an environment where Terraform configurations comprised of a blend of total new deployments, updates to existing configurations, and new additions to existing configurations. The missing piece is being able to filter on what type of resource we are dealing with. Details and possible options at the bottom.

We feel strongly that this deployment pattern represents a common case in many organizations, where the desire to prevent new compliance failures from being deployed while work is done to bring the rest of the infrastructure up to code. Atlantis serves as an engine that Terraform compliance can be used as a supercharger on. In the way that Atlantis would be a web browser and Terraform compliance is an add-on. Atlantis gives power of context to know which configuration to run Terraform-Compliance on, and its ability to run in tandem as seen below, greatly simplifying workflow and failures is very compelling. Also this allows meaningful tracking of configuration and helps ensure clean compliance code gets merged to Github. Since both are open source tools, together they help to create a rough equivalent of HashiCorp Sentinel without the cost.

Workflow Here, feature request at the end!

Full workflow below, apologies for the long read:

Atlantis listens for GitHub, GitLab or Bitbucket webhooks about Terraform pull requests. It then runs terraform plan and comments 
with the output back on the pull request, If the plan file looks proper, a comment on the Pull request of "Atlantis Apply" 
will result in atlantis deploying the infrastructure to AWS. 

We are taking this deployment one step further and running Github protected branches, so that if any checks fail, the branch is not mergeable. Since Terraform code doesn't work under the same principal as normal production code, where a merge to Main/Master represents deployment, Atlantis can block Approval of a PR, and thus not allowing an Atlantis Applyand prevent deployment.

Essentially Atlantis by itself when run as a check outputs a 0, if:

1. terraform code successfully runs a Terraform Plan
   2 code is mergeable

It will output a 1 if either of the above two requirements are not met

When Github protected branches are enabled and checks are enforced, Atlantis output of 1 also blocks Approval, this then blocks the ability toAtlantis Apply which is essentially 'Terraform Apply. With the removal of local Terraform Apply`, we can ensure that only changes made via PR, and recorded to the git log, are those that are deployed. This also ensures that the current terraform code base that is housed within github represents a plannable config.This prevents unintended changes the next time the code is checked out and a record of what has been applied over time. Atlantis writes to the PR twice, once for the plan output and once again for the apply output.

**How Atlantis knows what to do **

Atlantis gets its instructions for how to run via an atlantis.yaml that is located within the repo that is running it at the top lvl, it represents all of the projects that will be run through the Atlantis workflow.

Atlantis has two ways to run an atlantis plan aka terraform plan

1. via the autoplan and the parameters below it, which in our case is anytime a .tf file is modified within the given project
   2., run any time a comment of atlantis plan is commented to the Pull Request

How we "Hack"Terraform-Compliance into the Atlantis workflow

A sample section of a repo lvl atlantis.yaml is listed below. Atlantis also allows for run commands to be run in conjunction with the workflow at any point. Our idea was to install Terraform compliance on the Atlantis host, and have it run each time atlantis plan is run via autoplan or typing atlantis plan

Atlantis produces a planfile each time it runs, we ingress that plan into the terraform-compliance installation on the Atlantis host and use it to run, adding in our features repo and a set of awk output filters to make the output display clean and readable. (This filtering would not be possible without the adjustments to silent mode, Thanks again!)

 - name: tf-cybersecurity/tfcompliance_awk
    dir: tfcompliance_awk
    workspace: default
    autoplan:
      when_modified:
        - "*.tf"
      enabled: true
    apply_requirements: [mergeable, approved]
    workflow: default

workflows:
  default:
    plan:
      steps:
        - init
        - plan
        - run: terraform-compliance -i /root/.ssh/atlantis -f git:ssh://github.com/2uinc/tfcompliance.git//all?ref=main -p $PLANFILE -S --no-ansi | awk '/Feature\:/ || /Module Alert/ || /Fixes/ || /Failure/' | awk -F'#' 'BEGIN{print "***************   TERRAFORM COMPLIANCE SCAN RESULTS   ***************"}; {print$1}'
    apply:
      steps:
        - apply

Running Terraform Compliance this way solves several logistical hurdles and adds simplicity and positive user experience.
See below:

1. Since Terraform-Compliance outputs a 1 on failure and a 0 on pass, by running it on conjunction with atlantis, at the plan step, we influence the check for Atlantis to pass or failed based on Terraform compliance passing or failing. This helps us block approval eligibility , thus preventing terraform apply but not allowing `atlantis apply to be run since the check failed, and you can't approve. This allows us to fail Atlantis for compliance issues even if it planned succesfully.

2.Atlantis enforces terraform state locks, thus making a github action complex since we would need to terraform plan, scan, then release
the lock, and then conditionally trigger atlantis to run. Since Atlantis creates a planfile, we ingress this plan to Terraform-Compliance and bypass any statelocking issues, also Atlantis wipes the environment each run and keeps the sensitive planfile in one place and removed afterward, in the same way a github action would handle this.

2. Using the github action does not allow me to contextually choose what project I want Terraform-Compliance to run on, Atlantis feeds the project directly to terraform compliance via the specific planfile created, no mechanism to decided which project within a multi-project repo to scan.

3. Even with the github action outputting results to the PR comments, you end up with 2 sets of comments. When run in tandem with Atlantis you get one output and one message that has the terraform-compliance results below the plan where they are easy to action upon. Sample of my output below

4. Sequential Runs are easy since each new commit, triggers a new plan and terraform-compliance to run, and within the same window and workflow. Terraform-Compliance operates as part of Atlantis.

SAMPLE OUTPUT REPRESENTED on GITHUB PR from above run command in atlantis.yaml
Leveraging specfic pattern matches in our remote features repo and files, we can provide full path context to the feature, publish quick fixes and link back to full fixes, and alert for potential module use for a given resource. This creates a very easy way to fix failures since the resources are represented in the plan file above.

-GITBOT PR SNIPPET-

***RESOURCES ABOVE REDACTED******
  # module.uswest2.aws_s3_bucket_policy.backup will be created
+ resource "aws_s3_bucket_policy" "backup" {
      + bucket = (known after apply)
      + id     = (known after apply)
      + policy = (known after apply)
    }

Plan: 23 to add, 0 to change, 0 to destroy.
Releasing state lock. This may take a few moments...

***************   TERRAFORM COMPLIANCE SCAN RESULTS   ***************
Feature: Ensure SSH/RDP ingress is not set to 0.0.0.0/0 Full fixes: github.com/2uinc/tfcompliance/all/ingress_all_0s.feature  
    Fixes: Ingress aws_security_group_rule for ssh/rdp should not include all 0s,restrict to specific 2u cidr ranges, see feature for data source examples
          Failure: tcp/22 port is defined within 0.0.0.0/0 network in aws_security_group_rule.ssh_yolo.
          Failure: tcp/3389 port is defined within 0.0.0.0/0 network in aws_security_group_rule.RDP_YOLO.
Feature: s3 buckets must be encrypted. Full Resolution notes at: github.com/2uinc/tfcompliance/all/s3encrypt.feature  
    Fixes: server_side_encryption_configuration must be configured and sse_algorithm=AES256.
    Module Alert: For all new s3 buckets, please use: github.com/2uinc/tf-module-s3-bucket
          Failure: aws_s3_bucket.missing_Server_side_encryption (aws_s3_bucket) does not have server_side_encryption_configuration property.
	  Failure:  aws_s3_bucket.missing_Server_side_encryption_apply_sever_side (aws_s3_bucket) does not have server_side_encryption_configuration property.
          Failure: sse_algorithm property in aws_s3_bucket.server_side_config_yes_not_set_to_aes resource does not match with ^AES256$ case insensitive regex. It is set to aws:kms.

** FEATURE REQUEST TO RUN ONLY ON SPECIFIC RESOURCE CHANGE ACTIONS**

Our last request would be to mimic some of the functionality that HashiCorp Sentinel has and be able to differentiate between resources by Change representation:hashicorp link, via the resource changes block and how it relates to the resources you are loading up for scanning by Terraform compliance via the root module. Below as I am sure you know are the change actions:

Valid actions values are:
  //    ["no-op"]
  //    ["create"]
  //    ["read"]
  //    ["update"]
  //    ["delete", "create"]
  //    ["create", "delete"]
  //    ["delete"]

Many checks that we would like to run can result in odd behaviors and failed checks that should not be fixed at the code level. We would like to run checks on only those resources that are flagged as create because in our deployment, they represent resources that are not yet deployed and code that is not yet merged, so changes can be made and fixed can be processed without regard for unintended consequences.

Without this filtering we need to drastically reduce the number of features we can test against in a Github PR powered by Atlantis above.

Situations that create problematic or confusing failures

• Resources labeled ["no-op"] will fail checks and are not directly involved in the plan, but exist in the state. This creates contextual confusion as to why something failed, since this was deployed previous to our deployment of Terraform-Compliance. This gets more complex when related to the second point below

• Resources labeled ["update"] When failing checks on a resource that is being updated, there is a strong likelihood that it may be non compliant for several features that are not fixable via editing of code and replanning. For example any encryption features for those that we build to ensure EC2 root_block_volume, EBS and RDS would require replacement via terraform. This is obviously not a good idea since these cant be update in state and we would lose the resource as well as experiencing downtime. We would mitigate these types of misconfigs with either a second run with no-fail to advise of these, and not enforce failure. Or with any luck if we can get the action field populated, write conditional feature files against these

POSSIBLE SOLUTION SUGGESTIONS # 1. seems technically easier with the existing loop repurposed, but more complicated with limiting and messing with the deepcopy and haystack # 2. may result in an easier way to use the tool without an additional flag or filter since it just adds a new parameter to the mix

1. I see the functionality for filtering against the resource_change block to manipulate the values as before my feature/bug request, this was used to add in and populate known_after_apply values
   Line #133
   You are already influencing this to NOT write values for resources that are flagged as delete.

Would it be possible to leverage this same type of loop, triggered with an additional output flag for example --create, to run on only newly created infrastructure and rather than filter on if actions != ['delete'] and add values to the resource, instead filter on if actions = ['create'] and remove all resources that aren't create

for finding in self.raw.get('resource_changes', {}):
            resource = deepcopy(finding)
            change = resource.get('change', {})
            actions = change.get('actions', [])
            if actions != ['delete']:
                resource['values'] = change.get('after', {}) # dict_merge(change.get('after', {}), change.get('after_unknown', {}))
                if 'change' in resource:
                    del resource['change']

                if self.is_type(resource, 'data'):
                    self.data[resource['address']] = resource
                else:
                    self.resources[resource['address']] = resource

2. Alternatively, if the change block can be added in to the resource in the same way that name and address are populated above the values block and allows for filtering, we could write features conditionally depending on what the action on that resource was. The action is really the only parameter that matters and is as showed above a subset of absolute values.

"resource_changes": [
    {
      "address": "aws_instance.rapid7",
      "mode": "managed",
      "type": "aws_instnace",
      "name": "rapid7",
      "provider_name": "aws",
      "change": {
        "actions": [
          "no-op" <<-- Here
        ],

Sample Feature leveraging this:

Feature: ec2 root_block_device must be declared and encrypted set to true. 
  Background:Check for the presence of aws ec2 instances not named bastion
    Given I have aws_instance defined
    When its action is create

  Scenario: Root block device not declared
    Then it must have root_block_device

  Scenario: root block device encrypted must be declared
    When it has root_block_device
    Then it must have root_block_device
    And it must have encrypted

  Scenario: root block device encrypted must be declared and set to true
    When it has root_block_device
    Then it must have root_block_device
    When it has encrypted
    Then it must have encrypted
    And its value must be true

Other features can be written for actions of update and no-op that can check for fixes that do not represent resource teardown and rebuilt to fix, and this pass checks. Tagging is a good example.

Also use cases where we are attempting some level of must use module via filtering on name, we want to account for resources that were created previously to our creation of the module, but make sure all new instances created use it.

[mdesmarest]

@eerkunt i see it looks like you folks are quite busy with some of the indexing issues folks are posting. sorry for my delay in response here. Thanks so much for opening the forum. Please let know is there is a version or some fix you need some testing on. We enjoy testing out capability as we scale and automate our pipelines. We get a pretty broad range of configs launched but a wide level of devs at different skill levels. Always looking to tune and enhance. As stated in a number of my feature messages, this is by far the most capable compliance tool we have found for Terraform detection

[eerkunt]

Hi @mdesmarest,

Your feature requests and the details you are providing are immensely helpful to us. Please keep doing it :)

We are delighted to be used within the Atlantis 🎉 Its hectic these days, classic pre-xmas stuff. We will find some time in few weeks and will focus on delivering the functionalities you are looking for. One more week to go :)

[mdesmarest]

@eerkunt First off... WOW, so much work done, really amazed at the progress on the tool. I could hardly keep up. Things look much more stable and far more intuitive. Glad to see some of the features you posted are similar to the thought process we are using to write ours.

Is there any chance you can roadmap the the filtering we were looking for above? As stated, it represents an entirely new way to evaluate and filter resources and prevents flagging compliance issues on existing infrastructure that may result in unintended resource destruction.

Please let us know how we can help, unit tests, examples, etc. Very excited to see where the tool is climbing to.

[mdesmarest]

@eerkunt

We are moving to production on our Atlantis/terraform-compliance/github protected branches workflow!!!!

We like to call it the Ursa Amplexus compliance workflow. It's our "Bear Hug" of safe, strong guidance for AWS and Terraform cubs, new to the dangerous place that Infrastructure and Compliance as code can be.

We are in process of writing our deployment pattern docs for our org. We also wish to share how we are using terraform-compliance with Atlantis and github protected branches to power our multi project repositories for real time compliance checks and fixes.
Thanks to all of the additional features you built for us, we are able to run Terraform compliance within the Atlantis workflow and influence the Atlantis check and output our feature failures above the Terraform plan output. Each time the atlantis plan trigger runs, terraform-compliance runs as well and an organic easy to use interface is handled all within the same gitbot message Atlantis outputs. Terraform-compliance gets treated as part of the Atlantis workflow in an organic way and gets top billing!

In addition with Atlantis, we can have terraform-compliance run on as many projects in a given branch as there are changes.

The power of filtering on terraform change action has allowed us to run checks only on newly configured resources both in new projects and in existing projects at the point where we can influence the most change without causing resource destroying outcomes.

Thanks to the more robust silent mode in terraform-compliance we have created a unique feature style to guide users to a universal fix set for several levels of failure on a given resource without having to run more than once, and account for modules based on our standard module outputs to flag on negative testing and then guide the users to the appropriate module to use. We have figured out ways to organically give meaning to what a failure means in terms of our best practices and how to resolve simply and elegantly.
The way that we can now test for the absence of arguments as well as incorrect declarations in terms of our compliance scheme has taken the notion of negative testing to a much higher level.

Fixes are provided on output and more detailed fixes are available via links map back to our shared features repository. This is all done through a simple bash script based on how we reworked the Feature Description and how we lay out scenarios into a give feature to indicate an overall compliant configuration.

We very much want to share this to the terraform-compliance community first and then to the Atlantis community. To the community here, we can pair your EXCELLENT tool with the decision making process of Atlantis as to what project to run the checks on. This is much more dynamic than a github action as it all gets powered through the Atlantis workflow on one screen, AND there is no need to toggle back and forth through tabs on a PR or customize github action triggers. We also can centralize what features are running on what repository via the repo lvl or Atlantis host lvl yaml, freeing us up from managing individual github actions on our repositories.

When leveraging github protected branches, where all checks must pass to be able to merge, terraform-compliance can influence what Atlantis can do which is,to move Terraform plan/apply into github and verify plannable configs ,and have it evaluate passing not just on the ability to successfully plan but to flag in the following ways and prevent passing, applying and merging:

1. Lack of org compliant configurations both for security( encryption) and for drift detection(optional arguments that are necessary to account for drift)
2. run a set of scenarios in a given feature to test for all of the proper arguments that would indicate the officially recommended module, and provide a singular contextual output for a variety of failures that can all point to " use this module to create a compliant configuration that we can also assist you to updated via versioning"
3. Test against arguments that may Plan successfully, but result in a failure on apply, from resources as simple as s3 bucket name standards, to restricted resources, to configurations that would only be detectable post launch via tools in AWS.

From the standpoint of the junior AWS/Terraform engineer to the high lvl, we have created an organic action plan that feels like it is part of the Atlantis workflow and have received a lot of positive feedback from our team and is a solution we feel would result in a tremendous amount of adoption in our community here and drive some new innovation and ideas based on running terraform-compliance in an active moving multi repo pipeline.

We would obviously like to get this deployment over to the Atlantis boards as well and drive traffic your way, since an active Atlantis deployment can use this build pattern and add all of the awesome capabilities of terraform-compliance to their Atlantis deployment and create a compliance linting engine to lint for configs in the same way that TFlint would lint for formatting.

To say this is saving us massively over buying Sentinel from Hashicorp is just scratching the surface.

This configuration is currently being run on a base feature set to enforce encryption standards on a variety of resources, flagging of all 0's on SSH and RDP via sec group rules and enforcement of use on our s3 bucket module. We are looking to scale this out to all of our repositories with many more features to be built.

Aside from simply telling how awesome your tool is and how proud we are of our integration, we want to ask you, what type of format and where would you like to see us post this and attach it to one of the repos here?

Thus far I have not found anyone else using your awesome tool this way. In the write-ups I have seen where terraform-compliance is compared to some of the other tools out there, like code based scanners, what we have done here really seems to take this over the top and at least for us makes this a no-brainer. We want to assist in the growth and adoption of terraform-compliance to as many orgs like ours and those smaller and larger, and maybe help you hit the big time and when you sell to some big megacorp(bridgecrew got 200 million from Palo Alto!! :smh:, you folks do it better!) or make this an official deployment you don't forget us when you start charging for your services! Maybe a bit of a discount 😉

[eerkunt]

That is great news @mdesmarest! Looking forward for the details when one of your customer starts to use it. Thanks for all the kind words, terraform-compliance was a project where we spend time in our spare-times, but looks like we need to pay more attention from now on :)

I love the name Ursa Amplexus since it also sounds a bit Astronomical (thanks to Ursa), the dots on the terraform-compliance logo also represents Canis Major constellation, hence it sounds like this is going to be the start of a fellowship between a hawk, wolf and a bear.

Aside from simply telling how awesome your tool is and how proud we are of our integration, we want to ask you, what type of format and where would you like to see us post this and attach it to one of the repos here?

Frankly, we don't really have a preference 😄 I like what you guys are doing already, hence we would be delighted if you send us a link when you post the news :)

We were always thinking about how can we create a service around terraform-compliance, but frankly there is agreed plan about it yet. It started from a simple idea, then just becomes a tool trying to solve some compliance-as-code problems against terraform - well, it was the first tool about this by the way, even before Sentinel :). In case, we start to build something, of course Atlantis will have a very special place :)

Thanks again for the kind words! These means a lot for us, since we are trying to create time constantly from our personal lifes in order to maintain this tool.

[eerkunt]

Please let us know how we can help, unit tests, examples, etc. Very excited to see where the tool is climbing to.

Looks like I thought I answered this weeks ago, but not! Would be great to have some support on https://github.com/terraform-compliance/user-friendly-features :)

[sanjaykumar2311]

How do resolve the below error:
❗ ERROR: ["Cannot find step definition for step 'Given I have an AWS EC2 instance' in /home/ec2-user/sanjay/terraform-compliance/validate.feature:4", '', 'Error Oracle says:', "There is no step defintion for 'Given I have an AWS EC2 instance'.", 'All steps should be declared in a module located in /home/ec2-user/precommit/anaconda3/lib/python3.7/site-packages/terraform_compliance/steps.', 'For example you could do:', '', '@step(r"Given I have an AWS EC2 instance")', 'def my_step(step):', ' raise NotImplementedError("This step is not implemented yet")']. Looks like a syntax error.

[eerkunt]

You are having this error because you are missing some of the wording in the feature steps as described here

Try ;

Given I have AWS EC2 instance defined

or better

Given I have aws_instance defined

[doggoiswow]

Is there a way I can test the terraform version that is installed/being used?

Use case - Terraform version is set to "1.0.0" . If the version is accidentally changed and committed, I want to to have a test scenario that will check if the version is still "1.0.0" and not something else

Example-
Scenario: Check if terraform is running on the right versioned
Given I have terraform version defined
Then the terraform_version must be "1.0.0"
Output -
Syntax Error Cannot find step definition for step 'Then the terraform_version must be "1.0.0"

Problem( potential ) - The "terraform_version" is not under the "root_module" but above it
The is what the plan.out.json looks like -
"prior_state": {
"format_version": "0.1",
"terraform_version": "1.0.0",
"values": {
"root_module": {

Is there a scenario statement that i can use to test it?

[doggoiswow]

Terraform-complaince is unable to find aws_msk_cluster resources

Code-
Feature: Test for properties on Kafka
Test for broker logs, broker nodes and encryption
Scenario: kafka must have broker logs enabled
Given I have aws msk cluster resource configured
Then it must have logging_info
Then it must have broker_logs enabled

Output-
Scenario: kafka must have broker logs enabled
💡 SKIPPING: Can not find aws msk cluster defined in target terraform plan.
Given I have aws msk cluster resource configured
Then it must have logging_info
Then it must have broker_logs enabled

plan.out.json file -
aws_msk_cluster is under configuration/root_module