Error on sops_decrypt_file() function : failed: Error getting data key

[ashokpokharel977]

Context
Versions:
terraform: v0.14.4
terragrunt: v0.27.0
sops: 3.6.1 (latest)

Steps:
I have configured SOPS with generator:
.sops.yaml

creation_rules:
  - path_regex: \.dev\.yaml$
    kms: *kms_arn*
    aws_profile: dev

To generate a file

sops secrets.dev.yaml

It generates file with encryption as:

hello: ENC[AES256_GCM,data:8gtnzBNu2AG9l2zHFy3ovCS0gWFj3bdjgb3B/X8CUkvgox8GcxLQv/99aMUndQ==,iv:lw8VYzpWQUrm6bWQgJ6/KEYizhe8VxJAmdysF+Q6zTM=,tag:vRrdCo/iH4ec4dPzI7DB5Q==,type:str]
sops:
    kms:
    -   arn: *kms_arn*
        created_at: '2021-01-12T05:24:17Z'
        enc: *enc_key*
        aws_profile: dev
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    lastmodified: "2021-01-12T05:24:43Z"
    mac: *mac_key*
    pgp: []
    unencrypted_suffix: _unencrypted
    version: 3.6.1

SOPS usage

• Encryption: sops secrets.dev.yaml
• Decryption: sops -d secrets.dev.yaml

Both Encryption and Decryption are working as expected.

Using in terragrunt:

terragrunt.hcl

locals {
  secret_vars = yamldecode(sops_decrypt_file(find_in_parent_folders("secrets.dev.yaml")))
}

Error:

Error: Error in the function call:

Call to function "sops_decrypt_file" failed: Error getting data key: 0 successful groups required, got 0.

Expected Behaviour
Able to decrypt and use it in locals in terragrunt

[brikis98]

Thanks for the bug report. Are you using AWS KMS to encrypt secrets with kops? And if so, how are you authenticating to AWS?

[ashokpokharel977]

@brikis98 Yes I am using AWS KMS for both encryption and decryption. For auth, I am using named profiles with the variable export AWS_SDK_LOAD_CONFIG=1
Is there an option to configure named profile in terragrunt for SOPS

[brikis98]

Hm, it's possible that the current sops implementation isn't properly respecting named profiles... If someone has some time to dig in and figure that out, a PR is very welcome.

[tinder-tder]

you need to set your AWS_PROFILE env to dev it looks like, export AWS_PROFILE=dev then run tg.

[thnee]

Agree that it seems like terragrunts sops implementation is not respecting profiles correctly.

I get this error when the sops file has aws_profile on the keys set to something other than the AWS_PROFILE env var. This issue does not happen when using the actual sops program itself, but it does happen when using terragrunts sops_decrypt_file. Using terragrunt v0.28.7.

This is a legitimate use case, where I am executing all terraform/terragrunt code via an IAM user in a management account and AssumeRole, and my sops files are encrypted using keys that are specific to each concrete account. Then my AWS_PROFILE env var is set to the management account, but the sops files aws_profile points to the concrete accounts.

However, I have worked around this by having the sops files also set the aws_profile to the management account, and set the role field so it also does AssumeRole, just like the terraform/terragrunt code. And this is probably actually a better solution anyway. But still, this issue should be fixed nonetheless.

[ashokpokharel977]

Using export AWS_PROFILE=dev is not an option for me as I am using generate provider.tf

account_vars = read_terragrunt_config(find_in_parent_folders("account.hcl"))
generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "aws" {
  region = "${local.aws_region}"
  shared_credentials_file = "~/.aws/credentials"
  profile                 = "${local.account_name}"
}
EOF
}

I ended up with a workaround setting roles directly inside key groups

creation_rules:
  - path_regex: \.dev\.yaml$
    key_groups:
      - kms:
          - arn: <<kms_arn>>
            role: <<role_arn>>

[michelzanini]

Hi @ashokpokharel977,

Are you using AWS Single Sign-On and your profile is on ~/.aws/config ?
If that's the case then it will be fixed once this is done #1629.

If not, and your are using ~/.aws/credentials then it should be working already.
It has been working with that for quite a while for me.
My code is almost the same as yours...

[mpgeek]

Updating to terragrunt 0.28.22 fixed this issue for me.

[madsonic]

for what it's worth, the config spec is here

i haven't seen any examples/docs/tests with the role spec so it might be a little elusive.

[denis256]

Hi,
the issue should be fixed after upgrading to Terragrunt v0.36.8

https://github.com/gruntwork-io/terragrunt/releases/tag/v0.36.8