Add support for --unencrypted-regex

[renehernandez]

Description

Fix #538

This will allow to specify a regex so that only keys that match will not be encrypted. Essentially, the reverse of the --encrypted-regex functionality

Changes

• New --unencrypted-regex cli flag
• Config support for new unencrypted_regex field

[codecov-commenter]

Codecov Report

Merging #715 into develop will increase coverage by 0.21%.
The diff coverage is 66.66%.

@@             Coverage Diff             @@
##           develop     #715      +/-   ##
===========================================
+ Coverage    36.44%   36.65%   +0.21%     
===========================================
  Files           22       22              
  Lines         3205     3222      +17     
===========================================
+ Hits          1168     1181      +13     
- Misses        1918     1922       +4     
  Partials       119      119              

Impacted Files	Coverage Δ
keyservice/keyservice.go	0.00% <0.00%> (ø)
keyservice/keyservice.pb.go	4.12% <ø> (ø)
stores/stores.go	0.00% <0.00%> (ø)
config/config.go	71.42% <100.00%> (+0.19%)	⬆️
hcvault/keysource.go	48.12% <100.00%> (ø)
sops.go	57.68% <100.00%> (+1.31%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4bd640e...06af1b2. Read the comment docs.

[renehernandez]

@autrilla Looking forward to feedback on this PR

[autrilla]

LGTM. With this, I think we should deprecate encrypted and unencrypted suffixes and remove them eventually, since this is much more flexible.

Are all the files gofmted? I see some formatting differences in the diff.

[renehernandez]

Are all the files gofmted? I see some formatting differences in the diff.

I'll take a look. I would have expected that go fmt would run as part of the CI process to alert for any issues though

[renehernandez]

LGTM. With this, I think we should deprecate encrypted and unencrypted suffixes and remove them eventually, since this is much more flexible.

I could do the deprecation in a separate MR to avoid polluting this one

[autrilla]

LGTM. With this, I think we should deprecate encrypted and unencrypted suffixes and remove them eventually, since this is much more flexible.

I could do the deprecation in a separate MR to avoid polluting this one

Ah, I didn't mean we had to do it now. It's just an argument in favor of adding this feature -- we add one flag and now we can remove two :)

[autrilla]

Are all the files gofmted? I see some formatting differences in the diff.

I'll take a look. I would have expected that go fmt would run as part of the CI process to alert for any issues though

Yeah, we should get that set up at some point...

[renehernandez]

@autrilla I ran go fmt ./... and it formatted 4 files that I hadn't interacted with:

• hcvault/keysource.go
• keyservice/keyservice.go
• keyservice/keyservice.pb.go
• stores/dotenv/store_test.go

I have added them to the PR

[renehernandez]

@autrilla Anything else to be done here?

[autrilla]

If @ajvb or @jvehent have no objections to adding this, I think we can merge it. Note for them: with this, we can remove --encrypted-{suffix,prefix}.

[renehernandez]

Another thing that I ran today while testing the --unencrypted-regex feature.

Let's say I have .sops.yaml file as follows:

creation_rules:
  - kms: "key1,key2"
    pgp: "fingerprint3"
    unencrypted_regex: '^(name|last_name)$'

And I have a yaml file encrypted using that configuration, e.g:

name: rene
last_name: hernandez
... # rest of encrypted data
sops:
    kms:
       ... # kms section
    pgp:
       ... # pgp section   
    unencrypted_regex: ^(name|last_name)$
    version: 3.6.999

If I then, update the .sops.yaml config to exclude another field, e.g

creation_rules:
  - kms: "key1,key2"
    pgp: "fingerprint3"
    unencrypted_regex: '^(name|last_name|age)$'

When I edit the existing encrypted file and add the age field, it encrypts it instead of honoring the unencrypted_regex field from the .sops.yaml config. It looks like it continues to refer to the unencrypted_regex values stored in the metadata on the encrypted file, which is not updated to use the newer value from the config.

Is this expected? If so, what alternatives do I have to update the metadata on the encrypted files automatically?

[renehernandez]

I found that the affecting code is here. If the file to be edited already exists, we don't pass down the encrypted/unencrypted options.

Any suggestions?

[autrilla]

Is this expected? If so, what alternatives do I have to update the metadata on the encrypted files automatically?

Yes, it's intended. Currently, there are no options to do that, other than recreating the file.