aws kms - only works with storage in S3

[andre-lx]

Describe the bug:

Hello. We are trying to set the storage as a folder/pv in the cluster, and use the s3 unseal config, but till now, we can only achieve this using also the storage as s3 as in the example:

https://github.com/banzaicloud/bank-vaults/blob/ea6352f8f71b4a364ffa6a3ea9fd67cb73ec490c/operator/deploy/cr-aws.yaml#L15-L33

Why can't we use s3 to store the keys and a folder for the remaining vault files?

Expected behaviour:
It should be possible to use s3 only to store the keys

Environment details:

• Kubernetes version: v1.20.11
• Cloud-provider/provisioner: AWS
• bank-vaults version: 1.14.3
• Install method: static manifests
• Logs from the misbehaving component:

$ vault status -ca-cert=/vault/tls/ca.crt -client-cert=/vault/tls/server.crt -client-key=/vault/tls/server.key
Key                      Value
---                      -----
Recovery Seal Type       awskms
Initialized              true
Sealed                   true
Total Recovery Shares    0
Threshold                0
Unseal Progress          0/0
Unseal Nonce             n/a
Version                  1.8.2
Storage Type             file
HA Enabled               false

Logs:
==> Vault server started! Log data will stream in below:

2022-02-11T14:49:28.629Z [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
2022-02-11T14:49:31.902Z [INFO]  core: stored unseal keys supported, attempting fetch
2022-02-11T14:49:31.940Z [INFO]  core.cluster-listener.tcp: starting listener: listener_address=0.0.0.0:8201
2022-02-11T14:49:31.941Z [INFO]  core.cluster-listener: serving cluster requests: cluster_listen_address=[::]:8201
2022-02-11T14:49:31.948Z [INFO]  core: post-unseal setup starting
2022-02-11T14:49:31.969Z [INFO]  core: loaded wrapping token key
2022-02-11T14:49:31.969Z [INFO]  core: successfully setup plugin catalog: plugin-directory=""
2022-02-11T14:49:31.977Z [INFO]  core: successfully mounted backend: type=system path=sys/
2022-02-11T14:49:31.977Z [INFO]  core: successfully mounted backend: type=identity path=identity/
2022-02-11T14:49:31.977Z [INFO]  core: successfully mounted backend: type=cubbyhole path=cubbyhole/
2022-02-11T14:49:32.008Z [INFO]  core: successfully enabled credential backend: type=token path=token/
2022-02-11T14:49:32.010Z [INFO]  rollback: starting rollback manager
2022-02-11T14:49:32.010Z [INFO]  core: restoring leases
2022-02-11T14:49:32.011Z [INFO]  expiration: lease restore complete
2022-02-11T14:49:32.018Z [INFO]  identity: entities restored
2022-02-11T14:49:32.018Z [INFO]  identity: groups restored
2022-02-11T14:49:32.049Z [WARN]  core: post-unseal upgrade seal keys failed: error="no recovery key found"
2022-02-11T14:49:32.049Z [INFO]  core: usage gauge collection is disabled
2022-02-11T14:49:32.063Z [INFO]  core: post-unseal setup complete
2022-02-11T14:49:32.063Z [INFO]  core: vault is unsealed
2022-02-11T14:49:32.063Z [INFO]  core: unsealed with stored key
2022-02-11T14:49:37.721Z [INFO]  core.autoseal: seal configuration missing, not initialized: seal_type=recovery

/kind bug

[github-actions]

Thank you for your contribution! This issue has been automatically marked as stale because it has no recent activity in the last 60 days. It will be closed in 20 days, if no further activity occurs. If this issue is still relevant, please leave a comment to let us know, and the stale label will be automatically removed.