Skip to content

lang/funcs: Conversion functions can handle sensitive values #28446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 19, 2021

Conversation

apparentlymart
Copy link
Contributor

In order to avoid updating every one of our existing functions with explicit support for sensitive values, there's a default rule in the
functions system which makes the result of a function sensitive if any of its arguments contain sensitive values.

We were applying that default to the various type conversion functions, like tomap and tolist, which meant that converting a complex-typed value with a sensitive value anywhere inside it would result in a wholly-sensitive result.

That's unnecessarily conservative because the cty conversion layer (which these functions are wrapping) already knows how to handle sensitivity in a more precise way. Therefore we can opt in to handling marked values (which Terraform uses for sensitivity) here and the only special thing we need to do is handle errors related to sensitive values differently, so we won't print their values out literally in case of an error (and so that the attempt to print them out literally won't panic trying to extract the marked values).


The more-conservative-than-needed sensitivity here was mainly just a cosmetic annoyance, but it has a more practical implication when using tomap or toset to prepare a partially-sensitive collection for use in for_each:

locals {
  sensitive_parts = tomap({
    boop = "hello"
    beep = sensitive("goodbye")
  })
}

resource "null_resource" "example" {
  for_each = local.sensitive_parts
}

Without this PR, the for_each in null_resource.example is invalid because the whole map is marked as sensitive. After this PR it works because the map itself is known and only local.sensitive_parts["beep"] returns an unknown value.

In order to avoid updating every one of our existing functions with
explicit support for sensitive values, there's a default rule in the
functions system which makes the result of a function sensitive if any
of its arguments contain sensitive values.

We were applying that default to the various type conversion functions,
like tomap and tolist, which meant that converting a complex-typed value
with a sensitive value anywhere inside it would result in a
wholly-sensitive result.

That's unnecessarily conservative because the cty conversion layer (which
these functions are wrapping) already knows how to handle sensitivity
in a more precise way. Therefore we can opt in to handling marked values
(which Terraform uses for sensitivity) here and the only special thing
we need to do is handle errors related to sensitive values differently,
so we won't print their values out literally in case of an error (and so
that the attempt to print them out literally won't panic trying to
extract the marked values).
@apparentlymart apparentlymart added bug config 0.15-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged labels Apr 19, 2021
@apparentlymart apparentlymart added this to the v0.15.x milestone Apr 19, 2021
@apparentlymart apparentlymart requested a review from a team April 19, 2021 17:16
@apparentlymart apparentlymart self-assigned this Apr 19, 2021
@codecov
Copy link

codecov bot commented Apr 19, 2021

Codecov Report

Merging #28446 (ea7ae72) into main (85adad0) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted Files Coverage Δ
lang/funcs/conversion.go 90.47% <100.00%> (+0.73%) ⬆️
terraform/node_resource_plan.go 96.11% <0.00%> (-1.95%) ⬇️
backend/remote/backend_common.go 50.51% <0.00%> (+0.69%) ⬆️
internal/providercache/dir.go 73.46% <0.00%> (+6.12%) ⬆️

@apparentlymart
Copy link
Contributor Author

There are more functions like this which would benefit from special treatment of sensitive values in order to produce a more precise result, and so I expect PRs for those will follow soon too. There are opportunities for this for any function that works generically with the elements of a collection, without inspecting those element values directly: if an element gets copied verbatim into the result then the sensitivity of that element should come with it, but we should avoid having the whole collection become sensitive in that case whenever it's reasonable to do that.

@github-actions
Copy link
Contributor

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 20, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
0.15-backport If you add this label to a PR before merging, backport-assistant will open a new PR once merged bug config
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants