Update pulumi-yaml to recognize the new core config block

[Frassle]

pulumi/pulumi#10832 adds a new core config block to Pulumi.yaml that the engine understands as defining project config. We should update pulumi-yaml to recognize and use that block rather than defining it's own "configuration" block.

---

As YAML is a language host using the Go SDK, it should rely on on the Go SDK config package (GetConfig and IsConfigSecret) to derive configuration keys, and use the structure of the values to infer their types, leaving validation up to the engine. If that's not possible, we should should use shared code from the engine to parse the type and items fields.

As a consequence of using GetConfig, it may not be possible to statically analyze a Pulumi program absent a stack file to determine if the a key is defined. We may want an enhancement to the Go SDK's config package to allow us to query for the existence of a key or for static analysis to rely on the engine's parsing of the project file to be consistent.

As this would be a breaking change for configuration declarations; we should support the old key with a warning to avoid breaking users prior to GA.

[Frassle]

Well pretty much immediately hit an issue that yaml has "secret: true" flags for config variables, and the current MVP doesn't have any support for secrets at project level.

I think we could add "secret" to the core block, but it's a bit odd if we allow default to be set with (because the value can't actually be encrypted in Pulumi.yaml yet). It looks like YAML allows this:

  buzz:
    default: 42
    secret: true

So I guess we would also have to allow this for now? Bit lame but looks like we'll have to.

@AaronFriel any input?

[AaronFriel]

As I think I've shared, I'd like YAML to consume PULUMI_CONFIG and PULUMI_CONFIG_SECRET_KEYS env vars rather than parse the config block (reducing the # of special keys YAML adds to project files) as do other language hosts.

If we support secret: true, with the effect of adding it to PULUMI_CONFIG_SECRET_KEYS, that should do it? And I think it actually makes sense to do so as well:

config:
  # Each stack config must specify this, and it must be secret:
  superSecretPerEnvironmentToken:
    secret: true

I think these would be useful, but not high priority follow up features:

1. Use that project level annotation to warn/error when the stack config is not secret.
2. Use that project level annotation to have the effect of an implicit "--secret" flag on pulumi config set superSecretPerEnvironmentToken

[Frassle]

env vars rather than parse the config block

And just use the implicit typing from the values of those JSON blobs? I thought YAML needed extra type information?

And I think it actually makes sense to do so as well:

Yeh I think it makes sense to add this, I think checking that stack values are secret if it's set and automatically adding "--secret" all make sense, my only question was how we'd want to deal with setting secret and default together. Long term I think that'll make sense as well because we'll have project level secrets, but for now do we just allow it and assume the default value isn't actually that secret?

[Frassle]

I've raised pulumi/pulumi#11084 to add "secret" which ensures that if the stack has that config key that it is an encrypted value.

We're currently not doing any project level support in pulumi config, but I think adding an implicit --secret for these keys will make sense when we do.

[AaronFriel]

We ought to be able to recover the type info by inspecting the structure of the JSON.