diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md deleted file mode 100644 index dfa6b8241853..000000000000 --- a/.github/pull_request_template.md +++ /dev/null @@ -1,56 +0,0 @@ - - - - -Fixes # - -## Target Release - - - -1.4.x - -## Draft CHANGELOG entry - - - -### NEW FEATURES | UPGRADE NOTES | ENHANCEMENTS | BUG FIXES | EXPERIMENTS - - - -- diff --git a/CHANGELOG.md b/CHANGELOG.md index 85ecd5ee3c2b..7ab77718e908 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,14 +1,86 @@ -## 1.4.0 (Unreleased) +## 1.3.1 (Unreleased) BUG FIXES: -* The module installer will now record in its manifest a correct module source URL after normalization when the URL given as input contains both a query string portion and a subdirectory portion. Terraform itself doesn't currently make use of this information and so this is just a cosmetic fix to make the recorded metadata more correct. [GH-31636] +* Fixed a crash when using objects with optional attributes and default values in collections, most visible with nested modules. [GH-31847] + +## 1.3.0 (September 21, 2022) + +NEW FEATURES: + +* **Optional attributes for object type constraints:** When declaring an input variable whose type constraint includes an object type, you can now declare individual attributes as optional, and specify a default value to use if the caller doesn't set it. For example: + + ```terraform + variable "with_optional_attribute" { + type = object({ + a = string # a required attribute + b = optional(string) # an optional attribute + c = optional(number, 127) # an optional attribute with a default value + }) + } + ``` + + Assigning `{ a = "foo" }` to this variable will result in the value `{ a = "foo", b = null, c = 127 }`. + +* Added functions: `startswith` and `endswith` allow you to check whether a given string has a specified prefix or suffix. ([#31220](https://github.com/hashicorp/terraform/issues/31220)) + +UPGRADE NOTES: + +* `terraform show -json`: Output changes now include more detail about the unknown-ness of the planned value. Previously, a planned output would be marked as either fully known or partially unknown, with the `after_unknown` field having value `false` or `true` respectively. Now outputs correctly expose the full structure of unknownness for complex values, allowing consumers of the JSON output format to determine which values in a collection are known only after apply. +* `terraform import`: The `-allow-missing-config` has been removed, and at least an empty configuration block must exist to import a resource. +* Consumers of the JSON output format expecting on the `after_unknown` field to be only `false` or `true` should be updated to support [the change representation](https://www.terraform.io/internals/json-format#change-representation) described in the documentation, and as was already used for resource changes. ([#31235](https://github.com/hashicorp/terraform/issues/31235)) +* AzureRM Backend: This release concludes [the deprecation cycle started in Terraform v1.1](https://www.terraform.io/language/upgrade-guides/1-1#preparation-for-removing-azure-ad-graph-support-in-the-azurerm-backend) for the `azurerm` backend's support of "ADAL" authentication. This backend now supports only "MSAL" (Microsoft Graph) authentication. + + This follows from [Microsoft's own deprecation of Azure AD Graph](https://docs.microsoft.com/en-us/graph/migrate-azure-ad-graph-faq), and so you must follow the migration instructions presented in that Azure documentation to adopt Microsoft Graph and then change your backend configuration to use MSAL authentication before upgrading to Terraform v1.3. +* When making requests to HTTPS servers, Terraform will now reject invalid handshakes that have duplicate extensions, as required by RFC 5246 section 7.4.1.4 and RFC 8446 section 4.2. This may cause new errors when interacting with existing buggy or misconfigured TLS servers, but should not affect correct servers. + + This only applies to requests made directly by Terraform CLI, such as provider installation and remote state storage. Terraform providers are separate programs which decide their own policy for handling of TLS handshakes. +* The following backends, which were deprecated in v1.2.3, have now been removed: `artifactory`, `etcd`, `etcdv3`, `manta`, `swift`. The legacy backend name `azure` has also been removed, because the current Azure backend is named `azurerm`. ([#31711](https://github.com/hashicorp/terraform/issues/31711)) + +ENHANCEMENTS: + +* config: Optional attributes for object type constraints, as described under new features above. ([#31154](https://github.com/hashicorp/terraform/issues/31154)) +* config: New built-in function `timecmp` allows determining the ordering relationship between two timestamps while taking potentially-different UTC offsets into account. ([#31687](https://github.com/hashicorp/terraform/pull/31687)) +* config: When reporting an error message related to a function call, Terraform will now include contextual information about the signature of the function that was being called, as an aid to understanding why the call might have failed. ([#31299](https://github.com/hashicorp/terraform/issues/31299)) +* config: When reporting an error or warning message that isn't caused by values being unknown or marked as sensitive, Terraform will no longer mention any values having those characteristics in the contextual information presented alongside the error. Terraform will still return this information for the small subset of error messages that are specifically about unknown values or sensitive values being invalid in certain contexts. ([#31299](https://github.com/hashicorp/terraform/issues/31299)) +* config: `moved` blocks can now describe resources moving to and from modules in separate module packages. ([#31556](https://github.com/hashicorp/terraform/issues/31556)) +* `terraform fmt` now accepts multiple target paths, allowing formatting of several individual files at once. ([#31687](https://github.com/hashicorp/terraform/issues/31687)) +* `terraform init`: provider installation errors now mention which host Terraform was downloading from ([#31524](https://github.com/hashicorp/terraform/issues/31524)) +* CLI: Terraform will report more explicitly when it is proposing to delete an object due to it having moved to a resource instance that is not currently declared in the configuration. ([#31695](https://github.com/hashicorp/terraform/issues/31695)) +* CLI: When showing the progress of a remote operation running in Terraform Cloud, Terraform CLI will include information about pre-plan run tasks ([#31617](https://github.com/hashicorp/terraform/issues/31617)) +* The AzureRM Backend now only supports MSAL (and Microsoft Graph) and no longer makes use of ADAL (and Azure Active Directory Graph) for authentication ([#31070](https://github.com/hashicorp/terraform/issues/31070)) +* The COS backend now supports global acceleration. ([#31425](https://github.com/hashicorp/terraform/issues/31425)) +* provider plugin protocol: The Terraform CLI now calls `PlanResourceChange` for compatible providers when destroying resource instances. ([#31179](https://github.com/hashicorp/terraform/issues/31179)) +* As an implementation detail of the Terraform Cloud integration, Terraform CLI will now capture and upload [the JSON integration format for state](https://www.terraform.io/internals/json-format#state-representation) along with any newly-recorded state snapshots, which then in turn allows Terraform Cloud to provide that information to API-based external integrations. ([#31698](https://github.com/hashicorp/terraform/issues/31698)) + +BUG FIXES: + +* config: Terraform was not previously evaluating preconditions and postconditions during the apply phase for resource instances that didn't have any changes pending, which was incorrect because the outcome of a condition can potentially be affected by changes to _other_ objects in the configuration. Terraform will now always check the conditions for every resource instance included in a plan during the apply phase, even for resource instances that have "no-op" changes. This means that some failures that would previously have been detected only by a subsequent run will now be detected during the same run that caused them, thereby giving the feedback at the appropriate time. ([#31491](https://github.com/hashicorp/terraform/issues/31491)) +* `terraform show -json`: Fixed missing markers for unknown values in the encoding of partially unknown tuples and sets. ([#31236](https://github.com/hashicorp/terraform/issues/31236)) +* `terraform output` CLI help documentation is now more consistent with web-based documentation. ([#29354](https://github.com/hashicorp/terraform/issues/29354)) +* `terraform init`: Error messages now handle the situation where the underlying HTTP client library does not indicate a hostname for a failed request. ([#31542](https://github.com/hashicorp/terraform/issues/31542)) +* `terraform init`: Don't panic if a child module contains a resource with a syntactically-invalid resource type name. ([#31573](https://github.com/hashicorp/terraform/issues/31573)) +* CLI: The representation of destroying already-`null` output values in a destroy plan will no longer report them as being deleted, which avoids reporting the deletion of an output value that was already absent. ([#31471](https://github.com/hashicorp/terraform/issues/31471)) +* `terraform import`: Better handling of resources or modules that use `for_each`, and situations where data resources are needed to complete the operation. ([#31283](https://github.com/hashicorp/terraform/issues/31283)) + +EXPERIMENTS: + +* This release concludes the `module_variable_optional_attrs` experiment, which started in Terraform v0.14.0. The final design of the optional attributes feature is similar to the experimental form in the previous releases, but with two major differences: + * The `optional` function-like modifier for declaring an optional attribute now accepts an optional second argument for specifying a default value to use when the attribute isn't set by the caller. If not specified, the default value is a null value of the appropriate type as before. + * The built-in `defaults` function, previously used to meet the use-case of replacing null values with default values, will not graduate to stable and has been removed. Use the second argument of `optional` inline in your type constraint to declare default values instead. + + If you have any experimental modules that were participating in this experiment, you will need to remove the experiment opt-in and adopt the new syntax for declaring default values in order to migrate your existing module to the stablized version of this feature. If you are writing a shared module for others to use, we recommend declaring that your module requires Terraform v1.3.0 or later to give specific feedback when using the new feature on older Terraform versions, in place of the previous declaration to use the experimental form of this feature: + + ```hcl + terraform { + required_version = ">= 1.3.0" + } + ``` ## Previous Releases For information on prior major and minor releases, see their changelogs: -* [v1.3](https://github.com/hashicorp/terraform/blob/v1.3/CHANGELOG.md) * [v1.2](https://github.com/hashicorp/terraform/blob/v1.2/CHANGELOG.md) * [v1.1](https://github.com/hashicorp/terraform/blob/v1.1/CHANGELOG.md) * [v1.0](https://github.com/hashicorp/terraform/blob/v1.0/CHANGELOG.md) diff --git a/go.mod b/go.mod index a4256fb7329f..cb11deb64543 100644 --- a/go.mod +++ b/go.mod @@ -42,7 +42,7 @@ require ( github.com/hashicorp/go-uuid v1.0.3 github.com/hashicorp/go-version v1.6.0 github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f - github.com/hashicorp/hcl/v2 v2.14.0 + github.com/hashicorp/hcl/v2 v2.14.1 github.com/hashicorp/terraform-config-inspect v0.0.0-20210209133302-4fd17a0faac2 github.com/hashicorp/terraform-registry-address v0.0.0-20220623143253-7d51757b572c github.com/hashicorp/terraform-svchost v0.0.0-20200729002733-f050f53b9734 diff --git a/go.sum b/go.sum index 05b473fce2bb..3059afbcbca7 100644 --- a/go.sum +++ b/go.sum @@ -387,8 +387,8 @@ github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f h1:UdxlrJz4JOnY8W+DbLISwf2B8WXEolNRA8BGCwI9jws= github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f/go.mod h1:oZtUIOe8dh44I2q6ScRibXws4Ajl+d+nod3AaR9vL5w= github.com/hashicorp/hcl/v2 v2.0.0/go.mod h1:oVVDG71tEinNGYCxinCYadcmKU9bglqW9pV3txagJ90= -github.com/hashicorp/hcl/v2 v2.14.0 h1:jX6+Q38Ly9zaAJlAjnFVyeNSNCKKW8D0wvyg7vij5Wc= -github.com/hashicorp/hcl/v2 v2.14.0/go.mod h1:e4z5nxYlWNPdDSNYX+ph14EvWYMFm3eP0zIUqPc2jr0= +github.com/hashicorp/hcl/v2 v2.14.1 h1:x0BpjfZ+CYdbiz+8yZTQ+gdLO7IXvOut7Da+XJayx34= +github.com/hashicorp/hcl/v2 v2.14.1/go.mod h1:e4z5nxYlWNPdDSNYX+ph14EvWYMFm3eP0zIUqPc2jr0= github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d h1:9ARUJJ1VVynB176G1HCwleORqCaXm/Vx0uUi0dL26I0= github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d/go.mod h1:Yog5+CPEM3c99L1CL2CFCYoSzgWm5vTU58idbRUaLik= github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= diff --git a/internal/addrs/module_source.go b/internal/addrs/module_source.go index 82000dbc4181..905b77e2f5c1 100644 --- a/internal/addrs/module_source.go +++ b/internal/addrs/module_source.go @@ -316,17 +316,10 @@ func parseModuleSourceRemote(raw string) (ModuleSourceRemote, error) { func (s ModuleSourceRemote) moduleSource() {} func (s ModuleSourceRemote) String() string { - base := s.Package.String() - if s.Subdir != "" { - // Address contains query string - if strings.Contains(base, "?") { - parts := strings.SplitN(base, "?", 2) - return parts[0] + "//" + s.Subdir + "?" + parts[1] - } - return base + "//" + s.Subdir + return s.Package.String() + "//" + s.Subdir } - return base + return s.Package.String() } func (s ModuleSourceRemote) ForDisplay() string { diff --git a/internal/addrs/module_source_test.go b/internal/addrs/module_source_test.go index 2e38673f3e4a..d6b5626ec682 100644 --- a/internal/addrs/module_source_test.go +++ b/internal/addrs/module_source_test.go @@ -154,13 +154,6 @@ func TestParseModuleSource(t *testing.T) { Subdir: "bleep/bloop", }, }, - "git over HTTPS, URL-style, subdir, query parameters": { - input: "git::https://example.com/code/baz.git//bleep/bloop?otherthing=blah", - want: ModuleSourceRemote{ - Package: ModulePackage("git::https://example.com/code/baz.git?otherthing=blah"), - Subdir: "bleep/bloop", - }, - }, "git over SSH, URL-style": { input: "git::ssh://git@example.com/code/baz.git", want: ModuleSourceRemote{ @@ -408,56 +401,6 @@ func TestModuleSourceRemoteFromRegistry(t *testing.T) { }) } -func TestParseModuleSourceRemote(t *testing.T) { - - tests := map[string]struct { - input string - wantString string - wantForDisplay string - wantErr string - }{ - "git over HTTPS, URL-style, query parameters": { - // Query parameters should be correctly appended after the Package - input: `git::https://example.com/code/baz.git?otherthing=blah`, - wantString: `git::https://example.com/code/baz.git?otherthing=blah`, - wantForDisplay: `git::https://example.com/code/baz.git?otherthing=blah`, - }, - "git over HTTPS, URL-style, subdir, query parameters": { - // Query parameters should be correctly appended after the Package and Subdir - input: `git::https://example.com/code/baz.git//bleep/bloop?otherthing=blah`, - wantString: `git::https://example.com/code/baz.git//bleep/bloop?otherthing=blah`, - wantForDisplay: `git::https://example.com/code/baz.git//bleep/bloop?otherthing=blah`, - }, - } - - for name, test := range tests { - t.Run(name, func(t *testing.T) { - remote, err := parseModuleSourceRemote(test.input) - - if test.wantErr != "" { - switch { - case err == nil: - t.Errorf("unexpected success\nwant error: %s", test.wantErr) - case err.Error() != test.wantErr: - t.Errorf("wrong error messages\ngot: %s\nwant: %s", err.Error(), test.wantErr) - } - return - } - - if err != nil { - t.Fatalf("unexpected error: %s", err.Error()) - } - - if got, want := remote.String(), test.wantString; got != want { - t.Errorf("wrong String() result\ngot: %s\nwant: %s", got, want) - } - if got, want := remote.ForDisplay(), test.wantForDisplay; got != want { - t.Errorf("wrong ForDisplay() result\ngot: %s\nwant: %s", got, want) - } - }) - } -} - func TestParseModuleSourceRegistry(t *testing.T) { // We test parseModuleSourceRegistry alone here, in addition to testing // it indirectly as part of TestParseModuleSource, because general diff --git a/internal/cloud/state.go b/internal/cloud/state.go index 6f1a433b0d2b..04d9f7773439 100644 --- a/internal/cloud/state.go +++ b/internal/cloud/state.go @@ -92,13 +92,19 @@ func (s *State) WriteStateForMigration(f *statefile.File, force bool) error { } } + // The remote backend needs to pass the `force` flag through to its client. + // For backends that support such operations, inform the client + // that a force push has been requested + if force { + s.EnableForcePush() + } + // We create a deep copy of the state here, because the caller also has // a reference to the given object and can potentially go on to mutate // it after we return, but we want the snapshot at this point in time. s.state = f.State.DeepCopy() s.lineage = f.Lineage s.serial = f.Serial - s.forcePush = force return nil } @@ -130,7 +136,6 @@ func (s *State) WriteState(state *states.State) error { // a reference to the given object and can potentially go on to mutate // it after we return, but we want the snapshot at this point in time. s.state = state.DeepCopy() - s.forcePush = false return nil } @@ -409,6 +414,12 @@ func (s *State) Delete() error { return nil } +// EnableForcePush to allow the remote client to overwrite state +// by implementing remote.ClientForcePusher +func (s *State) EnableForcePush() { + s.forcePush = true +} + // GetRootOutputValues fetches output values from Terraform Cloud func (s *State) GetRootOutputValues() (map[string]*states.OutputValue, error) { ctx := context.Background() diff --git a/internal/command/command_test.go b/internal/command/command_test.go index 7998a554224a..fa498b438270 100644 --- a/internal/command/command_test.go +++ b/internal/command/command_test.go @@ -138,6 +138,9 @@ func metaOverridesForProvider(p providers.Interface) *testingOverrides { Providers: map[addrs.Provider]providers.Factory{ addrs.NewDefaultProvider("test"): providers.FactoryFixed(p), addrs.NewProvider(addrs.DefaultProviderRegistryHost, "hashicorp2", "test"): providers.FactoryFixed(p), + addrs.NewLegacyProvider("null"): providers.FactoryFixed(p), + addrs.NewLegacyProvider("azurerm"): providers.FactoryFixed(p), + addrs.NewProvider(addrs.DefaultProviderRegistryHost, "acmecorp", "aws"): providers.FactoryFixed(p), }, } } diff --git a/internal/terraform/context_apply_test.go b/internal/terraform/context_apply_test.go index b36b51c77a0f..4f16453310ee 100644 --- a/internal/terraform/context_apply_test.go +++ b/internal/terraform/context_apply_test.go @@ -12165,6 +12165,70 @@ output "out" { } } +func TestContext2Apply_moduleVariableOptionalAttributesDefaultChild(t *testing.T) { + m := testModuleInline(t, map[string]string{ + "main.tf": ` +variable "in" { + type = list(object({ + a = optional(set(string)) + })) + default = [ + { a = [ "foo" ] }, + { }, + ] +} + +module "child" { + source = "./child" + in = var.in +} + +output "out" { + value = module.child.out +} +`, + "child/main.tf": ` +variable "in" { + type = list(object({ + a = optional(set(string), []) + })) + default = [] +} + +output "out" { + value = var.in +} +`, + }) + + ctx := testContext2(t, &ContextOpts{}) + + // We don't specify a value for the variable here, relying on its defined + // default. + plan, diags := ctx.Plan(m, states.NewState(), SimplePlanOpts(plans.NormalMode, testInputValuesUnset(m.Module.Variables))) + if diags.HasErrors() { + t.Fatal(diags.ErrWithWarnings()) + } + + state, diags := ctx.Apply(plan, m) + if diags.HasErrors() { + t.Fatal(diags.ErrWithWarnings()) + } + + got := state.RootModule().OutputValues["out"].Value + want := cty.ListVal([]cty.Value{ + cty.ObjectVal(map[string]cty.Value{ + "a": cty.SetVal([]cty.Value{cty.StringVal("foo")}), + }), + cty.ObjectVal(map[string]cty.Value{ + "a": cty.SetValEmpty(cty.String), + }), + }) + if !want.RawEquals(got) { + t.Fatalf("wrong result\ngot: %#v\nwant: %#v", got, want) + } +} + func TestContext2Apply_provisionerSensitive(t *testing.T) { m := testModule(t, "apply-provisioner-sensitive") p := testProvider("aws") diff --git a/version/version.go b/version/version.go index 2e90b831e2bb..db12d1ac5c84 100644 --- a/version/version.go +++ b/version/version.go @@ -11,7 +11,7 @@ import ( ) // The main version number that is being run at the moment. -var Version = "1.4.0" +var Version = "1.3.1" // A pre-release marker for the version. If this is "" (empty string) // then it means that it is a final release. Otherwise, this is a pre-release diff --git a/website/data/language-nav-data.json b/website/data/language-nav-data.json index 35f869ea928c..2f068352db60 100644 --- a/website/data/language-nav-data.json +++ b/website/data/language-nav-data.json @@ -1035,7 +1035,7 @@ ] }, { - "title": "Upgrading to Terraform v1.4", + "title": "Upgrading to Terraform v1.3", "path": "upgrade-guides" }, { diff --git a/website/docs/cli/workspaces/index.mdx b/website/docs/cli/workspaces/index.mdx index 7a6b11f9e7c8..a5b84fe65b18 100644 --- a/website/docs/cli/workspaces/index.mdx +++ b/website/docs/cli/workspaces/index.mdx @@ -7,72 +7,80 @@ description: >- # Managing Workspaces -In Terraform CLI, _workspaces_ are separate instances of -[state data](/language/state) that can be used from the same working -directory. You can use workspaces to manage multiple non-overlapping groups of -resources with the same configuration. - -- Every [initialized working directory](/cli/init) has at least - one workspace. (If you haven't created other workspaces, it is a workspace - named `default`.) -- For a given working directory, only one workspace can be _selected_ at a time. -- Most Terraform commands (including [provisioning](/cli/run) - and [state manipulation](/cli/state) commands) only interact - with the currently selected workspace. -- Use [the `terraform workspace select` command](/cli/commands/workspace/select) - to change the currently selected workspace. -- Use the [`terraform workspace list`](/cli/commands/workspace/list), - [`terraform workspace new`](/cli/commands/workspace/new), and - [`terraform workspace delete`](/cli/commands/workspace/delete) commands - to manage the available workspaces in the current working directory. - --> **Note:** Terraform Cloud and Terraform CLI both have features called -"workspaces," but they're slightly different. Terraform Cloud's workspaces -behave more like completely separate working directories. - -## The Purpose of Workspaces - -Since most of the resources you can manage with Terraform don't include a unique -name as part of their configuration, it's common to use the same Terraform -configuration to provision multiple groups of similar resources. - -Terraform relies on [state](/language/state) to associate resources with -real-world objects, so if you run the same configuration multiple times with -completely separate state data, Terraform can manage many non-overlapping groups -of resources. In some cases you'll want to change -[variable values](/language/values/variables) for these different -resource collections (like when specifying differences between staging and -production deployments), and in other cases you might just want many instances -of a particular infrastructure pattern. - -The simplest way to maintain multiple instances of a configuration with -completely separate state data is to use multiple -[working directories](/cli/init) (with different -[backend](/language/settings/backends/configuration) configurations per directory, if you -aren't using the default `local` backend). - -However, this isn't always the most _convenient_ way to handle separate states. -Terraform installs a separate cache of plugins and modules for each working -directory, so maintaining multiple directories can waste bandwidth and disk -space. You must also update your configuration code from version control -separately for each directory, reinitialize each directory separately when -changing the configuration, etc. - -Workspaces allow you to use the same working copy of your configuration and the -same plugin and module caches, while still keeping separate states for each -collection of resources you manage. +Workspaces in the Terraform CLI refer to separate instances of [state data](/language/state) inside the same Terraform working directory. They are distinctly different from [workspaces in Terraform Cloud](/cloud-docs/workspaces), which each have their own Terraform configuration and function as separate working directories. + +Terraform relies on state to associate resources with real-world objects. When you run the same configuration multiple times with separate state data, Terraform can manage multiple sets of non-overlapping resources. + +Workspaces can be helpful for specific [use cases](#use-cases), but they are not required to use the Terraform CLI. We recommend using [alternative approaches](#alternatives-to-workspaces) for complex deployments requiring separate credentials and access controls. + + +## Managing CLI Workspaces + +Every [initialized working directory](/cli/init) starts with one workspace named `default`. + +Use the [`terraform workspace list`](/cli/commands/workspace/list), [`terraform workspace new`](/cli/commands/workspace/new), and [`terraform workspace delete`](/cli/commands/workspace/delete) commands to manage the available workspaces in the current working directory. + +Use [the `terraform workspace select` command](/cli/commands/workspace/select) to change the currently selected workspace. For a given working directory, you can only select one workspace can be at a time. Most Terraform commands only interact with the currently selected workspace. This includes [provisioning](/cli/run) and [state manipulation](/cli/state). + +When you provision infrastructure in each workspace, you usually need to manually specify different [input variables](/language/values/variables) to differentiate each collection. For example, you might deploy test infrastructure to a different region. + + +## Use Cases + +You can create multiple [working directories](/cli/init) to maintain multiple instances of a configuration with completely separate state data. However, Terraform installs a separate cache of plugins and modules for each working directory, so maintaining multiple directories can waste bandwidth and disk space. This approach also requires extra tasks like updating configuration from version control for each directory separately and reinitializing each directory when you change the configuration. Workspaces are convenient because they let you create different sets of infrastructure with the same working copy of your configuration and the same plugin and module caches. + +A common use for multiple workspaces is to create a parallel, distinct copy of +a set of infrastructure to test a set of changes before modifying production infrastructure. + +Non-default workspaces are often related to feature branches in version control. +The default workspace might correspond to the `main` or `trunk` branch, which describes the intended state of production infrastructure. When a developer creates a feature branch for a change, they might also create a corresponding workspace and deploy into it a temporary copy of the main infrastructure. They can then test changes on the copy without affecting the production infrastructure. Once the change is merged and deployed to the default workspace, they destroy the test infrastructure and delete the temporary workspace. + + +### When Not to Use Multiple Workspaces + +Workspaces let you quickly switch between multiple instances of a **single configuration** within its **single backend**. They are not designed to solve all problems. + +When using Terraform to manage larger systems, you should create separate Terraform configurations that correspond to architectural boundaries within the system. This lets teams manage different components separately. Workspaces alone are not a suitable tool for system decomposition because each subsystem should have its own separate configuration and backend. + +In particular, organizations commonly want to create a strong separation +between multiple deployments of the same infrastructure serving different +development stages or different internal teams. In this case, the backend for each deployment often has different credentials and access controls. CLI workspaces within a working directory use the same backend, so they are not a suitable isolation mechanism for this scenario. + +## Alternatives to Workspaces + +Instead of creating CLI workspaces, you can use one or more [re-usable modules](/language/modules/develop) to represent the common elements and then represent each instance as a separate configuration that instantiates those common elements in the context of a different [backend](/language/settings/backends/configuration). The root module of each configuration consists only of a backend configuration and a small number of `module` blocks with arguments describing any small differences between the deployments. + +When multiple configurations represent distinct system components rather than multiple deployments, you can pass data from one component to another using paired resources types and data sources. + +- When a shared [Consul](https://www.consul.io/) cluster is available, use [`consul_key_prefix`](https://registry.terraform.io/providers/hashicorp/consul/latest/docs/resources/key_prefix) to publish to the key/value store and [`consul_keys`](https://registry.terraform.io/providers/hashicorp/consul/latest/docs/data-sources/keys) to retrieve those values in other configurations. + +- In systems that support user-defined labels or tags, use a tagging convention to make resources automatically discoverable. For example, use [the `aws_vpc` resource type](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc) to assign suitable tags and then [the `aws_vpc` data source](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) to query by those tags in other configurations. + +- For server addresses, use a provider-specific resource to create a DNS record with a predictable name. Then you can either use that name directly or use [the `dns` provider](https://registry.terraform.io/providers/hashicorp/dns/latest/docs) to retrieve the published addresses in other configurations. + +- If you store a Terraform state for one configuration in a remote backend that other configurations can access, then the other configurations can use [`terraform_remote_state`](/language/state/remote-state-data) to directly consume its root module outputs. This setup creates a tighter coupling between configurations, and the root configuration does not need to publish its results in a separate system. + ## Interactions with Terraform Cloud Workspaces Terraform Cloud organizes infrastructure using workspaces, but its workspaces -act more like completely separate working directories; each Terraform Cloud +act more like completely separate working directories. Each Terraform Cloud workspace has its own Terraform configuration, set of variable values, state data, run history, and settings. -These two kinds of workspaces are different, but related. When [using Terraform -CLI as a frontend for Terraform Cloud](/cli/cloud), you can associate the current working -directory with one or more remote workspaces. If you associate the -directory with multiple workspaces (using workspace tags), you can use the -`terraform workspace` commands to select which remote workspace to use. +When you [integrate Terraform CLI with Terraform Cloud](/cli/cloud), you can associate the current CLI working directory with one or more remote Terraform Cloud workspaces. Then, use the `terraform workspace` commands to select the remote workspace you want to use for each run. + +Refer to [CLI-driven Runs](/cloud-docs/run/cli) in the Terraform Cloud documentation for more details. + + +## Workspace Internals + +Workspaces are technically equivalent to renaming your state file. Terraform then includes a set of protections and support for remote state. + +Workspaces are also meant to be a shared resource. They are not private, unless you use purely local state and do not commit your state to version control. + +For local state, Terraform stores the workspace states in a directory called `terraform.tfstate.d`. This directory should be treated similarly to local-only `terraform.tfstate`. Some teams commit these files to version control, but we recommend using a remote backend instead when there are multiple collaborators. + +For [remote state](/language/state/remote), the workspaces are stored directly in the configured [backend](/language/settings/backends). For example, if you use [Consul](/language/settings/backends/consul), the workspaces are stored by appending the workspace name to the state path. To ensure that workspace names are stored correctly and safely in all backends, the name must be valid to use in a URL path segment without escaping. -Refer to [CLI-driven Runs](/cloud-docs/run/cli) in the Terraform Cloud documentation for more details about using Terraform CLI with Terraform Cloud. +Terraform stores the current workspace name locally in the ignored `.terraform` directory. This allows multiple team members to work on different workspaces concurrently. Workspace names are also attached to associated remote workspaces in Terraform Cloud. For more details about workspace names in Terraform Cloud, refer to the [CLI Integration (recommended)](/cli/cloud/settings#arguments) and [remote backend](/language/settings/backends/remote#workspaces) and documentation. \ No newline at end of file diff --git a/website/docs/language/state/workspaces.mdx b/website/docs/language/state/workspaces.mdx index ecb3421751e6..ab8a659c8f46 100644 --- a/website/docs/language/state/workspaces.mdx +++ b/website/docs/language/state/workspaces.mdx @@ -7,71 +7,39 @@ description: >- # Workspaces -Each Terraform configuration has an associated [backend](/language/settings/backends) -that defines how operations are executed and where persistent data such as -[the Terraform state](/language/state/purpose) are -stored. - -The persistent data stored in the backend belongs to a _workspace_. Initially -the backend has only one workspace, called "default", and thus there is only -one Terraform state associated with that configuration. - -Certain backends support _multiple_ named workspaces, allowing multiple states -to be associated with a single configuration. The configuration still -has only one backend, but multiple distinct instances of that configuration -to be deployed without configuring a new backend or changing authentication +Each Terraform configuration has an associated [backend](/language/settings/backends) that defines how Terraform executes operations and where Terraform stores persistent data, like [state](/language/state/purpose). + +The persistent data stored in the backend belongs to a workspace. The backend initially has only one workspace containing one Terraform state associated with that configuration. Some backends support multiple named workspaces, allowing multiple states to be associated with a single configuration. The configuration still has only one backend, but you can deploy multiple distinct instances of that configuration without configuring a new backend or changing authentication credentials. -Multiple workspaces are currently supported by the following backends: - -* [AzureRM](/language/settings/backends/azurerm) -* [Consul](/language/settings/backends/consul) -* [COS](/language/settings/backends/cos) -* [GCS](/language/settings/backends/gcs) -* [Kubernetes](/language/settings/backends/kubernetes) -* [Local](/language/settings/backends/local) -* [OSS](/language/settings/backends/oss) -* [Postgres](/language/settings/backends/pg) -* [Remote](/language/settings/backends/remote) -* [S3](/language/settings/backends/s3) - -In the 0.9 line of Terraform releases, this concept was known as "environment". -It was renamed in 0.10 based on feedback about confusion caused by the -overloading of the word "environment" both within Terraform itself and within -organizations that use Terraform. - --> **Note**: The Terraform CLI workspace concept described in this document is -different from but related to the Terraform Cloud -[workspace](/cloud-docs/workspaces) concept. -If you use multiple Terraform CLI workspaces in a single Terraform configuration -and are migrating that configuration to Terraform Cloud, refer to [Initializing and Migrating](/cli/cloud/migrating). +-> **Note**: The Terraform CLI workspaces are different from [workspaces in Terraform Cloud](/cloud-docs/workspaces). Refer to [Initializing and Migrating](/cli/cloud/migrating) for details about migrating a configuration with multiple workspaces to Terraform Cloud. -## Using Workspaces +## Backends Supporting Multiple Workspaces -Terraform starts with a single workspace named "default". This -workspace is special both because it is the default and also because -it cannot ever be deleted. If you've never explicitly used workspaces, then -you've only ever worked on the "default" workspace. +You can use multiple workspaces with the following backends: -Workspaces are managed with the `terraform workspace` set of commands. To -create a new workspace and switch to it, you can use `terraform workspace new`; -to switch workspaces you can use `terraform workspace select`; etc. +- [AzureRM](/language/settings/backends/azurerm) +- [Consul](/language/settings/backends/consul) +- [COS](/language/settings/backends/cos) +- [GCS](/language/settings/backends/gcs) +- [Kubernetes](/language/settings/backends/kubernetes) +- [Local](/language/settings/backends/local) +- [OSS](/language/settings/backends/oss) +- [Postgres](/language/settings/backends/pg) +- [Remote](/language/settings/backends/remote) +- [S3](/language/settings/backends/s3) -For example, creating a new workspace: -```text -$ terraform workspace new bar -Created and switched to workspace "bar"! +## Using Workspaces -You're now on a new, empty workspace. Workspaces isolate their state, -so if you run "terraform plan" Terraform will not see any existing state -for this configuration. -``` +~> **Important:** Workspaces are not appropriate for system decomposition or deployments requiring separate credentials and access controls. Refer to [Use Cases](/cli/workspaces#use-cases) in the Terraform CLI documentation for details and recommended alternatives. + +Terraform starts with a single, default workspace named `default` that you cannot delete. If you have not created a new workspace, you are using the default workspace in your Terraform working directory. + +When you run `terraform plan` in a new workspace, Terraform does not access existing resources in other workspaces. These resources still physically exist, but you must switch workspaces to manage them. + +Refer to the [Terraform CLI workspaces](/cli/workspaces) documentation for full details about how to create and use workspaces. -As the command says, if you run `terraform plan`, Terraform will not see -any existing resources that existed on the default (or any other) workspace. -**These resources still physically exist,** but are managed in another -Terraform workspace. ## Current Workspace Interpolation @@ -103,103 +71,3 @@ resource "aws_instance" "example" { # ... other arguments } ``` - -## When to use Multiple Workspaces - -Named workspaces allow conveniently switching between multiple instances of -a _single_ configuration within its _single_ backend. They are convenient in -a number of situations, but cannot solve all problems. - -A common use for multiple workspaces is to create a parallel, distinct copy of -a set of infrastructure in order to test a set of changes before modifying the -main production infrastructure. For example, a developer working on a complex -set of infrastructure changes might create a new temporary workspace in order -to freely experiment with changes without affecting the default workspace. - -Non-default workspaces are often related to feature branches in version control. -The default workspace might correspond to the "main" or "trunk" branch, -which describes the intended state of production infrastructure. When a -feature branch is created to develop a change, the developer of that feature -might create a corresponding workspace and deploy into it a temporary "copy" -of the main infrastructure so that changes can be tested without affecting -the production infrastructure. Once the change is merged and deployed to the -default workspace, the test infrastructure can be destroyed and the temporary -workspace deleted. - -When Terraform is used to manage larger systems, teams should use multiple -separate Terraform configurations that correspond with suitable architectural -boundaries within the system so that different components can be managed -separately and, if appropriate, by distinct teams. Workspaces _alone_ -are not a suitable tool for system decomposition, because each subsystem should -have its own separate configuration and backend, and will thus have its own -distinct set of workspaces. - -In particular, organizations commonly want to create a strong separation -between multiple deployments of the same infrastructure serving different -development stages (e.g. staging vs. production) or different internal teams. -In this case, the backend used for each deployment often belongs to that -deployment, with different credentials and access controls. Named workspaces -are _not_ a suitable isolation mechanism for this scenario. - -Instead, use one or more [re-usable modules](/language/modules/develop) to -represent the common elements, and then represent each instance as a separate -configuration that instantiates those common elements in the context of a -different backend. In that case, the root module of each configuration will -consist only of a backend configuration and a small number of `module` blocks -whose arguments describe any small differences between the deployments. - -Where multiple configurations are representing distinct system components -rather than multiple deployments, data can be passed from one component to -another using paired resources types and data sources. For example: - -* Where a shared [Consul](https://www.consul.io/) cluster is available, use - [`consul_key_prefix`](https://registry.terraform.io/providers/hashicorp/consul/latest/docs/resources/key_prefix) to - publish to the key/value store and [`consul_keys`](https://registry.terraform.io/providers/hashicorp/consul/latest/docs/data-sources/keys) - to retrieve those values in other configurations. - -* In systems that support user-defined labels or tags, use a tagging convention - to make resources automatically discoverable. For example, use - [the `aws_vpc` resource type](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc) - to assign suitable tags and then - [the `aws_vpc` data source](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) - to query by those tags in other configurations. - -* For server addresses, use a provider-specific resource to create a DNS - record with a predictable name and then either use that name directly or - use [the `dns` provider](https://registry.terraform.io/providers/hashicorp/dns/latest/docs) to retrieve - the published addresses in other configurations. - -* If a Terraform state for one configuration is stored in a remote backend - that is accessible to other configurations then - [`terraform_remote_state`](/language/state/remote-state-data) - can be used to directly consume its root module outputs from those other - configurations. This creates a tighter coupling between configurations, - but avoids the need for the "producer" configuration to explicitly - publish its results in a separate system. - -## Workspace Internals - -Workspaces are technically equivalent to renaming your state file. They -aren't any more complex than that. Terraform wraps this simple notion with -a set of protections and support for remote state. - -For local state, Terraform stores the workspace states in a directory called -`terraform.tfstate.d`. This directory should be treated similarly to -local-only `terraform.tfstate`; some teams commit these files to version -control, although using a remote backend instead is recommended when there are -multiple collaborators. - -For [remote state](/language/state/remote), the workspaces are stored -directly in the configured [backend](/language/settings/backends). For example, if you -use [Consul](/language/settings/backends/consul), the workspaces are stored -by appending the workspace name to the state path. To ensure that -workspace names are stored correctly and safely in all backends, the name -must be valid to use in a URL path segment without escaping. - -The important thing about workspace internals is that workspaces are -meant to be a shared resource. They aren't a private, local-only notion -(unless you're using purely local state and not committing it). - -The "current workspace" name is stored locally in the ignored -`.terraform` directory. This allows multiple team members to work on -different workspaces concurrently. Workspace names are also attached to associated remote workspaces in Terraform Cloud. For more details about workspace names in Terraform Cloud, refer to the [remote backend](/language/settings/backends/remote#workspaces) and [CLI Integration (recommended)](/cli/cloud/settings#arguments) documentation. diff --git a/website/docs/language/upgrade-guides/index.mdx b/website/docs/language/upgrade-guides/index.mdx index 73670f510a35..1a26c14e0d69 100644 --- a/website/docs/language/upgrade-guides/index.mdx +++ b/website/docs/language/upgrade-guides/index.mdx @@ -1,13 +1,124 @@ --- -page_title: Upgrading to Terraform v1.4 -description: Upgrading to Terraform v1.4 +page_title: Upgrading to Terraform v1.3 +description: Upgrading to Terraform v1.3 --- -# Upgrading to Terraform v1.4 +# Upgrading to Terraform v1.3 --> Do you need the upgrade guide for an earlier version of Terraform? Use the -version selector in the navigation bar to select the version you are intending -to upgrade to. +-> **Note:** Use the version selector to view the upgrade guides for older Terraform versions. -Terraform v1.4 is still under development and so its upgrade guide is not yet -finalized. This should be updated before the final v1.4.0 release. +Terraform v1.3 is a minor release in the stable Terraform v1.0 series. + +Terraform v1.3 continues to honor [the Terraform v1.0 Compatibility Promises](https://www.terraform.io/language/v1-compatibility-promises), but there are some behavior changes outside of those promises that may affect a small number of users. Specifically, the following updates may require additional upgrade steps: + +* [Removal of Deprecated State Storage Backends](#removal-of-deprecated-state-storage-backends) +* [Concluding the Optional Attributes Experiment](#concluding-the-optional-attributes-experiment) +* [AzureRM Backend Requires Microsoft Graph](#azurerm-backend-requires-microsoft-graph) +* [Other Small Changes](#other-small-changes) + +If you encounter any problems during upgrading which are not by this guide, or if the migration instructions don't work for you, please start a topic in [the Terraform community forum](https://discuss.hashicorp.com/c/terraform-core/27) to discuss it. + +## Removal of Deprecated State Storage Backends + +Terraform currently requires that all supported state storage backends be maintained in the Terraform codebase and compiled into Terraform CLI. Terraform therefore contains a mixture of backends maintained by the Terraform CLI team, backends maintained by other teams at HashiCorp, and backends maintained by third-party contributors. + +There are a number of backends that we have so far preserved on a best-effort basis despite them not having any active maintainers. Due to the overhead of continuing to support them, we deprecated the following unmaintained backends in Terraform v1.2.3: +* `artifactory` +* `etcd` +* `etcdv3` +* `manta` +* `swift` + +All of these deprecated state storage backends are now removed in Terraform v1.3. If you are using any of these you will need to migrate to another state storage backend using Terraform v1.2 before you upgrade to Terraform v1.3. + +The following sections describe some specific migration considerations for each removed backend. + +### Migrating from the `artifactory` backend + +From JFrog Artifactory 7.38.4 or later, Artifactory has support for the state storage protocol used by Terraform's `remote` backend, using a special repository type called a [Terraform Backend Repository](https://www.jfrog.com/confluence/display/JFROG/Terraform+Backend+Repository). + +The `remote` backend was available in Terraform v1.2 and remains available in Terraform v1.3. If you are using the `artifactory` backend then we recommend migrating to the `remote` backend, using the configuration instructions provided by JFrog, before upgrading to Terraform v1.3. + +### Migrating from the `etcd` and `etcdv3` backends + +The two generations of state storage backend for [etcd](https://etcd.io/) have been removed and have no direct replacement. + +If you are [using etcd in conjunction with Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/), you might choose to migrate to [the `kubernetes` state storage backend](https://www.terraform.io/language/settings/backends/kubernetes), which stores Terraform state snapshots under a Kubernetes secret. + +### Migrating from the `manta` backend + +The Manta backend was written for an object storage system developed by Joyent. However, the backend was targeting the original implementation of that system which shut down in November 2019. + +This backend has therefore been unmaintained for several years and is now removed without replacement. + +### Migrating from the `swift` backend + +The `swift` backend was for OpenStack's object storage system, Swift. This backend has not had an active maintainer for some time and has not kept up with new features and changes to Swift itself, and so it is now removed. + +OpenStack Swift contains an implementation of the Amazon S3 API. Although [Terraform's `s3` backend](https://www.terraform.io/language/settings/backends/s3) officially supports only Amazon's implementation of that API, we have heard from users that they have had success using that backend to store Terraform state snapshots in Swift. + +If you intend to migrate to the `s3` backend then you should complete that migration with Terraform v1.2 before you upgrade to Terraform v1.3. + +## Concluding the Optional Attributes Experiment + +Terraform v0.14.0 introduced a new _experimental_ language feature for declaring object type constraints with optional attributes in your module's input variables. Thanks to feedback from those who tried the experiment, a refinement of that functionality is now stablized in Terraform v1.3. + +For general information on this new feature, see [Optional Object Type Attributes](/language/expressions/type-constraints#optional-object-type-attributes). + +If you have any experimental modules that were using the feature in its previous form, you can now adapt those modules for production use with the final form of the feature by making the following changes: + +1. Remove the `experiments = [module_variable_optional_attrs]` experiment opt-in from your module, and replace it with a Terraform version constraint inside the same `terraform` block: + + ```hcl + terraform { + required_version = ">= 1.3.0" + } + ``` + + This version constraint makes it explicit that your module is using language features added in Terraform v1.3.0, which earlier versions of Terraform can use to give better feedback about the module not being supported there. +2. If you were using the experimental `defaults` function, you will need to replace your use of it with the new syntax for declaring defaults as part of your main type constraint. + + For example, you can declare a default value for an optional string attribute using a second argument to the `optional` syntax, inline in your type constraint expression: + + ```hcl + type = object({ + example = optional(string, "default value") + }) + ``` + +Because the experiment is concluded, the experimental implementation of this feature is no longer available and Terraform v1.3.0 and later will not accept any module that contains the explicit experiment opt-in. + +As with all new language features, you should take care to upgrade Terraform for all configurations which use a shared module before you use optional attributes in that shared module. Any module which must remain compatible with older versions of Terraform must not declare any optional attributes. Once all users of a module are using Terraform v1.3.0 or later, you can safely begin using optional attribute declarations. + +## AzureRM Backend Requires Microsoft Graph + +In response to [Microsoft's deprecation of Azure AD Graph](https://docs.microsoft.com/en-us/graph/migrate-azure-ad-graph-faq), Terraform v1.1 marked the beginning of a deprecation cycle for support of Azure AD Graph in Terraform's `azurerm` backend. + +That deprecation cycle has now concluded with the total removal of Azure AD Graph support in Terraform v1.3. The AzureRM backend now supports only [Microsoft Graph](https://docs.microsoft.com/en-us/graph/overview). + +If you previously set `use_microsoft_graph = true` in your backend configuration to explicitly opt in to using the Microsoft Graph client instead of Azure AD Graph, you will need to now remove that argument from your backend configuration. + +If you remove this setting in an already-initialized Terraform working directory then Terraform will detect it as a configuration change and prompt you to decide whether to migrate state to a new location. Because removing that setting does not change the physical location of the state snapshots, you should _not_ tell Terraform to migrate the state to a new location and should instead use the `-reconfigure` option to `terraform init`: + +``` +terraform init -reconfigure +``` + +If you did not previously set the `use_microsoft_graph` argument then you do not need to make any changes. Microsoft Graph is now used by default and is the only available implementation. + +## Other Small Changes + +There are some other changes in Terraform v1.3 that we don't expect to have a great impact but may affect a small number of users: +* `terraform import` no longer supports the option `-allow-missing-config`. This option was originally added as a backward-compatibility helper when Terraform first began making use of the configuration during import, but the behavior of the import command was significantly limited by the requirement to be able to work without configuration, and so configuration is now required. + + In most cases it is sufficient to write just an empty `resource` block whose resource type and name matches the address given on the `terraform import` command line. This will cause Terraform to associate the import operation with the default provider configuration for the provider that the resource belongs to. +* `terraform show -json` previously simplified the "unknown" status for all output values to be a single boolean value, even though an output value of a collection or structural type can potentially be only partially unknown. + + The JSON output now accurately describes partially-unknown output values in the same way as it describes partially-unknown values in resource attributes. Any consumer of the plan JSON format which was relying on output values always being either known or entirely unknown must be changed to support more complex situations in the `after_unknown` property of [the JSON Change Representation](https://www.terraform.io/internals/json-format#change-representation). +* When making requests to HTTPS servers, Terraform will now reject invalid TLS handshakes that have duplicate extensions, as required by RFC 5246 section 7.4.1.4 and RFC 8446 section 4.2. This may cause new errors when interacting with existing buggy or misconfigured TLS servers, but should not affect correct servers. + + If you see new HTTPS, TLS, or SSL-related error messages after upgrading to Terraform v1.2, that may mean that the server that Terraform was trying to access has an incorrect implementation of the relevant protocols and will need to be upgraded to a correct version for continued use with Terraform. + + Similar problems can also arise on networks that use HTTPS-intercepting [middleboxes](https://en.wikipedia.org/wiki/Middlebox), such as deep packet inspection firewalls. In that case, the protocol implementation of the middlebox must also be correct in order for Terraform to successfully access HTTPS servers through it. + + This only applies to requests made directly by Terraform CLI, such as provider installation and remote state storage. Terraform providers are separate programs which decide their own policy for handling of TLS handshakes.