backport of commit ce02344

CHANGELOG.md

@@ -1,14 +1,86 @@
-## 1.4.0 (Unreleased)
+## 1.3.1 (Unreleased)
 
 BUG FIXES:
 
-* The module installer will now record in its manifest a correct module source URL after normalization when the URL given as input contains both a query string portion and a subdirectory portion. Terraform itself doesn't currently make use of this information and so this is just a cosmetic fix to make the recorded metadata more correct. [GH-31636]
+* Fixed a crash when using objects with optional attributes and default values in collections, most visible with nested modules. [GH-31847]
+
+## 1.3.0 (September 21, 2022)
+
+NEW FEATURES:
+
+* **Optional attributes for object type constraints:** When declaring an input variable whose type constraint includes an object type, you can now declare individual attributes as optional, and specify a default value to use if the caller doesn't set it. For example:
+
+    ```terraform
+    variable "with_optional_attribute" {
+      type = object({
+        a = string                # a required attribute
+        b = optional(string)      # an optional attribute
+        c = optional(number, 127) # an optional attribute with a default value
+      })
+    }
+    ```
+
+    Assigning `{ a = "foo" }` to this variable will result in the value `{ a = "foo", b = null, c = 127 }`.
+
+* Added functions: `startswith` and `endswith` allow you to check whether a given string has a specified prefix or suffix. ([#31220](https://github.com/hashicorp/terraform/issues/31220))
+
+UPGRADE NOTES:
+
+* `terraform show -json`: Output changes now include more detail about the unknown-ness of the planned value. Previously, a planned output would be marked as either fully known or partially unknown, with the `after_unknown` field having value `false` or `true` respectively. Now outputs correctly expose the full structure of unknownness for complex values, allowing consumers of the JSON output format to determine which values in a collection are known only after apply.
+* `terraform import`: The `-allow-missing-config` has been removed, and at least an empty configuration block must exist to import a resource.
+* Consumers of the JSON output format expecting on the `after_unknown` field to be only `false` or `true` should be updated to support [the change representation](https://www.terraform.io/internals/json-format#change-representation) described in the documentation, and as was already used for resource changes. ([#31235](https://github.com/hashicorp/terraform/issues/31235))
+* AzureRM Backend: This release concludes [the deprecation cycle started in Terraform v1.1](https://www.terraform.io/language/upgrade-guides/1-1#preparation-for-removing-azure-ad-graph-support-in-the-azurerm-backend) for the `azurerm` backend's support of "ADAL" authentication. This backend now supports only "MSAL" (Microsoft Graph) authentication.
+
+    This follows from [Microsoft's own deprecation of Azure AD Graph](https://docs.microsoft.com/en-us/graph/migrate-azure-ad-graph-faq), and so you must follow the migration instructions presented in that Azure documentation to adopt Microsoft Graph and then change your backend configuration to use MSAL authentication before upgrading to Terraform v1.3.
+* When making requests to HTTPS servers, Terraform will now reject invalid handshakes that have duplicate extensions, as required by RFC 5246 section 7.4.1.4 and RFC 8446 section 4.2. This may cause new errors when interacting with existing buggy or misconfigured TLS servers, but should not affect correct servers.
+
+    This only applies to requests made directly by Terraform CLI, such as provider installation and remote state storage. Terraform providers are separate programs which decide their own policy for handling of TLS handshakes.
+* The following backends, which were deprecated in v1.2.3, have now been removed: `artifactory`, `etcd`, `etcdv3`, `manta`, `swift`. The legacy backend name `azure` has also been removed, because the current Azure backend is named `azurerm`. ([#31711](https://github.com/hashicorp/terraform/issues/31711))
+
+ENHANCEMENTS:
+
+* config: Optional attributes for object type constraints, as described under new features above. ([#31154](https://github.com/hashicorp/terraform/issues/31154))
+* config: New built-in function `timecmp` allows determining the ordering relationship between two timestamps while taking potentially-different UTC offsets into account. ([#31687](https://github.com/hashicorp/terraform/pull/31687))
+* config: When reporting an error message related to a function call, Terraform will now include contextual information about the signature of the function that was being called, as an aid to understanding why the call might have failed. ([#31299](https://github.com/hashicorp/terraform/issues/31299))
+* config: When reporting an error or warning message that isn't caused by values being unknown or marked as sensitive, Terraform will no longer mention any values having those characteristics in the contextual information presented alongside the error. Terraform will still return this information for the small subset of error messages that are specifically about unknown values or sensitive values being invalid in certain contexts. ([#31299](https://github.com/hashicorp/terraform/issues/31299))
+* config: `moved` blocks can now describe resources moving to and from modules in separate module packages. ([#31556](https://github.com/hashicorp/terraform/issues/31556))
+* `terraform fmt` now accepts multiple target paths, allowing formatting of several individual files at once. ([#31687](https://github.com/hashicorp/terraform/issues/31687))
+* `terraform init`: provider installation errors now mention which host Terraform was downloading from ([#31524](https://github.com/hashicorp/terraform/issues/31524))
+* CLI: Terraform will report more explicitly when it is proposing to delete an object due to it having moved to a resource instance that is not currently declared in the configuration. ([#31695](https://github.com/hashicorp/terraform/issues/31695))
+* CLI: When showing the progress of a remote operation running in Terraform Cloud, Terraform CLI will include information about pre-plan run tasks ([#31617](https://github.com/hashicorp/terraform/issues/31617))
+* The AzureRM Backend now only supports MSAL (and Microsoft Graph) and no longer makes use of ADAL (and Azure Active Directory Graph) for authentication ([#31070](https://github.com/hashicorp/terraform/issues/31070))
+* The COS backend now supports global acceleration. ([#31425](https://github.com/hashicorp/terraform/issues/31425))
+* provider plugin protocol: The Terraform CLI now calls `PlanResourceChange` for compatible providers when destroying resource instances. ([#31179](https://github.com/hashicorp/terraform/issues/31179))
+* As an implementation detail of the Terraform Cloud integration, Terraform CLI will now capture and upload [the JSON integration format for state](https://www.terraform.io/internals/json-format#state-representation) along with any newly-recorded state snapshots, which then in turn allows Terraform Cloud to provide that information to API-based external integrations. ([#31698](https://github.com/hashicorp/terraform/issues/31698))
+
+BUG FIXES:
+
+* config: Terraform was not previously evaluating preconditions and postconditions during the apply phase for resource instances that didn't have any changes pending, which was incorrect because the outcome of a condition can potentially be affected by changes to _other_ objects in the configuration. Terraform will now always check the conditions for every resource instance included in a plan during the apply phase, even for resource instances that have "no-op" changes. This means that some failures that would previously have been detected only by a subsequent run will now be detected during the same run that caused them, thereby giving the feedback at the appropriate time. ([#31491](https://github.com/hashicorp/terraform/issues/31491))
+* `terraform show -json`: Fixed missing markers for unknown values in the encoding of partially unknown tuples and sets. ([#31236](https://github.com/hashicorp/terraform/issues/31236))
+* `terraform output` CLI help documentation is now more consistent with web-based documentation. ([#29354](https://github.com/hashicorp/terraform/issues/29354))
+* `terraform init`: Error messages now handle the situation where the underlying HTTP client library does not indicate a hostname for a failed request. ([#31542](https://github.com/hashicorp/terraform/issues/31542))
+* `terraform init`: Don't panic if a child module contains a resource with a syntactically-invalid resource type name. ([#31573](https://github.com/hashicorp/terraform/issues/31573))
+* CLI: The representation of destroying already-`null` output values in a destroy plan will no longer report them as being deleted, which avoids reporting the deletion of an output value that was already absent. ([#31471](https://github.com/hashicorp/terraform/issues/31471))
+* `terraform import`: Better handling of resources or modules that use `for_each`, and situations where data resources are needed to complete the operation. ([#31283](https://github.com/hashicorp/terraform/issues/31283))
+
+EXPERIMENTS:
+
+* This release concludes the `module_variable_optional_attrs` experiment, which started in Terraform v0.14.0. The final design of the optional attributes feature is similar to the experimental form in the previous releases, but with two major differences:
+    * The `optional` function-like modifier for declaring an optional attribute now accepts an optional second argument for specifying a default value to use when the attribute isn't set by the caller. If not specified, the default value is a null value of the appropriate type as before.
+    * The built-in `defaults` function, previously used to meet the use-case of replacing null values with default values, will not graduate to stable and has been removed. Use the second argument of `optional` inline in your type constraint to declare default values instead.
+
+    If you have any experimental modules that were participating in this experiment, you will need to remove the experiment opt-in and adopt the new syntax for declaring default values in order to migrate your existing module to the stablized version of this feature. If you are writing a shared module for others to use, we recommend declaring that your module requires Terraform v1.3.0 or later to give specific feedback when using the new feature on older Terraform versions, in place of the previous declaration to use the experimental form of this feature:
+
+    ```hcl
+    terraform {
+      required_version = ">= 1.3.0"
+    }
+    ```
 
 ## Previous Releases
 
 For information on prior major and minor releases, see their changelogs:
 
-* [v1.3](https://github.com/hashicorp/terraform/blob/v1.3/CHANGELOG.md)
 * [v1.2](https://github.com/hashicorp/terraform/blob/v1.2/CHANGELOG.md)
 * [v1.1](https://github.com/hashicorp/terraform/blob/v1.1/CHANGELOG.md)
 * [v1.0](https://github.com/hashicorp/terraform/blob/v1.0/CHANGELOG.md)

go.mod

@@ -42,7 +42,7 @@ require (
 	github.com/hashicorp/go-uuid v1.0.3
 	github.com/hashicorp/go-version v1.6.0
 	github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f
-	github.com/hashicorp/hcl/v2 v2.14.0
+	github.com/hashicorp/hcl/v2 v2.14.1
 	github.com/hashicorp/terraform-config-inspect v0.0.0-20210209133302-4fd17a0faac2
 	github.com/hashicorp/terraform-registry-address v0.0.0-20220623143253-7d51757b572c
 	github.com/hashicorp/terraform-svchost v0.0.0-20200729002733-f050f53b9734

go.sum

@@ -387,8 +387,8 @@ github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ
 github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f h1:UdxlrJz4JOnY8W+DbLISwf2B8WXEolNRA8BGCwI9jws=
 github.com/hashicorp/hcl v0.0.0-20170504190234-a4b07c25de5f/go.mod h1:oZtUIOe8dh44I2q6ScRibXws4Ajl+d+nod3AaR9vL5w=
 github.com/hashicorp/hcl/v2 v2.0.0/go.mod h1:oVVDG71tEinNGYCxinCYadcmKU9bglqW9pV3txagJ90=
-github.com/hashicorp/hcl/v2 v2.14.0 h1:jX6+Q38Ly9zaAJlAjnFVyeNSNCKKW8D0wvyg7vij5Wc=
-github.com/hashicorp/hcl/v2 v2.14.0/go.mod h1:e4z5nxYlWNPdDSNYX+ph14EvWYMFm3eP0zIUqPc2jr0=
+github.com/hashicorp/hcl/v2 v2.14.1 h1:x0BpjfZ+CYdbiz+8yZTQ+gdLO7IXvOut7Da+XJayx34=
+github.com/hashicorp/hcl/v2 v2.14.1/go.mod h1:e4z5nxYlWNPdDSNYX+ph14EvWYMFm3eP0zIUqPc2jr0=
 github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d h1:9ARUJJ1VVynB176G1HCwleORqCaXm/Vx0uUi0dL26I0=
 github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d/go.mod h1:Yog5+CPEM3c99L1CL2CFCYoSzgWm5vTU58idbRUaLik=
 github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=

internal/addrs/module_source.go

@@ -316,17 +316,10 @@ func parseModuleSourceRemote(raw string) (ModuleSourceRemote, error) {
 func (s ModuleSourceRemote) moduleSource() {}
 
 func (s ModuleSourceRemote) String() string {
-	base := s.Package.String()
-
 	if s.Subdir != "" {
-		// Address contains query string
-		if strings.Contains(base, "?") {
-			parts := strings.SplitN(base, "?", 2)
-			return parts[0] + "//" + s.Subdir + "?" + parts[1]
-		}
-		return base + "//" + s.Subdir
+		return s.Package.String() + "//" + s.Subdir
 	}
-	return base
+	return s.Package.String()
 }
 
 func (s ModuleSourceRemote) ForDisplay() string {

internal/addrs/module_source_test.go

@@ -154,13 +154,6 @@ func TestParseModuleSource(t *testing.T) {
 				Subdir:  "bleep/bloop",
 			},
 		},
-		"git over HTTPS, URL-style, subdir, query parameters": {
-			input: "git::https://example.com/code/baz.git//bleep/bloop?otherthing=blah",
-			want: ModuleSourceRemote{
-				Package: ModulePackage("git::https://example.com/code/baz.git?otherthing=blah"),
-				Subdir:  "bleep/bloop",
-			},
-		},
 		"git over SSH, URL-style": {
 			input: "git::ssh://git@example.com/code/baz.git",
 			want: ModuleSourceRemote{
@@ -408,56 +401,6 @@ func TestModuleSourceRemoteFromRegistry(t *testing.T) {
 	})
 }
 
-func TestParseModuleSourceRemote(t *testing.T) {
-
-	tests := map[string]struct {
-		input          string
-		wantString     string
-		wantForDisplay string
-		wantErr        string
-	}{
-		"git over HTTPS, URL-style, query parameters": {
-			// Query parameters should be correctly appended after the Package
-			input:          `git::https://example.com/code/baz.git?otherthing=blah`,
-			wantString:     `git::https://example.com/code/baz.git?otherthing=blah`,
-			wantForDisplay: `git::https://example.com/code/baz.git?otherthing=blah`,
-		},
-		"git over HTTPS, URL-style, subdir, query parameters": {
-			// Query parameters should be correctly appended after the Package and Subdir
-			input:          `git::https://example.com/code/baz.git//bleep/bloop?otherthing=blah`,
-			wantString:     `git::https://example.com/code/baz.git//bleep/bloop?otherthing=blah`,
-			wantForDisplay: `git::https://example.com/code/baz.git//bleep/bloop?otherthing=blah`,
-		},
-	}
-
-	for name, test := range tests {
-		t.Run(name, func(t *testing.T) {
-			remote, err := parseModuleSourceRemote(test.input)
-
-			if test.wantErr != "" {
-				switch {
-				case err == nil:
-					t.Errorf("unexpected success\nwant error: %s", test.wantErr)
-				case err.Error() != test.wantErr:
-					t.Errorf("wrong error messages\ngot:  %s\nwant: %s", err.Error(), test.wantErr)
-				}
-				return
-			}
-
-			if err != nil {
-				t.Fatalf("unexpected error: %s", err.Error())
-			}
-
-			if got, want := remote.String(), test.wantString; got != want {
-				t.Errorf("wrong String() result\ngot:  %s\nwant: %s", got, want)
-			}
-			if got, want := remote.ForDisplay(), test.wantForDisplay; got != want {
-				t.Errorf("wrong ForDisplay() result\ngot:  %s\nwant: %s", got, want)
-			}
-		})
-	}
-}
-
 func TestParseModuleSourceRegistry(t *testing.T) {
 	// We test parseModuleSourceRegistry alone here, in addition to testing
 	// it indirectly as part of TestParseModuleSource, because general

internal/cloud/state.go

@@ -92,13 +92,19 @@ func (s *State) WriteStateForMigration(f *statefile.File, force bool) error {
 		}
 	}
 
+	// The remote backend needs to pass the `force` flag through to its client.
+	// For backends that support such operations, inform the client
+	// that a force push has been requested
+	if force {
+		s.EnableForcePush()
+	}
+
 	// We create a deep copy of the state here, because the caller also has
 	// a reference to the given object and can potentially go on to mutate
 	// it after we return, but we want the snapshot at this point in time.
 	s.state = f.State.DeepCopy()
 	s.lineage = f.Lineage
 	s.serial = f.Serial
-	s.forcePush = force
 
 	return nil
 }
@@ -130,7 +136,6 @@ func (s *State) WriteState(state *states.State) error {
 	// a reference to the given object and can potentially go on to mutate
 	// it after we return, but we want the snapshot at this point in time.
 	s.state = state.DeepCopy()
-	s.forcePush = false
 
 	return nil
 }
@@ -409,6 +414,12 @@ func (s *State) Delete() error {
 	return nil
 }
 
+// EnableForcePush to allow the remote client to overwrite state
+// by implementing remote.ClientForcePusher
+func (s *State) EnableForcePush() {
+	s.forcePush = true
+}
+
 // GetRootOutputValues fetches output values from Terraform Cloud
 func (s *State) GetRootOutputValues() (map[string]*states.OutputValue, error) {
 	ctx := context.Background()

internal/command/command_test.go

@@ -138,6 +138,9 @@ func metaOverridesForProvider(p providers.Interface) *testingOverrides {
 		Providers: map[addrs.Provider]providers.Factory{
 			addrs.NewDefaultProvider("test"):                                           providers.FactoryFixed(p),
 			addrs.NewProvider(addrs.DefaultProviderRegistryHost, "hashicorp2", "test"): providers.FactoryFixed(p),
+			addrs.NewLegacyProvider("null"):                                            providers.FactoryFixed(p),
+			addrs.NewLegacyProvider("azurerm"):                                         providers.FactoryFixed(p),
+			addrs.NewProvider(addrs.DefaultProviderRegistryHost, "acmecorp", "aws"):    providers.FactoryFixed(p),
 		},
 	}
 }