Auto truncate comment in release making process

[ScottSuarez]

When making a release the release note thats generated contains a lot of unrelated content that the user needs to manually truncate. I would be nice to automatically detect and truncate this string before generating the note.

steps 4 and 7

https://github.com/hashicorp/terraform-provider-google/wiki/Release-Process#bug-on-duty

[SarahFrench]

I took a quick look at this problem and I wanted to share what I learned:

The release process owned by HashiCorp expects there to be an artifact, containing the release notes for a given release, uploaded to the repo containing the provider. This is the section of the GHA workflow for this repo that defines how the release notes are generated and then uploaded.

The sed command we currently use to pull out the section of CHANGELOG.md relevant to the latest release is something made by HashiCorp as an example of how to get inputs to pass into the release workflow : see it here in the release tooling's README. It's something that isn't guaranteed to work for every provider repo, though.

The sed command uses git describe to pull in the latest tag and then uses that output to identify the previous tag. This is used to prevent lines in the changelog from previous releases being streamed into the output release-notes.txt file. So if you tag v4.50.0 and the previous tag is v4.49.0 then you should see all of your changelog file above the entry for v4.49.0 being put into release-notes.txt.

Here's a clue to the problem I found about the issue: The command below is nested in the sed command. When I check out the main branch of this repo and run the command:

git checkout dbb48bed753b55ec4a1f6464111dbe75b12f757a
git describe --abbrev=0 --match='v*.*.*' --tags

#v4.40.0

...it returns v4.40.0 but I'd expect there to be newer tags because that's a few releases ago. When I check to see what the previous tag was (i.e. run the same command but exclude v4.40.0) I see:

git describe --abbrev=0 --exclude="v4.40.0" --match='v*.*.*' --tags

# v4.27.0

I then found out about this command, which will "Only list tags whose commits are reachable from the specified commit (HEAD if not specified)". The grep avoids noise and focuses on recent history:

git tag --merged | grep v4.

# v4.17.0
# v4.27.0
# v4.40.0

I believe this is the reason why old entries are found in the release notes!

Either we need to get a clearer idea of the tagging & merging process or we need to work out a different process to generate that release-notes.txt artifact.

[SarahFrench]

This solution seems hacky and still has the previous problem of needing to reliably discover the previous version tag

git checkout v4.42.0

export PREV_VERSION=4.41

sed -e '1,/'$PREV_VERSION'/!d' -e '/'$PREV_VERSION'/d'  CHANGELOG.md > release-notes.txt

Edit:

Getting tag names in reliable order is possible with git tag -l --sort=-version:refname

git checkout v4.42.0

export LAST_TAG=$(git tag -l --sort=-version:refname |head -n 1)
export PREV_TAG=$(git tag -l --sort=-version:refname |head -n 2 | tail -n 1)
export PREV_VERSION=${PREV_TAG//v}

echo $LAST_TAG
# v4.42.0

echo $PREV_TAG
# v4.41.0

echo $PREV_VERSION
# 4.41.0

Edit 2:

Possible solution but could be brittle as it assumes tags are added to the repo in the correct format and sequential order:

git checkout 2178c996a562153ab06bf104a1bb1509ccb74510
# HEAD of main today

export PREV_TAG=$(git tag -l --sort=-version:refname |head -n 2 | tail -n 1)

export PREV_VERSION=${PREV_TAG//v}

sed -n -e "1{/# /d;}" -e "2{/^$/d;}" -e "/# $PREV_VERSION/q;p" CHANGELOG.md > release-notes.txt

Note:

• The "1{/# /d;}" command removes the first line (and I think checks it contains a #) - e.g. removes ## 4.43.0 (Unreleased) from your release notes for v4.42.0
• The "-e "2{/^$/d;}" command removes an empty newline at the start of the release notes
• The "/# $PREV_VERSION/q;p" command causes sed to quit when encountering a line matching the pattern including the prev release number - so no lines from previous releases are added to the output file

⭐ **Output release notes produced from above code block** ⭐

## 4.42.0 (October 31, 2022)

FEATURES:
* **New Data Source:** `google_compute_addresses` ([#12829](https://github.com/hashicorp/terraform-provider-google/pull/12829))
* **New Data Source:** `google_compute_region_network_endpoint_group` ([#12849](https://github.com/hashicorp/terraform-provider-google/pull/12849))
* **New Resource:** `google_alloydb_cluster` ([#12772](https://github.com/hashicorp/terraform-provider-google/pull/12772))
* **New Resource:** `google_bigquery_analytics_hub_data_exchange_iam` ([#12845](https://github.com/hashicorp/terraform-provider-google/pull/12845))
* **New Resource:** `google_bigquery_analytics_hub_data_exchange` ([#12845](https://github.com/hashicorp/terraform-provider-google/pull/12845))
* **New Resource:** `google_bigquery_analytics_hub_listing_iam` ([#12845](https://github.com/hashicorp/terraform-provider-google/pull/12845))
* **New Resource:** `google_bigquery_analytics_hub_listing` ([#12845](https://github.com/hashicorp/terraform-provider-google/pull/12845))
* **New Resource:** `google_iam_workforce_pool` ([#12863](https://github.com/hashicorp/terraform-provider-google/pull/12863))
* **New Resource:** `google_monitoring_generic_service` ([#12796](https://github.com/hashicorp/terraform-provider-google/pull/12796))
* **New Resource:** `google_scc_source_iam_binding` ([#12840](https://github.com/hashicorp/terraform-provider-google/pull/12840))
* **New Resource:** `google_scc_source_iam_member` ([#12840](https://github.com/hashicorp/terraform-provider-google/pull/12840))
* **New Resource:** `google_scc_source_iam_policy` ([#12840](https://github.com/hashicorp/terraform-provider-google/pull/12840))
* **New Resource:** `google_vertex_ai_endpoint` ([#12858](https://github.com/hashicorp/terraform-provider-google/pull/12858))
* **New Resource:** `google_vertex_ai_featurestore_entitytype_feature` ([#12797](https://github.com/hashicorp/terraform-provider-google/pull/12797))
* **New Resource:** `google_vertex_ai_featurestore_entitytype` ([#12797](https://github.com/hashicorp/terraform-provider-google/pull/12797))
* **New Resource:** `google_vertex_ai_featurestore` ([#12797](https://github.com/hashicorp/terraform-provider-google/pull/12797))

IMPROVEMENTS:
* appengine: added `member` field to `google_app_engine_default_service_account` datasource ([#12768](https://github.com/hashicorp/terraform-provider-google/pull/12768))
* bigquery: added `max_time_travel_hours` field in `google_bigquery_dataset` resource ([#12830](https://github.com/hashicorp/terraform-provider-google/pull/12830))
* bigquery: added `member` field to `google_bigquery_default_service_account` datasource ([#12768](https://github.com/hashicorp/terraform-provider-google/pull/12768))
* cloudbuild: added `script` field to `google_cloudbuild_trigger` resource ([#12841](https://github.com/hashicorp/terraform-provider-google/pull/12841))
* cloudplatform: validated `project_id` for `google_project` data-source ([#12846](https://github.com/hashicorp/terraform-provider-google/pull/12846))
* compute: added `source_disk` field to `google_compute_disk` and `google_compute_region_disk` resource ([#12779](https://github.com/hashicorp/terraform-provider-google/pull/12779))
* compute: added general field `rules` to `google_compute_router_nat` ([#12815](https://github.com/hashicorp/terraform-provider-google/pull/12815))
* container: added support for in-place update of `node_config.0.tags` for `google_container_node_pool` resource ([#12773](https://github.com/hashicorp/terraform-provider-google/pull/12773))
* container: added support for the Disk type and size configuration on the GKE Node Auto-provisioning ([#12786](https://github.com/hashicorp/terraform-provider-google/pull/12786))
* container: promote `enable_cost_allocation` field in `google_container_cluster` to GA ([#12866](https://github.com/hashicorp/terraform-provider-google/pull/12866))
* datastream: added `private_connectivity` field to `google_datastream_connection_profile` ([#12844](https://github.com/hashicorp/terraform-provider-google/pull/12844))
* dns: added `enable_geo_fencing` to `routing_policy` block of `google_dns_record_set` resource ([#12859](https://github.com/hashicorp/terraform-provider-google/pull/12859))
* dns: added `health_checked_targets` to `wrr` and `geo` blocks of `google_dns_record_set` resource ([#12859](https://github.com/hashicorp/terraform-provider-google/pull/12859))
* dns: added `primary_backup` to `routing_policy` block of `google_dns_record_set` resource ([#12859](https://github.com/hashicorp/terraform-provider-google/pull/12859))
* firebase: added deletion support and new field `deletion_policy` for `google_firebase_web_app` ([#12812](https://github.com/hashicorp/terraform-provider-google/pull/12812))
* privateca: added a new field `skip_grace_period` to skip the grace period when deleting a CertificateAuthority. ([#12784](https://github.com/hashicorp/terraform-provider-google/pull/12784))
* serviceaccount: added `member` field to `google_service_account` resource and datasource ([#12768](https://github.com/hashicorp/terraform-provider-google/pull/12768))
* sql: added `time_zone` field in `google_sql_database_instance` ([#12760](https://github.com/hashicorp/terraform-provider-google/pull/12760))
* storage: added `member` field to `google_storage_project_service_account` and `google_storage_transfer_project_service_account` datasource ([#12768](https://github.com/hashicorp/terraform-provider-google/pull/12768))
* storage: promoted `public_access_prevention` field on `google_storage_bucket` resource to GA ([#12766](https://github.com/hashicorp/terraform-provider-google/pull/12766))
* vpcaccess: promoted `machine_type`, `min_instances`, `max_instances`, and `subnet` in `google_vpc_access_connector` to GA ([#12838](https://github.com/hashicorp/terraform-provider-google/pull/12838))

BUG FIXES:
* compute: made `vm_count` in `google_compute_resource_policy` optional ([#12807](https://github.com/hashicorp/terraform-provider-google/pull/12807))
* container: fixed inability to update `datapath_provider` on `google_container_cluster` by making field changes trigger resource recreation ([#12887](https://github.com/hashicorp/terraform-provider-google/pull/12887))
* pubsub: ensured topics are recreated when their schemas change. ([#12806](https://github.com/hashicorp/terraform-provider-google/pull/12806))
* redis: updated `persistence_config.rdb_snapshot_period` to optional in the `google_redis_instance` resource. ([#12872](https://github.com/hashicorp/terraform-provider-google/pull/12872))

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.