`azurerm_automation_job_schedule`: update jobSchedule resource id format #22164

wuxu92 · 2023-06-14T03:29:39Z

Draft: Wait hashicorp/go-azure-helpers#208 to be released and update the verion in azurerm.

The job schdule is a link between a runbook and a schdule. there should be only one such link between the runbook and schedule pair. When we update a runbook, request a PUT of runbook resource, the job schedule id linked to this runbook will be automatically updated, and the old job schedule id should be NOT FOUND then.

The current implementation introduced in #7555 will try to delete all job schedules linked to the runbook and then it need a next terraform apply to recreate these job schedules. This is not neccessary at first, and it make the terraform plan can not report what terraform apply really does, it makes users confused when update a runbook.

This PR is to replace the azurerm_automation_job_schedule resource id with a customized one. The new one contains the runbook name and the schedule resource name, so we can query the job schdule id in runtime and made the id responded from API as computed attribute resource_manager_id. this PR also introduces a schema migration to make sure not break exsiting resources.

the old resource id format: terraform import azurerm_automation_job_schedule.example /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/jobSchedules/10000000-1001-1001-1001-000000000001
the new resourec id format: /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/runBook/book1/schedule/schedule1

resolves: #8634

--- PASS: TestAccAutomationJobSchedule_basic (136.84s)
--- PASS: TestAccAutomationJobSchedule_complete (198.69s)
--- PASS: TestAccAutomationJobSchedule_update (298.63s)
--- PASS: TestAccAutomationJobSchedule_requiresImport (209.73s)
--- PASS: TestAccAutomationRunbook_PSWorkflow (184.43s)
--- PASS: TestAccAutomationRunbook_WithDraft (184.13s)
--- PASS: TestAccAutomationRunbook_requiresImport (193.49s)
--- PASS: TestAccAutomationRunbook_PSWorkflowWithHash (141.74s)
--- PASS: TestAccAutomationRunbook_PSWithContent (186.88s)
--- PASS: TestAccAutomationRunbook_PSWorkflowWithoutUri (177.20s)
--- PASS: TestAccAutomationRunbook_withJobSchedule (543.13s)
--- PASS: TestAccAutomationRunbook_withJobScheduleResourceAndUpdate (211.70s)

Test Pass in TC

magodo

Thanks for this PR!
I've taken a look through and left some comments inline. The main ask is that this solution is built on an assumption that the schedules linked to a runbook must have different names, would you please help confirm whether this is the case?

internal/services/automation/migration/automation_job_schedule_migration_v0_to_v1.go

internal/services/automation/automation_runbook_resource_test.go

internal/services/automation/automation_runbook_resource.go

internal/services/automation/automation_job_schedule_resource_test.go

internal/services/automation/automation_job_schedule_resource.go

wuxu92 · 2023-07-03T06:35:27Z

The Schedule name should be unique inside the same automation account, and there MUST be only 1 link between the runbook and the schedule. so the new id schema should work.

magodo

Thanks for this PR!
I have left some additional mostly minor comments that once addressed this should be good to merge 👍

internal/services/automation/automation_job_schedule_resource.go

internal/services/automation/automation_job_schedule_resource_test.go

internal/services/automation/automation_runbook_resource.go

internal/services/automation/automation_runbook_resource_test.go

internal/services/automation/automation_runbook_resource.go

website/docs/r/automation_runbook.html.markdown

internal/services/automation/automation_runbook_resource.go

tombuildsstuff · 2023-07-04T10:37:35Z

internal/services/automation/resourceids.go

@@ -0,0 +1,3 @@
+package automation
+
+//go:generate go run ../../tools/generator-resource-id/main.go -path=./ -name=AutomationJobSchedule -id=/subscriptions/12345678-1234-9876-4563-123456789012/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/runBook/book1/schedule/schedule1


The functionality provided by resourceids.go shouldn't be used for new resources and will be removed once we've finished migrating off them - instead, since this is already using hashicorp/go-azure-sdk - the relevant Resource ID from hashicorp/go-azure-sdk should be used instead

the Resource ID in go-azure-sdk cannot work as expect in the issue of this PR is tring to fix. because the job schedule id will be updated automatically when the runbook resource updated. so I create this new job schedule ID Format to represent the link between the runbook and schedule.

I updated this PR to fix some logic and tested it locally. Although the ID format of the resource has changed, I verified that this update should not introduce any breaking changes.

We can remove this now?

katbyte · 2024-01-30T16:38:41Z

website/docs/r/automation_job_schedule.html.markdown

@@ -67,5 +71,5 @@ The `timeouts` block allows you to specify [timeouts](https://www.terraform.io/l
 Automation Job Schedules can be imported using the `resource id`, e.g.

 ```shell
-terraform import azurerm_automation_job_schedule.example /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/jobSchedules/10000000-1001-1001-1001-000000000001
+terraform import azurerm_automation_job_schedule.example /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/runBook/book1/schedule/schedule1


Is there a reason we are creating a new "fake" resource id instead of creating a composite resource id? ie
/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1| /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/runbooks/Get-AzureVMTutorial

this is something we are looking to implement first class support for soon and will work with the up coming casing fixes - see https://github.com/hashicorp/go-azure-helpers/pull/208/files

‌‌‌‌‌‌A composite resource ID would definitely be effective. Is there a plan to merge PR 208? I will update this PR accordingly.

Yes, hopefully it'll be merged in the next couple weeks

@wuxu92 - the above PR has been merged

updated this PR with CompositeResourceID but need new version/tag of go-azure-helpers to be released. so I'm going to make this PR draft before new version of go-azure-helpers available.

tombuildsstuff · 2024-04-05T15:15:25Z

internal/services/automation/automation_job_schedule_resource.go

+	accountID := jobschedule.NewAutomationAccountID(id.First.SubscriptionId, id.First.ResourceGroupName, id.First.AutomationAccountName)
+	filter := fmt.Sprintf("properties/schedule/name eq '%s' and properties/runbook/name eq '%s'", id.First.ScheduleName, id.Second.RunbookName)
+	jsList, err := client.ListByAutomationAccountComplete(ctx, accountID, jobschedule.ListByAutomationAccountOperationOptions{Filter: &filter})
+	if err != nil {
+		return nil, fmt.Errorf("loading Automation Account %q Job Schedule List: %+v", accountID.AutomationAccountName, err)
+	}


why is this needed? The Go SDK has a Get method?

func (c RunbookClient) Get(ctx context.Context, id RunbookId) (result GetOperationResponse, err error) {

We need to get the job schedule here, but we don't know the job schedule id from the composite id, because the job schedule id can be updated when the related account or runbook is updated. So we need the List operation to get the correct job schedule id here.

tombuildsstuff · 2024-04-05T15:15:41Z

internal/services/automation/resourceids.go

@@ -0,0 +1,3 @@
+package automation


can we remove this file?

katbyte · 2024-04-05T15:09:09Z

internal/services/automation/automation_job_schedule_resource.go

+	runbookID := runbook.NewRunbookID(subscriptionId, resourceGroup, accountName, runbookName)
+	id := jobschedule.NewJobScheduleID(subscriptionId, resourceGroup, accountName, jobScheduleUUID.String())
+
+	// tfID := parse.NewAutomationJobScheduleID(id.SubscriptionId, id.ResourceGroupName, id.AutomationAccountName, runbookName, scheduleName)


we can remove this line now?

katbyte · 2024-04-05T15:10:36Z

internal/services/automation/resourceids.go

@@ -0,0 +1,3 @@
+package automation
+
+//go:generate go run ../../tools/generator-resource-id/main.go -path=./ -name=AutomationJobSchedule -id=/subscriptions/12345678-1234-9876-4563-123456789012/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/runBook/book1/schedule/schedule1


We can remove this now?

katbyte · 2024-04-05T15:11:15Z

website/docs/r/automation_job_schedule.html.markdown

@@ -10,6 +10,8 @@ description: |-

 Links an Automation Runbook and Schedule.

+~> **NOTE** AzureRM provides this stand-alone [azurerm_automation_job_schedule](automation_job_schedule.html.markdown) and an inlined `job_schdule` property in [azurerm_runbook](automation_runbook.html.markdown) to manage the job schedules. At this time you should choose one of them to manage the job schedule resources.


Suggested change

~> **NOTE** AzureRM provides this stand-alone [azurerm_automation_job_schedule](automation_job_schedule.html.markdown) and an inlined `job_schdule` property in [azurerm_runbook](automation_runbook.html.markdown) to manage the job schedules. At this time you should choose one of them to manage the job schedule resources.

~> **NOTE** AzureRM provides this stand-alone [azurerm_automation_job_schedule](automation_job_schedule.html.markdown) and an inlined `job_schdule` property in [azurerm_runbook](automation_runbook.html.markdown) to manage the job schedules. You can only make use of one of these methods to manage a job schedule.

catriona-m

Hi @wuxu92 I had a look though this and left a couple of comments inline, but once those are addressed we can take another look. Thanks!

catriona-m · 2024-05-03T10:23:39Z

internal/services/automation/automation_job_schedule_resource.go

@@ -107,43 +125,24 @@ func resourceAutomationJobScheduleCreate(d *pluginsdk.ResourceData, meta interfa
 		jobScheduleUUID = uuid.FromStringOrNil(jobScheduleID.(string))
 	}

-	id := jobschedule.NewJobScheduleID(subscriptionId, d.Get("resource_group_name").(string), d.Get("automation_account_name").(string), jobScheduleUUID.String())
+	// accountID := automationaccount.NewAutomationAccountID(subscriptionId, resourceGroup, accountName)


should this be removed?

catriona-m · 2024-05-03T10:38:36Z

internal/services/automation/automation_job_schedule_resource.go

+	js, err := GetJobScheduleFromTFID(ctx, client, tfID)
 	if err != nil {
-		if response.WasNotFound(resp.HttpResponse) {
-			d.SetId("")
-			return nil
-		}
-		return fmt.Errorf("making Read request on %s: %+v", *id, err)
+		return err
+	}
+	if js == nil {
+		d.SetId("")
+		return nil
 	}


In the case where the resource group has been deleted outside of terraform, this will continually give us an error when trying to list the job schedules and it is never marked as gone. We should update this to cover this case.

updated GetJobScheduleFromTFID to return nil error if API response is 404.

catriona-m · 2024-05-03T10:43:54Z

website/docs/r/automation_runbook.html.markdown

-* `draft` - (Optional) A `draft` block as defined below .
+* `draft` - (Optional) A `draft` block as defined below.
+
+* `job_schedule` - (Optional) Onr or more `job_schedule` block as defined below.


Suggested change

* `job_schedule` - (Optional) Onr or more `job_schedule` block as defined below.

* `job_schedule` - (Optional) One or more `job_schedule` block as defined below.

catriona-m

Thanks for making the changes @wuxu92 - I tested this and still got an error in some cases, could you take another look? Thanks!

internal/services/automation/automation_job_schedule_resource.go

catriona-m

Thanks for fixing this up @wuxu92 - I spotted a couple of minor things on looking through this again but otherwise this is looking good. Thanks!

catriona-m · 2024-06-11T10:33:02Z

website/docs/r/automation_job_schedule.html.markdown

@@ -67,5 +71,5 @@ The `timeouts` block allows you to specify [timeouts](https://www.terraform.io/l
 Automation Job Schedules can be imported using the `resource id`, e.g.

 ```shell
-terraform import azurerm_automation_job_schedule.example /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/jobSchedules/10000000-1001-1001-1001-000000000001
+terraform import azurerm_automation_job_schedule.example "/subscriptions/85b3dbca-5974-4067-9669-67a141095a76/resourceGroups/acctestRG-auto-240326135902419761/providers/Microsoft.Automation/automationAccounts/acctestAA-240328091346495956/schedules/acctestAS-240328091346495956|/subscriptions/85b3dbca-5974-4067-9669-67a141095a76/resourceGroups/acctestRG-auto-240326135902419761/providers/Microsoft.Automation/automationAccounts/acctestAA-240328091346495956/runbooks/Output-HelloWorld"


Suggested change

terraform import azurerm_automation_job_schedule.example "/subscriptions/85b3dbca-5974-4067-9669-67a141095a76/resourceGroups/acctestRG-auto-240326135902419761/providers/Microsoft.Automation/automationAccounts/acctestAA-240328091346495956/schedules/acctestAS-240328091346495956|/subscriptions/85b3dbca-5974-4067-9669-67a141095a76/resourceGroups/acctestRG-auto-240326135902419761/providers/Microsoft.Automation/automationAccounts/acctestAA-240328091346495956/runbooks/Output-HelloWorld"

terraform import azurerm_automation_job_schedule.example "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/schedules/schedule1|/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/runbooks/runbook1"

catriona-m · 2024-06-11T10:34:27Z

internal/services/automation/automation_job_schedule_resource.go

+
+	id, err := jobschedule.ParseJobScheduleID(pointer.From(js.Id))
+	if err != nil {
+		return fmt.Errorf("parse jobSchedule id %s: %v", pointer.From(js.Id), err)


Suggested change

return fmt.Errorf("parse jobSchedule id %s: %v", pointer.From(js.Id), err)

return err

github-actions bot added documentation service/automation size/XL labels Jun 14, 2023

mybayern1974 assigned magodo and unassigned magodo Jun 30, 2023

magodo requested changes Jun 30, 2023

View reviewed changes

magodo added the enhancement label Jun 30, 2023

github-actions bot added waiting-response and removed waiting-response labels Jun 30, 2023

magodo requested changes Jul 4, 2023

View reviewed changes

github-actions bot added the waiting-response label Jul 4, 2023

tombuildsstuff reviewed Jul 4, 2023

View reviewed changes

tombuildsstuff added the breaking-change label Jul 4, 2023

github-actions bot added dependencies waiting-response and removed waiting-response dependencies labels Jul 10, 2023

wuxu92 force-pushed the autojobschedule branch from 9be75ad to 48e1117 Compare November 8, 2023 10:22

github-actions bot added state-migration waiting-response and removed waiting-response labels Nov 8, 2023

wuxu92 force-pushed the autojobschedule branch from 7713e60 to b783f72 Compare November 27, 2023 09:18

tombuildsstuff self-assigned this Dec 7, 2023

katbyte requested changes Jan 30, 2024

View reviewed changes

github-actions bot added the waiting-response label Jan 30, 2024

wuxu92 force-pushed the autojobschedule branch from b783f72 to 52cde2e Compare March 21, 2024 09:50

use composite id for job schdule

b19ac4e

wuxu92 force-pushed the autojobschedule branch from 11444e5 to b19ac4e Compare March 28, 2024 01:12

github-actions bot removed the waiting-response label Mar 28, 2024

update document

5c9703a

wuxu92 marked this pull request as ready for review March 28, 2024 01:36

stephybun added this to the v4.0.0 milestone Apr 3, 2024

tombuildsstuff reviewed Apr 5, 2024

View reviewed changes

internal/services/automation/resourceids.go Outdated

@@ -0,0 +1,3 @@

package automation

Copy link

Member

tombuildsstuff Apr 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we remove this file?

github-actions bot added the waiting-response label Apr 5, 2024

katbyte requested changes Apr 5, 2024

View reviewed changes

update format ci and remove unsed resource id go:generate

194f32e

github-actions bot removed the waiting-response label Apr 7, 2024

tolower key name of job schedule in runbook resource

9a3432a

katbyte unassigned tombuildsstuff Apr 23, 2024

catriona-m self-assigned this Apr 23, 2024

catriona-m requested changes May 3, 2024

View reviewed changes

github-actions bot added the waiting-response label May 3, 2024

code format and skip error if not found when list jobs

cf21e94

github-actions bot removed the waiting-response label May 7, 2024

catriona-m requested changes May 10, 2024

View reviewed changes

internal/services/automation/automation_job_schedule_resource.go Show resolved Hide resolved

github-actions bot added the waiting-response label May 10, 2024

Merge branch 'main' into autojobschedule

6dc90a8

github-actions bot removed the waiting-response label May 11, 2024

github-actions bot added the waiting-response label May 23, 2024

Merge remote-tracking branch 'origin/main' into autojobschedule

637b3dc

github-actions bot removed the waiting-response label May 24, 2024

wuxu92 mentioned this pull request May 24, 2024

Dependency: update go-azure-sdk to v0.20240523.1095414 #26090

Merged

Merge branch 'main' into autojobschedule

e7fabe2

catriona-m requested changes Jun 11, 2024

View reviewed changes

github-actions bot added the waiting-response label Jun 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`azurerm_automation_job_schedule`: update jobSchedule resource id format #22164

`azurerm_automation_job_schedule`: update jobSchedule resource id format #22164

wuxu92 commented Jun 14, 2023 •

edited

magodo left a comment

wuxu92 commented Jul 3, 2023

magodo left a comment

tombuildsstuff Jul 4, 2023

wuxu92 Jul 10, 2023

wuxu92 Nov 13, 2023

katbyte Apr 5, 2024

katbyte Jan 30, 2024

wuxu92 Feb 26, 2024

katbyte Feb 27, 2024

katbyte Mar 20, 2024

wuxu92 Mar 21, 2024

tombuildsstuff Mar 27, 2024

tombuildsstuff Apr 5, 2024

wuxu92 Apr 7, 2024

tombuildsstuff Apr 5, 2024

katbyte Apr 5, 2024

wuxu92 Apr 7, 2024

katbyte Apr 5, 2024

katbyte Apr 5, 2024

wuxu92 Apr 7, 2024

catriona-m left a comment

catriona-m May 3, 2024

wuxu92 May 7, 2024

catriona-m May 3, 2024

wuxu92 May 7, 2024

catriona-m May 3, 2024

catriona-m left a comment

catriona-m left a comment

catriona-m Jun 11, 2024

catriona-m Jun 11, 2024

		@@ -0,0 +1,3 @@
		package automation

		//go:generate go run ../../tools/generator-resource-id/main.go -path=./ -name=AutomationJobSchedule -id=/subscriptions/12345678-1234-9876-4563-123456789012/resourceGroups/group1/providers/Microsoft.Automation/automationAccounts/account1/runBook/book1/schedule/schedule1

		@@ -10,6 +10,8 @@ description: \|-

		Links an Automation Runbook and Schedule.

		~> NOTE AzureRM provides this stand-alone [azurerm_automation_job_schedule](automation_job_schedule.html.markdown) and an inlined `job_schdule` property in [azurerm_runbook](automation_runbook.html.markdown) to manage the job schedules. At this time you should choose one of them to manage the job schedule resources.

	* `job_schedule` - (Optional) Onr or more `job_schedule` block as defined below.
	* `job_schedule` - (Optional) One or more `job_schedule` block as defined below.

	return fmt.Errorf("parse jobSchedule id %s: %v", pointer.From(js.Id), err)
	return err

azurerm_automation_job_schedule: update jobSchedule resource id format #22164

Are you sure you want to change the base?

azurerm_automation_job_schedule: update jobSchedule resource id format #22164

Conversation

wuxu92 commented Jun 14, 2023 • edited

magodo left a comment

Choose a reason for hiding this comment

wuxu92 commented Jul 3, 2023

magodo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

catriona-m left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

catriona-m left a comment

Choose a reason for hiding this comment

catriona-m left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

`azurerm_automation_job_schedule`: update jobSchedule resource id format #22164

`azurerm_automation_job_schedule`: update jobSchedule resource id format #22164

wuxu92 commented Jun 14, 2023 •

edited