Add Azure SQL Managed Instance Module #1307

sweanan · 2023-06-14T14:24:00Z

Description

This PR adds below:
Examples SQL MI : examples/azure/sqlmanagedinstance
AzureSQL Managed Instance Module under: modules/azure/sqlmanagedinstance*
Corresponding tests are added: tests/azure/terraform-azure-sqlmanagedinstanceexample

NOTE:
As described in the README.md for sqlmanagedinstance, the deployment for this module take more that 4-6 hours as per documents, so the tests are excluded from the workflow by adding a tag !azure so that this test file will not be picked up from the ci workflow

Test with sqlmanaged instance long running:
https://github.com/sweanan/terratest/actions/runs/5322721656/jobs/9639554889
TestTerraformAzureSQLManagedInstanceExample 2023-06-20T14:04:18Z logger.go:66: azurerm_mssql_managed_instance.sqlmi_mi: Still creating... [1h5m20s elapsed] TestTerraformAzureSQLManagedInstanceExample 2023-06-20T14:04:28Z logger.go:66: azurerm_mssql_managed_instance.sqlmi_mi: Still creating... [1h5m30s elapsed]

Test excluding sqlmanaged instance : terraform_azure_sqlmanagedinstance_example_test.go
https://github.com/sweanan/terratest/actions/runs/5350335787/jobs/9702803292

HadwaAbdelhalem · 2023-06-19T23:04:32Z

Hi @sweanan can you drop a link for the CI execution on the PR branch.?

HadwaAbdelhalem · 2023-06-19T23:06:15Z

test/azure/terraform_azure_sqlmanagedinstance_example_test.go

+	t.Parallel()
+
+	uniquePostfix := strings.ToLower(random.UniqueId())
+	expectedLocation := "West US2"


should this be westus2

sweanan · 2023-06-20T12:42:13Z

[Microsoft CI Bot] TL;DR; failure 🤦

You can check the status of the CI Pipeline logs here ; https://github.com/sweanan/terratest/actions/runs/5322652361

modules/azure/sql_managedinstance.go

HadwaAbdelhalem · 2023-06-21T22:02:10Z

test/azure/terraform_azure_sqlmanagedinstance_example_testlong.go

@@ -0,0 +1,63 @@
+// SQL Managed Instance takes 6-8 hours for deployment, Exclude this from the worflow by naming it as *_testlong.go


I would rename the file to a be a valid test file 'sql_mangedinstace_test' and since get rid of all the tags with a minor change to the azure ci workflow to run tests based on the azure tag this would exclude the test from getting executed by using 'go test ./azure/* -v -timeout 90m --tags=azure'

terratest/.github/workflows/ci.yml

Line 134 in b8689cd

go test ./azure/* -v -timeout 90m

Thanks Hadwa updated it

sweanan · 2023-06-22T22:03:27Z

[Microsoft CI Bot] TL;DR; failure 🤦

You can check the status of the CI Pipeline logs here ; https://github.com/sweanan/terratest/actions/runs/5350335787

HadwaAbdelhalem

Thanks @sweanan ,LGTM!

denis256 · 2023-06-23T20:40:49Z

modules/azure/sql_managedinstance_test.go

+	managedInstanceName := ""
+	subscriptionID := ""
+
+	_, err := GetManagedInstanceE(t, subscriptionID, resGroupName, managedInstanceName)


modules/azure/sql_managedinstance_test.go:39:65: too many arguments in call to GetManagedInstanceE have (*"testing".T, string, string, string) want (string, string, string)

denis256 · 2023-06-26T09:44:44Z

test/azure/terraform_azure_sqlmanagedinstance_example_test.go

@@ -0,0 +1,67 @@
+//go:build !azure
+// +build !azure


I think should be used same build parameters as for other azure tests

//go:build azure // +build azure

Now, the regular CICD test job tries to run TestTerraformAzureSQLManagedInstanceExample and fails since az is not configured, other Azure tests aren't attempted to be executed

TestTerraformAzureSQLManagedInstanceExample 2023-06-25T21:29:15Z logger.go:66: ╵ TestTerraformAzureSQLManagedInstanceExample 2023-06-25T21:29:15Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; ╷ │ Error: building AzureRM Client: please ensure you have installed Azure CLI version 2.0.79 or newer. Error parsing json result from the Azure CLI: launching Azure CLI: exec: "az": executable file not found in $PATH. │ │ with provider["registry.terraform.io/hashicorp/azurerm"], │ on main.tf line 12, in provider "azurerm": │ 12: provider "azurerm" { │ ╵} apply.go:15: Error Trace: /home/circleci/project/test/azure/apply.go:15 /home/circleci/project/test/azure/terraform_azure_sqlmanagedinstance_example_test.go:46 Error: Received unexpected error: FatalError{Underlying: error while running command: exit status 1; ╷ │ Error: building AzureRM Client: please ensure you have installed Azure CLI version 2.0.79 or newer. Error parsing json result from the Azure CLI: launching Azure CLI: exec: "az": executable file not found in $PATH. │ │ with provider["registry.terraform.io/hashicorp/azurerm"], │ on main.tf line 12, in provider "azurerm": │ 12: provider "azurerm" { │ ╵} Test: TestTerraformAzureSQLManagedInstanceExample

circle-test.txt

References:

https://github.com/gruntwork-io/terratest/blob/master/test/azure/terraform_azure_aci_example_test.go#L2

Hi @denis256, just a quick question, Can you please confirm, the workflow you are triggering is by using the .circleci/config.yml correct?

Hi, correct

Hi @denis256, looking at the circleci yaml file, I see that the azure tests are only compiled and not executed. I am not sure which part of the workflow triggered the execution of the azure tests. Can you please point me to the test stage/task where is got triggered.

Also can you share how/where a subset of the tests are mared as short? I see in the log file you shared earlier that the test cmd is using the -short flag ' go test -v -timeout 45m -parallel 128 -p 1 ./... -short'

Step:

- run: run-go-tests --packages "-p 1 ./..." | tee /tmp/logs/test_output.log

Included execution of TestTerraformAzureSQLManagedInstanceExample

I suspect it is because of(I need to do more tests to validate):

//go:build !azure // +build !azure

Other tests have:

//go:build azure // +build azure

In https://github.com/gruntwork-io/terratest/tree/add-sqlmi-module-azure-test
I set

//go:build azure // +build azure

and TestTerraformAzureSQLManagedInstanceExample wasn't executed

@denis256 I updated the PR to skip the test for the short executions, and update the tags as well. Testing locally the test has not been picked up by the go test cmd. please take a look when you get a chance and share the Circle ci log file as well.

sweanan · 2023-07-05T22:26:17Z

[Microsoft CI Bot] TL;DR; failure 🤦

You can check the status of the CI Pipeline logs here ; https://github.com/sweanan/terratest/actions/runs/5469099387

sweanan · 2023-07-05T23:22:02Z

[Microsoft CI Bot] TL;DR; failure 🤦

You can check the status of the CI Pipeline logs here ; https://github.com/sweanan/terratest/actions/runs/5469531305

denis256 · 2023-07-06T19:10:22Z

Hi, looks like imports should be fixed...

[INFO] Initializing environment for https://github.com/gruntwork-io/pre-commit.
goimports................................................................Failed
- hook id: goimports
- files were modified by this hook

test/azure/terraform_azure_sqlmanagedinstance_example_test.go

Terraform fmt............................................................Passed
test-interfaces-used.....................................................Passed

denis256

Failing tests aren't related to implemented changes, most of tests I already fixed in master

sweanan added 2 commits June 13, 2023 14:35

initial draft

9dd670d

update sqlmi code after complete testing

c251303

sweanan requested review from zackproser and denis256 as code owners June 14, 2023 14:24

sweanan added 4 commits June 14, 2023 12:40

added a note for duration

5896a68

update for fmt

3aee5cc

update location as well as lint issue for ci

6516905

update lint error

84219bd

HadwaAbdelhalem reviewed Jun 19, 2023

View reviewed changes

sweanan added 3 commits June 20, 2023 09:42

renamed file to exclude from workflow

283194c

revert back the ci changes

b75dcd6

update readme

02698fe

denis256 reviewed Jun 21, 2023

View reviewed changes

modules/azure/sql_managedinstance.go Show resolved Hide resolved

HadwaAbdelhalem reviewed Jun 21, 2023

View reviewed changes

sweanan added 3 commits June 22, 2023 06:31

Merge branch 'master' into add-sqlmi-module

811e666

rename the test file

5b658ca

update ci.yml

0298721

added the exists helper function

e158951

HadwaAbdelhalem approved these changes Jun 23, 2023

View reviewed changes

denis256 reviewed Jun 23, 2023

View reviewed changes

changed the api signature

c4be13c

denis256 reviewed Jun 26, 2023

View reviewed changes

HadwaAbdelhalem added 2 commits July 5, 2023 14:18

mark the test as long running test

91c117a

update the test tag

7d9d89d

fixed the imports

0fd2dfd

denis256 approved these changes Jul 7, 2023

View reviewed changes

denis256 merged commit d58da78 into gruntwork-io:master Jul 7, 2023
2 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Azure SQL Managed Instance Module #1307

Add Azure SQL Managed Instance Module #1307

sweanan commented Jun 14, 2023 •

edited

HadwaAbdelhalem commented Jun 19, 2023

HadwaAbdelhalem Jun 19, 2023

sweanan Jun 20, 2023

sweanan commented Jun 20, 2023

HadwaAbdelhalem Jun 21, 2023 •

edited

sweanan Jun 23, 2023

sweanan commented Jun 22, 2023

HadwaAbdelhalem left a comment

denis256 Jun 23, 2023

sweanan Jun 23, 2023

denis256 Jun 26, 2023

sweanan Jun 26, 2023 •

edited

denis256 Jun 27, 2023

HadwaAbdelhalem Jun 30, 2023 •

edited

denis256 Jun 30, 2023

denis256 Jul 3, 2023

HadwaAbdelhalem Jul 5, 2023

sweanan commented Jul 5, 2023

sweanan commented Jul 5, 2023

denis256 commented Jul 6, 2023

denis256 left a comment

		@@ -0,0 +1,63 @@
		// SQL Managed Instance takes 6-8 hours for deployment, Exclude this from the worflow by naming it as *_testlong.go

Add Azure SQL Managed Instance Module #1307

Add Azure SQL Managed Instance Module #1307

Conversation

sweanan commented Jun 14, 2023 • edited

Description

HadwaAbdelhalem commented Jun 19, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sweanan commented Jun 20, 2023

HadwaAbdelhalem Jun 21, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sweanan commented Jun 22, 2023

HadwaAbdelhalem left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sweanan Jun 26, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HadwaAbdelhalem Jun 30, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sweanan commented Jul 5, 2023

sweanan commented Jul 5, 2023

denis256 commented Jul 6, 2023

denis256 left a comment

Choose a reason for hiding this comment

sweanan commented Jun 14, 2023 •

edited

HadwaAbdelhalem Jun 21, 2023 •

edited

sweanan Jun 26, 2023 •

edited

HadwaAbdelhalem Jun 30, 2023 •

edited