Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fields for specifying GKE logging variant at the cluster-wide and node pool level. #13049

Conversation

modular-magician
Copy link
Collaborator

Fixes #12667

This PR implements the feature request from Add GKE logging variant field for increasing log agent throughput #12667. By adding a logging_variant field within the node_pool_defaults, GKE users will be able to select a cluster-level default value for the logging agent of the node pools in a cluster. For example, by specifying

resource "google_container_cluster" "with_logging_variant_node_pool_default" {
  name               = "example-cluster"
  location           = "us-central1-f"
  initial_node_count = 1

  node_pool_defaults {
    node_config_defaults {
      logging_variant = "MAX_THROUGHPUT"
    }
  }
}

every node pool (i.e. the default node pool) in the cluster will have the max throughput logging agent configured by default (see the GKE docs for more information).

GKE users will also be able to select a logging variant at the node pool level. For example, by specifying

resource "google_container_cluster" "with_logging_variant_node_pool_default" {
  name               = "example-cluster"
  location           = "us-central1-f"
  initial_node_count = 1

  node_pool_defaults {
    node_config_defaults {
      logging_variant = "DEFAULT"
    }
  }
}
resource "google_container_node_pool" "with_high_throughput_logging_variant" {
  name    = "example-node-pool"
  cluster = google_container_cluster.with_logging_variant_node_pool_default.name
  node_config {
    logging_variant = "MAX_THROUGHPUT"
  }
}

node pools in the cluster (e.g. the default node pool) will have the default logging agent configured (see the GKE docs for more information), but the specified node pool will have the max throughput agent.

I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).

  • Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.

The output of make test is

sh -c "'/usr/local/google/home/giulianosider/go/src/github.com/hashicorp/terraform-provider-google/scripts/gofmtcheck.sh'"
==> Checking that code complies with gofmt requirements...
go vet
go generate  ./...
go test  -timeout=30s $(go list ./... | grep -v github.com/hashicorp/terraform-provider-google/scripts)
?   	github.com/hashicorp/terraform-provider-google	[no test files]
ok  	github.com/hashicorp/terraform-provider-google/google	22.793s
?   	github.com/hashicorp/terraform-provider-google/version	[no test files]

but only after I run make fmt. There were 2 files unrelated to my change that required Go formatting.

  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).

I added the following tests:

  • TestAccContainerCluster_withNoSpecifiedLoggingVariant

  • TestAccContainerCluster_withDefaultLoggingVariant

  • TestAccContainerCluster_withMaxThroughputLoggingVariant

  • TestAccContainerCluster_withLoggingVariantUpdates

  • TestAccNodePool_withNoSpecifiedLoggingVariant

  • TestAccNodePool_withDefaultLoggingVariant

  • TestAccNodePool_withMaxThroughputLoggingVariant

  • TestAccNodePool_withLoggingVariantUpdates

    • Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).

I ran the tests that I added myself (see previous items). When I ran TF_LOG=TRACE make testacc TEST=./google TESTARGS='-run=TestAccContainerCluster' | tee output.log in the terraform-provider-google repo, I ran into some failures apparently unrelated to my change, but I'm as of yet unable to figure out why. The TRACE level logs suggested by the command in the README generate about 475 MB of log output, and it's difficult to find anything unless you know what to search for (i.e. "Check failed").

Release Note Template for Downstream PRs (will be copied)

container: Added `node_pool_defaults.node_config_defaults.logging_variant`, `node_pool.node_config.logging_variant`, and `node_config.logging_variant` to `google_container_cluster`.
container: Added `node_config.logging_variant` to `google_container_node_pool`.

Derived from GoogleCloudPlatform/magic-modules#6744

…ults for GKE clusters. (hashicorp#6744)

This PR implements the feature request from [Add GKE logging variant field for increasing log agent throughput hashicorp#12667](hashicorp#12667).

By adding a logging_variant field within the node_pool_defaults, GKE users will be able to select a cluster-wide default value for the logging agent of the node pools in a cluster. For example, by specifying
```terraform
resource "google_container_cluster" "with_logging_variant_node_pool_default" {
  name               = "example-cluster"
  location           = "us-central1-f"
  initial_node_count = 1

  node_pool_defaults {
    node_config_defaults {
      logging_variant = "MAX_THROUGHPUT"
    }
  }
}
```
every newly created node pool in the cluster will have the max throughput logging agent unless this is explicitly overridden at the node pool level (see the [GKE docs](https://cloud.google.com/stackdriver/docs/solutions/gke/managing-logs#high_throughput_for_all_nodes_in_a_cluster) for more information).

GKE users will also be able to select a logging variant at the node pool level. For example, by specifying
```terraform
resource "google_container_cluster" "with_logging_variant_node_pool_default" {
  name               = "example-cluster"
  location           = "us-central1-f"
  initial_node_count = 1

  node_pool_defaults {
    node_config_defaults {
      logging_variant = "DEFAULT"
    }
  }
}
resource "google_container_node_pool" "with_high_throughput_logging_variant" {
  name    = "example-node-pool-0"
  cluster = google_container_cluster.with_logging_variant_node_pool_default.name
}
resource "google_container_node_pool" "with_high_throughput_logging_variant" {
  name    = "example-node-pool-1"
  cluster = google_container_cluster.with_logging_variant_node_pool_default.name
  node_config {
    logging_variant = "MAX_THROUGHPUT"
  }
}
```
example-node-pool-0 (as well as the default node pool) will have the default logging agent (see the [GKE docs](https://cloud.google.com/stackdriver/docs/solutions/gke/managing-logs#high_throughput_for_all_nodes_in_a_cluster) for more information), but example-node-pool-1 will have the max throughput agent.

Signed-off-by: Modular Magician <magic-modules@google.com>
@modular-magician modular-magician merged commit eca88a2 into hashicorp:main Nov 16, 2022
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add GKE logging variant field for increasing log agent throughput
1 participant