Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: Plugin did not respond... Is there a way to implement on TF stream tables on external tables for snowflake? #17927

Open
JU1N-CPA opened this issue Apr 20, 2024 · 0 comments
Labels

Comments

@JU1N-CPA
Copy link

Question

I've been having an issue trying to implement a stream table on an external table using tf. I have an error, but I would like to ask if there is a way to solve this before reporting it as a bug. This is my set-up

#snowflake external table
resource "snowflake_external_table" "this" {
  name = "RAW_CORE_CITIES"
  database = snowflake_database.db.name
  schema = snowflake_schema.schema.name
  file_format =  "FORMAT_NAME = ${snowflake_database.db.name}.${snowflake_schema.schema.name}.${snowflake_file_format.this.name}"
  #location = snowflake_stage.this.name
  location = "@${snowflake_database.db.name}.${snowflake_schema.schema.name}.${snowflake_stage.this.name}"
  #location = format("@%s.%s.%s",snowflake_database.db.name,snowflake_schema.schema.name,snowflake_stage.this.name)
  auto_refresh = true
  partition_by = [ "date_today" ]

  column {
  name = upper("date_today")
  type = "date"
  as = "TO_DATE(SPLIT_PART(metadata$filename, '/', 2), 'YYYY-MM-DD')"
  }

  column {
    name = upper("operation_status")
    type = "string"
    as = "SPLIT_PART(metadata$filename, '/', 3)"
  }

  column {
    name = upper("id")
    type = "string"
    as = "(VALUE:_id::string)"
  }

  column {
    name = upper("cityId")
    type = "string"
    as = "(VALUE:cityId::string)"
  }

  column {
    name = upper("stateId")
    type = "string"
    as = "(VALUE:stateId::string)"
  }

  column {
    name = upper("countryId")
    type = "string"
    as = "(VALUE:countryId::string)"
  }

  column {
    name = upper("cityName")
    type = "string"
    as = "(VALUE:cityName::string)"
  }

  column {
    name = upper("stateName")
    type = "string"
    as = "(VALUE:stateName::string)"
  }

  column {
    name = upper("countryName")
    type = "string"
    as = "(VALUE:countryName::string)"
  }

  column {
    name = upper("lat")
    type = "double"
    as = "(VALUE:location.lat::double)"
  }

  column {
    name = upper("lon")
    type = "double"
    as = "(VALUE:location.lon::double)"
  }

  column {
    name = upper("timestamp")
    type = "TIMESTAMP_NTZ"
    as = "(VALUE:timestamp::TIMESTAMP_NTZ)"
  }

  depends_on = [ snowflake_storage_integration.this,snowflake_stage.this,aws_iam_role.snowflake_aws_integration_role ]

}

# snowflake stream table
resource "snowflake_stream" "this" {
  comment = "Streaming table on external table"
  database = snowflake_database.db.name
  schema   = snowflake_schema.schema.name
  name     = upper("STREAM_CORE_CITIES")
  on_table    = snowflake_external_table.this.name
  insert_only = true
}

I have this current error

snowflake_stream.this: Creating...

│ Error: Plugin did not respond

│ The plugin encountered an error, and failed to respond to the plugin6.(*GRPCProvider).ApplyResourceChange
│ call. The plugin logs may contain more details.

image

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant