Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run, compile, test should use same dbt profile. #170

Open
daniel-bartley opened this issue Dec 21, 2023 · 0 comments
Open

run, compile, test should use same dbt profile. #170

daniel-bartley opened this issue Dec 21, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@daniel-bartley
Copy link

Describe the bug
compile, test, run should all you the same profile.
bigquery project "foo" should only be referenced if is in my profiles.yml file.
Using oauth method to connect

To Reproduce

$ tree -a .
.
|-- monorepo
| |-- .dbt-checkpoint.yaml
| |-- .pre-commit-config.yaml
| -- src | -- dbt
| |-- dbt_project.yml
| |-- models.sql
| | -- example.sql | -- profiles.yml
.
-- src -- dbt
|-- dbt_project.yml
|-- models.sql
| -- example.sql -- profiles.yml

`--.pre-commit-config.yaml
fail-fast

repos:

terminal
$ git commit -m "chore: dbt-checkpoint"
don't commit to branch...................................................Passed
check toml...........................................(no files to check)Skipped
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
dbt compile..............................................................Passed
dbt test.................................................................Failed

  • hook id: dbt-test
  • exit code: 1
  • files were modified by this hook

Executing cmd: dbt test -m tag:unit-test src/dbt/tests/macros/macro_boolean_to_yes_no_with_null_value.sql src/dbt/tests/macros/macro_boolean_to_yes_no_without_null_value.sql --project-dir src/dbt/
07:24:45 Running with dbt=1.6.8
07:24:46 Registered adapter: bigquery=1.6.9
07:24:46 Unable to do partial parsing because profile has changed
07:24:46 Unable to do partial parsing because env vars used in profiles.yml have changed
07:25:03 Found n models, n analyses, n tests, 14 seeds, 189 sources, 117 exposures, 0 metrics, 1066 macros, 0 groups, 0 semantic models
07:25:03 The selection criterion 'src/dbt/tests/macros/macro_boolean_to_yes_no_with_null_value.sql' does not match any nodes
07:25:03 The selection criterion 'src/dbt/tests/macros/macro_boolean_to_yes_no_without_null_value.sql' does not match any nodes
07:25:03
07:25:26
07:25:26 Finished running in 0 hours 0 minutes and 23.08 seconds (23.08s).
07:25:26 Encountered an error:
400 GET https://bigquery.googleapis.com/bigquery/v2/projects/foo/datasets/curated_activedirectory/tables?maxResults=100000&prettyPrint=false: The project foo has not enabled BigQuery.
07:25:26 Traceback (most recent call last):
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\cli\requires.py", line 87, in wrapper
result, success = func(*args, **kwargs)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\cli\requires.py", line 72, in wrapper
return func(*args, **kwargs)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\cli\requires.py", line 143, in wrapper
return func(*args, **kwargs)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\cli\requires.py", line 172, in wrapper
return func(*args, **kwargs)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\cli\requires.py", line 219, in wrapper
return func(*args, **kwargs)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\cli\requires.py", line 259, in wrapper
return func(*args, **kwargs)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\cli\main.py", line 864, in test
results = task.run()
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\task\runnable.py", line 469, in run
result = self.execute_with_hooks(selected_uids)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\task\runnable.py", line 429, in execute_with_hooks
self.before_run(adapter, selected_uids)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\task\run.py", line 447, in before_run
self.populate_adapter_cache(adapter, required_schemas)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\task\runnable.py", line 407, in populate_adapter_cache
adapter.set_relations_cache(self.manifest)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\adapters\base\impl.py", line 478, in set_relations_cache
self._relations_cache_for_schemas(manifest, required_schemas)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\adapters\base\impl.py", line 455, in _relations_cache_for_schemas
for relation in future.result():
File "C:\Python310\lib\concurrent\futures_base.py", line 451, in result
return self.__get_result()
File "C:\Python310\lib\concurrent\futures_base.py", line 403, in __get_result
raise self._exception
File "C:\Python310\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\utils.py", line 465, in connected
return func(*args, **kwargs)
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\adapters\bigquery\impl.py", line 368, in list_relations_without_caching
return [self._bq_table_to_relation(table) for table in all_tables] # type: ignore[misc]
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\dbt\adapters\bigquery\impl.py", line 368, in
return [self._bq_table_to_relation(table) for table in all_tables] # type: ignore[misc]
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\api_core\page_iterator.py", line 208, in _items_iter
for page in self._page_iter(increment=False):
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\api_core\page_iterator.py", line 244, in _page_iter
page = self._next_page()
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\api_core\page_iterator.py", line 373, in _next_page
response = self._get_next_page_response()
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\api_core\page_iterator.py", line 432, in _get_next_page_response
return self.api_request(
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\cloud\bigquery\client.py", line 1551, in api_request
return self._call_api(
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\cloud\bigquery\client.py", line 808, in call_api
return call()
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\api_core\retry.py", line 372, in retry_wrapped_func
return retry_target(
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\api_core\retry.py", line 207, in retry_target
result = target()
File "C:\Users\12345678\cyberda\src\dbt.venv\lib\site-packages\google\cloud_http_init
.py", line 494, in api_request
raise exceptions.from_http_response(response)
google.api_core.exceptions.BadRequest: 400 GET https://bigquery.googleapis.com/bigquery/v2/projects/foo/datasets/curated_activedirectory/tables?maxResults=100000&prettyPrint=false: The project foo has not enabled BigQuery.

`

Expected behavior
compile, test, run should all you the same profile. bigquery project "foo" should only be referenced if is in my profiles.yml file

Version:
v0.1.0

Additional context
Add any other context about the problem here.

@daniel-bartley daniel-bartley added the bug Something isn't working label Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant