Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check-model-has-description raises IsADirectoryError intermittently #221

Open
shippy opened this issue May 6, 2024 · 0 comments
Open

check-model-has-description raises IsADirectoryError intermittently #221

shippy opened this issue May 6, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@shippy
Copy link

shippy commented May 6, 2024

Describe the bug
When running the pre-commit hook check-model-has-description with non-standard directory and target locations (greybox_conversion/transform and greybox_conversion/.meltano) errors out with a IsADirectoryError because it somehow picks up the target location for a source of YAML files, and then it's surprised when the target directory YAML file actually is a directory that contains the SQL for the tests defined in the project directory YAML file.

To Reproduce
Steps to reproduce the behavior:

  1. Set up a dbt project in meltano_pipeline/transform
  2. Set up a dbt target directory in meltano_pipeline/.meltano via DBT_TARGET_PATH
  3. Set up pre-commit hook with the following definition:
    - id: check-model-has-properties-file
      files: ^meltano_pipeline/transform/models/
      args: [ --manifest, meltano_pipeline/.meltano/manifest.json ]
  4. Set up a models/base/model.sql with models/base/model.yml file that contains at least one model test
  5. Stage and run pre-commit

Expected behavior
This hook should only check against YAML files in the project directory.

Version:
v2.0.1

Additional context
Here's what the traceback looks like - most other hooks don't have this problem:

Check the model has properties file......................................Passed
Check the model has description..........................................Failed
- hook id: check-model-has-description
- exit code: 1

Traceback (most recent call last):
  File "/Users/simon/.cache/pre-commit/repojqwpib0c/py_env-python3.11/bin/check-model-has-description", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/simon/.cache/pre-commit/repojqwpib0c/py_env-python3.11/lib/python3.11/site-packages/dbt_checkpoint/check_model_has_description.py", line 69, in main
    hook_properties = has_description(
                      ^^^^^^^^^^^^^^^^
  File "/Users/simon/.cache/pre-commit/repojqwpib0c/py_env-python3.11/lib/python3.11/site-packages/dbt_checkpoint/check_model_has_description.py", line 42, in has_description
    in_schemas = {
                 ^
  File "/Users/simon/.cache/pre-commit/repojqwpib0c/py_env-python3.11/lib/python3.11/site-packages/dbt_checkpoint/check_model_has_description.py", line 42, in <setcomp>
    in_schemas = {
                 ^
  File "/Users/simon/.cache/pre-commit/repojqwpib0c/py_env-python3.11/lib/python3.11/site-packages/dbt_checkpoint/utils.py", line 307, in get_model_schemas
    with open(yml_file, "r") as file:
         ^^^^^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: 'greybox_conversion/.meltano/run/greybox_transformation/models/achievements/achievements.yml'

Check the model columns have description.............(no files to check)Skipped
Check the model has all columns in properties file...(no files to check)Skipped
Check that model has tests...............................................Passed

I tested whether including an exclude: ^meltano_pipeline/.meltano/.* would do anything, since my reading of the related hook suggested it might, but either I'm using an incorrect format or it doesn't work.

@shippy shippy added the bug Something isn't working label May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant