feature: Add the ability to run dbt-docs-generate on only changed files #210
Conversation
From docs_generate_cmd to prepare_cmd
From docs_generate_cmd to prepare_cmd
| @@ -236,7 +236,8 @@ | |||
| description: The command is responsible for generating your project's documentation website. | |||
| entry: dbt-docs-generate | |||
| language: python | |||
| pass_filenames: false | |||
There was a problem hiding this comment.
I have turn this flags OFF to be able to get back filenames in the args
| run_dbt_cmd, | ||
| ) | ||
|
|
||
|
|
||
| def docs_generate_cmd( | ||
| def prepare_cmd( |
There was a problem hiding this comment.
I have changed docs_generate_cmdto prepare_cmd to be aligned with the rest of the codebase
| config: Dict[str, Any] = {} | ||
| prefix: str = "", | ||
| postfix: str = "", | ||
| models: Optional[Sequence[str]] = None, | ||
| config: Dict[str, Any] = {}, | ||
| ) -> List[str]: | ||
| global_flags = get_flags(global_flags) | ||
| cmd_flags = get_flags(cmd_flags) | ||
| if models: | ||
| dbt_models = models | ||
| else: | ||
| dbt_models = paths_to_dbt_models(paths, prefix, postfix) |
There was a problem hiding this comment.
I tried to not reinvent the wheel here, this is heavily inspired by dbt_runcurrent hook.
| return extend_dbt_project_dir_flag(cmd, cmd_flags, dbt_project_dir) | ||
|
|
||
|
|
||
| def main(argv: Optional[Sequence[str]] = None) -> int: | ||
| parser = argparse.ArgumentParser() | ||
| add_filenames_args(parser) |
There was a problem hiding this comment.
Get the filenames to be available
| add_dbt_cmd_args(parser) | ||
| add_dbt_cmd_model_args(parser) |
There was a problem hiding this comment.
If models are passed, let them be go through to args
| require_serial: true | ||
| types_or: [sql] |
There was a problem hiding this comment.
I want only .sql files to be considered. We are missing dbt models .py here
|
Any news on this one ? This is a feature we'd also like to have for our workflows |
|
So with this MR, we need another restriction of dbt-core>=1.7? |
This is addressing #196
dbt docs generateis very slow and is a bottleneck in development. The idea is to mimic the behavior of the other cli hooks such asdbt runto pick only the changed models to be documented in order to pursue check on column documentations, tests, etc.