Add brand new Dagster project example to dbt quickstart #21816

maximearmstrong · 2024-05-13T16:59:46Z

Summary & Motivation

This PR updates the quickstart example for dbt + Dagster - it adds the steps to create a brand new Dagster project using the scaffold to wrap a dbt project.

This work is the continuation of #21240

How I Tested These Changes

BK
+
local

make mdx-full-format
make next-watch-build

maximearmstrong · 2024-05-13T17:00:01Z

Add brand new Dagster project example to dbt quickstart #21816 👈
Add quickstart example for dbt & Dagster #21240 : 2 other dependent PRs (#21568 , #21817 )
Add dbt + Dagster+ guide #21229 : 1 other dependent PR (#21950 )
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @maximearmstrong and the rest of your teammates on Graphite

github-actions · 2024-05-13T17:04:00Z

Deploy preview for dagster-docs ready!

Preview available at https://dagster-docs-i86wexok8-elementl.vercel.app
https://maxime-brand-new-project-in-dbt-quickstart.dagster.dagster-docs.io

Direct link to changed pages:

https://dagster-docs-i86wexok8-elementl.vercel.app
https://maxime-brand-new-project-in-dbt-quickstart.dagster.dagster-docs.io/integrations/dbt/quickstart

rexledesma

We have the opportunity to simplify the dagster-dbt project scaffold CLI by making --project-name an optional parameter. We should do this to simplify the instructions a bit.

rexledesma · 2024-05-13T19:17:45Z

docs/content/integrations/dbt/quickstart.mdx

+
+You can create a Dagster project that wraps your dbt project by using the `dagster-dbt` command line interface.
+
+To use the command, you'll need to provide 2 options - `--project-name`, the name of your Dagster project to be created, and `--dbt-project-dir`, the path to your dbt project. In our example, our Dagster project is named `my-dagster-project`and the relative path to our dbt project is `./my-dbt-project`, meaning that we are in the directory where `my-dbt-project` is located.


We can change this so that the user only has to provide one parameter --dbt-project-dir.

We can do this by reading the user's dbt_project.yml and using the current name as the project name. Maybe event append _dagster to that dbt project name,

Currently, if we don't provide the --project-name parameter, the user is prompted for the project name. Should we update this behaviour?

If so, could we do it in another PR? My goal here was to make the command as explicit as possible to make it easier for the user to understand.

Yeah we should do it in another PR.

rexledesma · 2024-05-13T19:19:26Z

docs/content/integrations/dbt/quickstart.mdx

+<TabItem name="Option 1: Create a brand new Dagster project">
+
+### Option 1: Create a brand new Dagster project


Just to remove a word

Suggested change

<TabItem name="Option 1: Create a brand new Dagster project">

### Option 1: Create a brand new Dagster project

<TabItem name="Option 1: Create a new Dagster project">

### Option 1: Create a new Dagster project

Done in 6e2bb9f

rexledesma · 2024-05-13T19:21:26Z

docs/content/integrations/dbt/quickstart.mdx

+dagster-dbt project scaffold --project-name my-dagster-project --dbt-project-dir ./my-dbt-project --use-experimental-dbt-project
+```
+
+This creates a directory called `my-dagster-project/` inside the current directory. The `my-dagster-project/` directory contains a set of files that define a Dagster project.


nit: let's be explicit about what's happening

Suggested change

This creates a directory called `my-dagster-project/` inside the current directory. The `my-dagster-project/` directory contains a set of files that define a Dagster project.

This command creates a new directory called `my-dagster-project/` inside the current directory. The new `my-dagster-project/` directory will contain a set of files that define a Dagster project to load the dbt project provided in `./my-dbt-project`.

Done in 6e2bb9f

rexledesma · 2024-05-13T19:25:31Z

docs/content/integrations/dbt/quickstart.mdx

@@ -41,6 +41,28 @@ The `dagster-dbt` library installs both `dbt-core` and `dagster` as dependencies

 ---

+## Wrap your dbt project with Dagster


I know we use the "wrap" lingo here, but maybe we should consolidate on "load"

Suggested change

## Wrap your dbt project with Dagster

## Load your dbt project in Dagster

Done in 6e2bb9f

rexledesma

One comment, but can land after that!

rexledesma · 2024-05-14T16:10:03Z

docs/content/integrations/dbt/quickstart.mdx

+
+You can create a Dagster project that wraps your dbt project by using the `dagster-dbt` command line interface.
+
+To use the command, you'll need to provide 2 options - `--project-name`, the name of your Dagster project to be created, and `--dbt-project-dir`, the path to your dbt project. In our example, our Dagster project is named `my_dagster_project`and the relative path to our dbt project is `./my_dbt_project`, meaning that we are in the directory where `my_dbt_project` is located.


Just to explicitly call out the two options. Also nit: should spell out numbers.

Suggested change

To use the command, you'll need to provide 2 options - `--project-name`, the name of your Dagster project to be created, and `--dbt-project-dir`, the path to your dbt project. In our example, our Dagster project is named `my_dagster_project`and the relative path to our dbt project is `./my_dbt_project`, meaning that we are in the directory where `my_dbt_project` is located.

To use the command, you'll need to provide two options:

- `--project-name`, the name of your Dagster project to be created, and

- `--dbt-project-dir`, the path to your dbt project.

In our example, our Dagster project is named `my_dagster_project`and the relative path to our dbt project is `./my_dbt_project`, meaning that we are in the directory where `my_dbt_project` is located.

## Summary & Motivation This PR updates the quickstart example for dbt + Dagster - it adds the steps to create a brand new Dagster project using the scaffold to wrap a dbt project. This work is the continuation of #21240 ## How I Tested These Changes BK + local ``` make mdx-full-format make next-watch-build ```

## Summary & Motivation This PR updates the quickstart example for dbt + Dagster - it adds the steps to update an existing Dagster project to wrap a dbt project. This work is the continuation of #21240 and #21816 ## How I Tested These Changes BK + local ``` make mdx-full-format make next-watch-build ```

## Summary & Motivation This PR updates the quickstart example for dbt + Dagster - it adds the steps to create a brand new Dagster project using the scaffold to wrap a dbt project. This work is the continuation of #21240 ## How I Tested These Changes BK + local ``` make mdx-full-format make next-watch-build ```

This was referenced May 13, 2024

Add quickstart example for dbt & Dagster #21240

Merged

Add dbt + Dagster+ guide #21229

Draft

maximearmstrong mentioned this pull request May 13, 2024

Update dbt docs with DbtProject #21568

Draft

maximearmstrong force-pushed the maxime/ds-169/add-full-copy-pasteable-example-to-dbt-integrate-page branch from 54ed678 to 88b03c6 Compare May 13, 2024 18:43

maximearmstrong force-pushed the maxime/brand-new-project-in-dbt-quickstart branch from 095a31f to 5ff29d3 Compare May 13, 2024 18:43

maximearmstrong mentioned this pull request May 13, 2024

Add existing Dagster project example to dbt quickstart #21817

Merged

maximearmstrong self-assigned this May 13, 2024

maximearmstrong marked this pull request as ready for review May 13, 2024 18:49

maximearmstrong requested a review from tacastillo May 13, 2024 18:49

graphite-app bot added the area: docs Related to documentation in general label May 13, 2024

graphite-app bot requested a review from erinkcochran87 May 13, 2024 18:49

maximearmstrong requested review from rexledesma, erinkcochran87 and yuhan and removed request for erinkcochran87 May 13, 2024 18:49

maximearmstrong force-pushed the maxime/brand-new-project-in-dbt-quickstart branch from 5ff29d3 to 0ddd004 Compare May 13, 2024 19:07

rexledesma requested changes May 13, 2024

View reviewed changes

maximearmstrong force-pushed the maxime/ds-169/add-full-copy-pasteable-example-to-dbt-integrate-page branch from c8ecf81 to f8f6482 Compare May 14, 2024 15:43

maximearmstrong force-pushed the maxime/brand-new-project-in-dbt-quickstart branch from 0ddd004 to 6e2bb9f Compare May 14, 2024 15:43

maximearmstrong requested a review from rexledesma May 14, 2024 15:44

maximearmstrong force-pushed the maxime/brand-new-project-in-dbt-quickstart branch from 6e2bb9f to b75e81f Compare May 14, 2024 16:07

rexledesma approved these changes May 14, 2024

View reviewed changes

maximearmstrong added 4 commits May 14, 2024 12:45

Add brand new Dagster project steps

9f0c57a

Update quickstart post review

6b31264

Use underscore in dir names

968006a

Update quickstart post-review

ca9f008

maximearmstrong force-pushed the maxime/ds-169/add-full-copy-pasteable-example-to-dbt-integrate-page branch from b6b450b to 170b531 Compare May 14, 2024 16:45

maximearmstrong force-pushed the maxime/brand-new-project-in-dbt-quickstart branch from b75e81f to ca9f008 Compare May 14, 2024 16:45

maximearmstrong merged commit 927bbc5 into maxime/ds-169/add-full-copy-pasteable-example-to-dbt-integrate-page May 14, 2024
1 of 3 checks passed

maximearmstrong deleted the maxime/brand-new-project-in-dbt-quickstart branch May 14, 2024 17:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add brand new Dagster project example to dbt quickstart #21816

Add brand new Dagster project example to dbt quickstart #21816

maximearmstrong commented May 13, 2024 •

edited

maximearmstrong commented May 13, 2024 •

edited

github-actions bot commented May 13, 2024

rexledesma left a comment

rexledesma May 13, 2024

maximearmstrong May 14, 2024

rexledesma May 14, 2024

rexledesma May 13, 2024

maximearmstrong May 14, 2024

rexledesma May 13, 2024

maximearmstrong May 14, 2024

rexledesma May 13, 2024

maximearmstrong May 14, 2024

rexledesma left a comment

rexledesma May 14, 2024


		You can create a Dagster project that wraps your dbt project by using the `dagster-dbt` command line interface.

		To use the command, you'll need to provide 2 options - `--project-name`, the name of your Dagster project to be created, and `--dbt-project-dir`, the path to your dbt project. In our example, our Dagster project is named `my-dagster-project`and the relative path to our dbt project is `./my-dbt-project`, meaning that we are in the directory where `my-dbt-project` is located.

		<TabItem name="Option 1: Create a brand new Dagster project">

		### Option 1: Create a brand new Dagster project

	This creates a directory called `my-dagster-project/` inside the current directory. The `my-dagster-project/` directory contains a set of files that define a Dagster project.
	This command creates a new directory called `my-dagster-project/` inside the current directory. The new `my-dagster-project/` directory will contain a set of files that define a Dagster project to load the dbt project provided in `./my-dbt-project`.

		@@ -41,6 +41,28 @@ The `dagster-dbt` library installs both `dbt-core` and `dagster` as dependencies

		---

		## Wrap your dbt project with Dagster

	## Wrap your dbt project with Dagster
	## Load your dbt project in Dagster

Add brand new Dagster project example to dbt quickstart #21816

Add brand new Dagster project example to dbt quickstart #21816

Conversation

maximearmstrong commented May 13, 2024 • edited

Summary & Motivation

How I Tested These Changes

maximearmstrong commented May 13, 2024 • edited

github-actions bot commented May 13, 2024

rexledesma left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rexledesma left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maximearmstrong commented May 13, 2024 •

edited

maximearmstrong commented May 13, 2024 •

edited