diff --git a/docs/pages/repo/docs/core-concepts/monorepos/_meta.json b/docs/pages/repo/docs/core-concepts/monorepos/_meta.json index d66bf5ddef45e..6ce429d0dbdc6 100644 --- a/docs/pages/repo/docs/core-concepts/monorepos/_meta.json +++ b/docs/pages/repo/docs/core-concepts/monorepos/_meta.json @@ -1,5 +1,6 @@ { "running-tasks": "Running Tasks", + "task-dependencies": "Task Dependencies", "filtering": "Filtering Workspaces", "skipping-tasks": "Skipping Tasks in CI" } diff --git a/docs/pages/repo/docs/core-concepts/monorepos/running-tasks.mdx b/docs/pages/repo/docs/core-concepts/monorepos/running-tasks.mdx index 96d638a139838..9639e6ff75988 100644 --- a/docs/pages/repo/docs/core-concepts/monorepos/running-tasks.mdx +++ b/docs/pages/repo/docs/core-concepts/monorepos/running-tasks.mdx @@ -1,6 +1,6 @@ --- title: Running Tasks -description: Turborepo helps you specify task dependencies declaratively. +description: Turborepo can run all your tasks. --- import Callout from "../../../../../components/Callout"; @@ -17,7 +17,7 @@ Here, both `apps/web` and `apps/doc` use code from `packages/shared`. In fact, w ## Most tools don't optimize for speed -Let's imagine we want to run all our tasks across all our workspaces. In a tool like `yarn`, you might run a script like this: +Let's imagine we want to run all our tasks across all our workspaces. With a tool like `yarn`, you might run a script like this: ``` yarn workspaces run lint @@ -44,9 +44,9 @@ First, we declare our tasks inside `turbo.json`: "$schema": "https://turbo.build/schema.json", "pipeline": { "build": { - // ^build means build must be run in dependencies + "outputs": [".next/**", "!.next/cache/**", ".svelte-kit/**"], + // ^build means `build` must be run in dependencies // before it can be run in this workspace - "outputs": [".next/**", "!.next/cache/**",".svelte-kit/**"], "dependsOn": ["^build"] }, "test": {}, @@ -74,7 +74,8 @@ The `build` task inside `shared` completes first, then `web` and `docs` build af ## Defining a pipeline -The `pipeline` configuration declares which tasks depend on each other in your monorepo. Here's a kitchen sink example: +The `pipeline` configuration declares which tasks depend on each other in your +monorepo. Here's a kitchen sink example: ```jsonc filename="turbo.json" { @@ -109,114 +110,9 @@ The `pipeline` configuration declares which tasks depend on each other in your m } ``` -Let's walk through some common patterns you'll want to get to know before diving in to `turbo.json`. +You can read more about how to setup your tasks in the next section on [Task Dependencies](/repo/docs/core-concepts/monorepos/task-dependencies). -### Dependencies between tasks - -#### In the same workspace - -There might be tasks which need to run _before_ other tasks. For instance, `build` might need to be run before `deploy`. - -If both tasks are in the same workspace, you can specify the relationship like this: - -```jsonc filename="turbo.json" -{ - "$schema": "https://turbo.build/schema.json", - "pipeline": { - "build": { - "dependsOn": ["^build"], - "outputs": [".next/**", "!.next/cache/**", ".svelte-kit/**"] - }, - "deploy": { - // A workspace's `deploy` task depends on the `build`, - // task of the same workspace being completed. - "dependsOn": ["build"] - } - } -} -``` - -This means that whenever `turbo run deploy` is run, `build` will also be run inside the same workspace. - -#### In a different workspace - -A common pattern in monorepos is to declare that a workspace's `build` task should only run once the `build` tasks of all _the workspaces it depends on_ are complete. - -The `^` symbol explicitly declares that the task has a dependency on a task in a workspace it depends on. - -```jsonc filename="turbo.json" -{ - "$schema": "https://turbo.build/schema.json", - "pipeline": { - "build": { - // "A workspace's `build` command depends on its dependencies' - // and devDependencies' `build` commands being completed first" - "dependsOn": ["^build"], - "outputs": [".next/**", "!.next/cache/**", ".svelte-kit/**"] - } - } -} -``` - -#### No dependencies - -An empty dependency list (`dependsOn` is either undefined or `[]`) means that nothing needs to run before this task! After all, it has NO dependencies. - -```jsonc filename="turbo.json" -{ - "$schema": "https://turbo.build/schema.json", - "pipeline": { - // A workspace's `lint` command has no dependencies and can be run - // whenever. - "lint": {} - } -} -``` - -#### Specific workspace-tasks - -Sometimes, you may want to create a workspace-task dependency on another workspace-task. This can be especially helpful for repos migrating from `lerna` or `rush`, where tasks are run in separate phases by default. Sometimes these configurations make assumptions that cannot be expressed in a simple `pipeline` configuration, as seen above. Or you may just want to express sequences of tasks between applications or microservices when using `turbo` in CI/CD. - -For these cases, you can express these relationships in your `pipeline` configuration using the `#` syntax. -The example below describes the `deploy` script of a `frontend` application that depends on the `deploy` and `health-check` scripts of `backend`, as well as the `test` script of a `ui` workspace: - -```jsonc filename="turbo.json" -{ - "$schema": "https://turbo.build/schema.json", - "pipeline": { - // Standard configuration - "build": { - "dependsOn": ["^build"], - "outputs": [".next/**", "!.next/cache/**", ".svelte-kit/**"] - }, - "test": { - "dependsOn": ["^build"] - }, - "deploy": { - "dependsOn": ["test", "build"] - }, - - // Explicit workspace-task to workspace-task dependency - "frontend#deploy": { - "dependsOn": ["ui#test", "backend#deploy", "backend#health-check"] - } - } -} -``` - -This explicit configuration for `frontend#deploy` may seem to conflict with the `test` and `deploy` task configurations, but it does not. Since `test` and `deploy` do not have dependencies on other workspaces (e.g. `^`), they can execute any time after their workspace's `build` and `test` scripts have finished. - - - Notes: - -1. Although this `#` syntax is a useful escape hatch, we generally recommend using it for deployment orchestration tasks such as health checks, rather than build-time dependencies, so that Turborepo can optimize these tasks more efficiently -1. Package-tasks do not inherit cache configuration. You must redeclare - [`outputs`](/repo/docs/reference/configuration#outputs) at the moment. -1. `` must match the `name` key in the workspace's `package.json` or the task will be ignored. - - - -### Running tasks from the root +## Running tasks from the root `turbo` can run tasks that exist in the `package.json` file at the root of the monorepo. These must be explicitly added to the pipeline configuration using the key syntax `"//#"`. This is @@ -263,48 +159,6 @@ be explicitly opted into via including `//#` in the pipeline configuration some best-effort checking to produce an error in the recursion situations, but it is up to you to only opt in those tasks which don't themselves trigger a `turbo` run that would recurse. -### Dependencies outside of a task - -When your task has topological dependencies that are outside of that given task, you'll still want to enjoy the parallelism of Turborepo and ensure that your caching behavior is correct according to your code changes. - -To demonstrate how to do this, let's say you have a set of workspaces to do a little bit of math: `add`, `subtract`, and `multiply`. `subtract` is implemented by calling `add` with a negative number and your `multiply` works by calling `add` in a loop. So, `add` is a dependency of both `subtract` and `multiply`. - -You've written tests in all three of these workspaces and it's time to run them. There are two requirements here: - -1. All tests run in parallel to keep things speedy -2. A change in a dependency should result in a cache miss - -To accomplish this, we can set up a pipeline like so: - -```jsonc filename="turbo.json" -{ - "$schema": "https://turbo.build/schema.json", - "pipeline": { - "topo": { - "dependsOn": ["^topo"] - }, - "test": { - "dependsOn": ["^topo"] - } - } - } -``` - -![](/images/docs/task-graph-with-placeholder-task.png) - -In this pipeline, we create an intermediary placeholder `topo` task. Since we don't have a `topo` command in our workspaces, the pipeline will go straight to running `test` scripts in parallel, meeting our first requirement. The second requirement will also be taken care of, falling back on Turborepo's default behavior of creating hashes for a workspace task and it's dependencies as a tree. - - - ### Incremental Adoption After you've declared a task in `turbo.json`, it's up to you to implement it in @@ -312,14 +166,14 @@ your `package.json` manifests. You can add scripts all at once, or one workspace at at a time. Turborepo will gracefully skip workspaces that don't include the task in their respective package.json manifest. -For example, if your repository has three workspaces: +For example, if your repository has the three workspaces (similar to the ones mentioned above on this page): ```bash apps/ web/package.json docs/package.json packages/ - ui/package.json + shared/package.json turbo.json package.json ``` @@ -327,7 +181,7 @@ package.json where `turbo.json` declares a `build` task, but only two `package.json`'s implement that `build` task: - + ```jsonc filename="turbo.json" { @@ -360,9 +214,9 @@ task: ⚠️ Note the missing `build` script! - ```jsonc filename="packages/ui/package.json" + ```jsonc filename="packages/shared/package.json" { - "name": "ui", + "name": "shared", "scripts": {} } ``` @@ -374,4 +228,15 @@ turbo run build ``` A turbo build will only execute the `build` script for the `web` and `docs` workspaces. The -`ui` package will still be in the task graph, but will gracefully be skipped. +`shared` package will still be part of the task graph, but will gracefully be skipped. + + diff --git a/docs/pages/repo/docs/core-concepts/monorepos/task-dependencies.mdx b/docs/pages/repo/docs/core-concepts/monorepos/task-dependencies.mdx new file mode 100644 index 0000000000000..c67ae5e92dfbd --- /dev/null +++ b/docs/pages/repo/docs/core-concepts/monorepos/task-dependencies.mdx @@ -0,0 +1,189 @@ +--- +title: Task Dependencies +description: Turborepo helps you specify task dependencies declaratively. +--- + +import Callout from "../../../../../components/Callout"; +import HeartIcon from "@heroicons/react/solid/HeartIcon"; +import { Tabs, Tab } from '../../../../../components/Tabs' + +# Task Dependencies + +Turborepo is most powerful when you express how your tasks relate to each other. +We refer to these relationships as "dependencies", but they are not the same as +package dependencies that you install from your `package.json` files. While Turborepo +does understand your workspaces, it does not automatically draw any +relationships between their tasks, unless you express them in `turbo.json` via +the `dependsOn` configuration. + +Let's walk through some common patterns on how to make a task depend on other tasks. + +## From the same workspace + +There might be tasks that need to run _before_ other tasks. For instance, +`build` might need to be run before `deploy`. + +If both tasks are in the same workspace, you can specify the relationship like +this: + +```jsonc filename="turbo.json" +{ + "$schema": "https://turbo.build/schema.json", + "pipeline": { + "build": {}, + "deploy": { + // A workspace's `deploy` task depends on the `build` task of the same workspace. + "dependsOn": ["build"] + } + } +} +``` + +This means that whenever `turbo deploy` is run, `build` will also be run +inside the same workspace. + +## From dependent workspaces + +A common pattern in monorepos is to declare that a workspace's `build` task +should only run once the `build` tasks of all _the workspaces it depends on_ are +complete. + + +This can be confusing as it refers to both _workspace_ dependencies +and _task_ dependencies, which are different concepts. Workspace dependencies are `dependencies` +and `devDependencies` in `package.json`, whereas, task dependencies are `dependsOn` key in `turbo.json`. + + +The `^` symbol (called a "caret") explicitly declares that the task depends on the task in a +workspace it depends on. + +```jsonc filename="turbo.json" +{ + "$schema": "https://turbo.build/schema.json", + "pipeline": { + "build": { + // A workspace's `build` command depends on its dependencies' + // and devDependencies' `build` commands being completed first + "dependsOn": ["^build"], + } + } +} +``` + +With the configuration above, if an app installs a package from another workspace, the package's +`build` script will always run before the app's `build` script. + +## From arbitrary workspaces + +Sometimes, you may want a workspace-task to depend on another +workspace-task. This can be especially helpful for repos migrating from `lerna` +or `rush`, where tasks are run in separate phases by default. Sometimes these +configurations make assumptions that cannot be expressed in a simple `pipeline` +configuration, as seen above. Or you may just want to express sequences of tasks +between applications or microservices when using `turbo` in CI/CD. + +For these cases, you can express these relationships in your `pipeline` +configuration using the `#` syntax. The example below describes +the `deploy` script of a `frontend` application that depends on the `deploy` and +`health-check` scripts of `backend`, as well as the `test` script of a `ui` +workspace: + +```jsonc filename="turbo.json" +{ + "$schema": "https://turbo.build/schema.json", + "pipeline": { + // Explicit workspace-task to workspace-task dependency + "frontend#deploy": { + "dependsOn": ["ui#test", "backend#deploy", "backend#health-check"] + } + } +} +``` + +This explicit configuration for `frontend#deploy` may seem to conflict with the +`test` and `deploy` task configurations, but it does not. Since `test` and +`deploy` do not have dependencies on other workspaces (e.g. `^`), they can +execute any time after their workspace's `build` and `test` scripts have +finished. + + + Notes: + +1. Although this `#` syntax is a useful escape hatch, we + generally recommend using it for deployment orchestration tasks such as + health checks, rather than build-time dependencies, so that Turborepo can + optimize these tasks more efficiently +1. Workspace-tasks do not inherit cache configuration. You must redeclare + [`outputs`](/repo/docs/reference/configuration#outputs) at the moment. +1. `` must match the `name` key in the workspace's `package.json` or + the task will be ignored. + + + +## Dependencies outside of a task + +When your task has topological dependencies that are outside of that given task, +you'll still want to enjoy the parallelism of Turborepo and ensure that your +caching behavior is reflects code changes. + +To demonstrate how to do this, let's say you have a set of packages (modeled as +workspaces in your codebase) to define some math functions: `add`, `subtract`, and +`multiply`. `subtract` is implemented by calling `add` with a negative number +and your `multiply` works by calling `add` in a loop. So, `add` is a dependency +of both `subtract` and `multiply`. + +```bash +packages/ + add/package.json # does not depend on anything + multiply/package.json # depends on add + subtract/package.json # depends on add +turbo.json +package.json +``` + +You've written tests in all three of these workspaces and it's time to run them. +There are two requirements here: + +1. All tests run in parallel to keep things speedy +2. A change in a dependency should result in a cache miss + +To accomplish this, we can set up a pipeline like this: + +```jsonc filename="turbo.json" +{ + "$schema": "https://turbo.build/schema.json", + "pipeline": { + "topo": { + "dependsOn": ["^topo"] + }, + "test": { + "dependsOn": ["^topo"] + } + } + } +``` + +![](/images/docs/task-graph-with-placeholder-task.png) + +In this pipeline, we declare a "synthetic" task called `topo`. Since we don't +have a `topo` script in any of our `package.json` files, on `turbo test` the +pipeline will go straight to running `test` scripts in parallel, meeting our first requirement. + +But this `topo` task also creates a "synthetic" workspace-task dependency from `multiple` -> `add`, +and from `subtract` -> `add`. This means that when you change code in `add`, you will also get +a cache miss for the workspaces in `multiply` and `subtract`, meeting the second requirement. + +## No dependencies + +An empty dependency list (`dependsOn` is either undefined or `[]`) means that +nothing needs to run before this task! After all, it has no dependencies. + +```jsonc filename="turbo.json" +{ + "$schema": "https://turbo.build/schema.json", + "pipeline": { + // A workspace's `lint` command has no dependencies and can be run any time. + "lint": {} + } +} +```