Skip to content

Latest commit

 

History

History
297 lines (252 loc) · 8.5 KB

running-tasks.mdx

File metadata and controls

297 lines (252 loc) · 8.5 KB
title description
Running Tasks
Turborepo can run all your tasks.

import Callout from "../../../../../components/Callout"; import HeartIcon from "@heroicons/react/solid/HeartIcon"; import { Tabs, Tab } from '../../../../../components/Tabs'

Running Tasks in a Monorepo

Every monorepo has two main building blocks: workspaces and tasks. Let's imagine you have a monorepo containing three workspaces, each with three tasks:

Here, both apps/web and apps/doc use code from packages/shared. In fact, when they're built (via build) they need packages/shared to be built first.

Most tools don't optimize for speed

Let's imagine we want to run all our tasks across all our workspaces. In a tool like yarn, you might run a script like this:

yarn workspaces run lint
yarn workspaces run test
yarn workspaces run build

This would mean the tasks run like this:

As you can see, lint gets run in all the workspaces. Then, build gets run - with shared going first. Finally, test gets run.

This is the slowest possible way to run these tasks. Each task needs to wait for the previous one to finish before it can start. To improve on this, we'll need a tool that can multitask.

Turborepo can multitask

Turborepo can schedule our tasks for maximum speed by understanding the dependencies between our tasks.

First, we declare our tasks inside turbo.json:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      // ^build means build must be run in dependencies
      // before it can be run in this workspace
      "outputs": [".next/**", "!.next/cache/**",".svelte-kit/**"],
      "dependsOn": ["^build"]
    },
    "test": {},
    "lint": {}
  }
}

Next, we can replace our yarn workspaces script with this:

- yarn workspaces run lint
- yarn workspaces run test
- yarn workspaces run build
+ turbo run lint test build

When we run it, Turborepo will multitask as many tasks as possible over all available CPU's, meaning our tasks run like this:

Both lint and test run immediately, because they have no dependsOn specified in turbo.json.

The build task inside shared completes first, then web and docs build afterwards.

Defining a pipeline

The pipeline configuration declares which tasks depend on each other in your monorepo. Here's a kitchen sink example:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      // A workspace's `build` task depends on that workspace's
      // topological dependencies' and devDependencies'
      // `build` tasks  being completed first. The `^` symbol
      // indicates an upstream dependency.
      "dependsOn": ["^build"],
      "outputs": [".next/**", "!.next/cache/**", ".svelte-kit/**"]
    },
    "test": {
      // A workspace's `test` task depends on that workspace's
      // own `build` task being completed first.
      "dependsOn": ["build"],
      // A workspace's `test` task should only be rerun when
      // either a `.tsx` or `.ts` file has changed.
      "inputs": ["src/**/*.tsx", "src/**/*.ts", "test/**/*.ts", "test/**/*.tsx"]
    },
    // A workspace's `lint` task has no dependencies and
    // can be run whenever.
    "lint": {},
    "deploy": {
      // A workspace's `deploy` task depends on the `build`,
      // `test`, and `lint` tasks of the same workspace
      // being completed.
      "dependsOn": ["build", "test", "lint"]
    }
  }
}

Running tasks from the root

turbo can run tasks that exist in the package.json file at the root of the monorepo. These must be explicitly added to the pipeline configuration using the key syntax "//#<task>". This is true even for tasks that already have their own entry. For example, if your pipeline declares a "build" task, and you want to include the build script defined in the monorepo's root package.json file with turbo run build, you must opt the root into it by declaring "//#build": {...} in your configuration. Conversely, you do not need to define a generic "my-task": {...} entry if all you need is "//#my-task": {...}.

A sample pipeline that defines the root task format and opts the root into test might look like:

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": [".next/**", "!.next/cache/**", ".svelte-kit/**"]
    },
    "test": {
      "dependsOn": ["^build"],
    },
    // This will cause the "test" script to be included when
    // "turbo run test" is run
    "//#test": {
      "dependsOn": [],
    },
    // This will cause the "format" script in the root package.json
    // to be run when "turbo run format" is run. Since the general
    // "format" task is not defined, only the root's "format" script
    // will be run.
    "//#format": {
      "dependsOn": [],
      "outputs": ["dist/**/*"],
      "inputs": ["version.txt"]
    }
  }
}

A note on recursion: Scripts defined in the monorepo's root package.json often call turbo themselves. For example, the build script might be turbo run build. In this situation, including //#build in turbo run build will cause infinite recursion. It is for this reason that tasks run from the monorepo's root must be explicitly opted into via including //#<task> in the pipeline configuration. turbo includes some best-effort checking to produce an error in the recursion situations, but it is up to you to only opt in those tasks which don't themselves trigger a turbo run that would recurse.

Incremental Adoption

After you've declared a task in turbo.json, it's up to you to implement it in your package.json manifests. You can add scripts all at once, or one workspace at at a time. Turborepo will gracefully skip workspaces that don't include the task in their respective package.json manifest. We think this is a great way to try out Turborepo without making large changes to your codebase.

For example, if your repository has three workspaces:

apps/
  web/package.json
  docs/package.json
packages/
  ui/package.json
turbo.json
package.json

where turbo.json declares a build task, but only two package.json's implement that build task:

<Tabs items={['turbo.json', 'web', 'docs', 'ui']} storageKey="skipped-tasks-example">

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {}
  }
}
```jsonc filename="apps/web/package.json" { "name": "web", "scripts": { "build": "next build" } } ``` ```jsonc filename="apps/docs/package.json" { "name": "docs", "scripts": { "build": "svelte build" } } ``` ⚠️ Note the missing `build` script! ```jsonc filename="packages/ui/package.json" { "name": "ui", "scripts": {} } ```
turbo run build

Incremental Adoption

After you've declared a task in turbo.json, it's up to you to implement it in your package.json manifests. You can add scripts all at once, or one workspace at at a time. Turborepo will gracefully skip workspaces that don't include the task in their respective package.json manifest.

For example, if your repository has three workspaces:

apps/
  web/package.json
  docs/package.json
packages/
  ui/package.json
turbo.json
package.json

where turbo.json declares a build task, but only two package.json's implement that build task:

<Tabs items={['turbo.json', 'web', 'docs', 'ui']} storageKey="skipped-tasks-example">

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "build": {}
  }
}
```jsonc filename="apps/web/package.json" { "name": "web", "scripts": { "build": "next build" } } ``` ```jsonc filename="apps/docs/package.json" { "name": "docs", "scripts": { "build": "vite build" } } ``` ⚠️ Note the missing `build` script! ```jsonc filename="packages/ui/package.json" { "name": "ui", "scripts": {} } ```
turbo run build

A turbo build will only execute the build script for the web and docs workspaces. The ui package will still be in the task graph, but will gracefully be skipped.