Skip to content

Commit

Permalink
fix(run): update docs for v6 (#3366)
Browse files Browse the repository at this point in the history
  • Loading branch information
vsavkin committed Oct 12, 2022
1 parent 8a1660e commit 62b1fc8
Show file tree
Hide file tree
Showing 13 changed files with 120 additions and 124 deletions.
6 changes: 3 additions & 3 deletions commands/run/index.js
Expand Up @@ -285,21 +285,21 @@ class RunCommand extends Command {
if (this.options.parallel || this.options.sort !== undefined) {
this.logger.warn(
this.name,
`"parallel", "sort", and "no-sort" are ignored when nx.json has targetDefaults defined. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks for details.`
`"parallel", "sort", and "no-sort" are ignored when nx.json has targetDefaults defined. See https://lerna.js.org/docs/lerna6-obsolete-options for details.`
);
}

if (this.options.includeDependencies) {
this.logger.info(
this.name,
`Using the "include-dependencies" option when nx.json has targetDefaults defined will include both task dependencies detected by Nx and project dependencies detected by Lerna. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks#--include-dependencies for details.`
`Using the "include-dependencies" option when nx.json has targetDefaults defined will include both task dependencies detected by Nx and project dependencies detected by Lerna. See https://lerna.js.org/docs/lerna6-obsolete-options#--include-dependencies for details.`
);
}

if (this.options.ignore) {
this.logger.info(
this.name,
`Using the "ignore" option when nx.json has targetDefaults defined will exclude only tasks that are not determined to be required by Nx. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks#--ignore for details.`
`Using the "ignore" option when nx.json has targetDefaults defined will exclude only tasks that are not determined to be required by Nx. See https://lerna.js.org/docs/lerna6-obsolete-options#--ignore for details.`
);
}
} else {
Expand Down
2 changes: 2 additions & 0 deletions core/global-options/README.md
Expand Up @@ -31,6 +31,8 @@ Disable progress bars. This is always the case in a CI environment.

### `--no-sort`

Note: As of Lerna 6 this property is ignored when `nx.json` is present.

By default, all tasks execute on packages in topologically sorted order as to respect the dependency relationships of the packages in question. Cycles are broken on a best-effort basis in a way not guaranteed to be consistent across Lerna invocations.

Topological sorting can cause concurrency bottlenecks if there are a small number of packages with many dependents or if some packages take a disproportionately long time to execute. The `--no-sort` option disables sorting, instead executing tasks in an arbitrary order with maximum concurrency.
Expand Down
2 changes: 1 addition & 1 deletion core/lerna/commands/add-caching/index.js
Expand Up @@ -93,7 +93,7 @@ class AddCachingCommand extends Command {
);
this.logger.info(
"add-caching",
"Note that the legacy task runner options of --sort, --no-sort and --parallel no longer apply. Learn more here: https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks"
"Note that the legacy task runner options of --sort, --no-sort and --parallel no longer apply. Learn more here: https://lerna.js.org/docs/lerna6-obsolete-options"
);
}

Expand Down
4 changes: 2 additions & 2 deletions e2e/tests/lerna-run/lerna-run-nx-include-dependencies.spec.ts
Expand Up @@ -255,7 +255,7 @@ describe("lerna-run-nx-include-dependencies", () => {
lerna notice filter including dependencies
lerna info filter [ 'package-X' ]
lerna verb run nx.json with targetDefaults was found. Task dependencies will be automatically included.
lerna info run Using the "include-dependencies" option when nx.json has targetDefaults defined will include both task dependencies detected by Nx and project dependencies detected by Lerna. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks#--include-dependencies for details.
lerna info run Using the "include-dependencies" option when nx.json has targetDefaults defined will include both task dependencies detected by Nx and project dependencies detected by Lerna. See https://lerna.js.org/docs/lerna6-obsolete-options#--include-dependencies for details.
`);
});
Expand Down Expand Up @@ -312,7 +312,7 @@ describe("lerna-run-nx-include-dependencies", () => {
lerna notice filter excluding "package-X"
lerna info filter [ 'package-X', '!package-X' ]
lerna verb run nx.json with targetDefaults was found. Task dependencies will be automatically included.
lerna info run Using the "ignore" option when nx.json has targetDefaults defined will exclude only tasks that are not determined to be required by Nx. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks#--ignore for details.
lerna info run Using the "ignore" option when nx.json has targetDefaults defined will exclude only tasks that are not determined to be required by Nx. See https://lerna.js.org/docs/lerna6-obsolete-options#--ignore for details.
`);
});
Expand Down
8 changes: 4 additions & 4 deletions e2e/tests/lerna-run/lerna-run-nx-incompatible-options.spec.ts
Expand Up @@ -153,7 +153,7 @@ describe("lerna-run-nx-incompatible-options", () => {
lerna notice cli v999.9.9-e2e.0
lerna WARN run "parallel", "sort", and "no-sort" are ignored when nx.json has targetDefaults defined. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks for details.
lerna WARN run "parallel", "sort", and "no-sort" are ignored when nx.json has targetDefaults defined. See https://lerna.js.org/docs/lerna6-obsolete-options for details.
`);
});
Expand Down Expand Up @@ -201,7 +201,7 @@ describe("lerna-run-nx-incompatible-options", () => {
lerna notice cli v999.9.9-e2e.0
lerna WARN run "parallel", "sort", and "no-sort" are ignored when nx.json has targetDefaults defined. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks for details.
lerna WARN run "parallel", "sort", and "no-sort" are ignored when nx.json has targetDefaults defined. See https://lerna.js.org/docs/lerna6-obsolete-options for details.
`);
});
Expand Down Expand Up @@ -249,7 +249,7 @@ describe("lerna-run-nx-incompatible-options", () => {
lerna notice cli v999.9.9-e2e.0
lerna WARN run "parallel", "sort", and "no-sort" are ignored when nx.json has targetDefaults defined. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks for details.
lerna WARN run "parallel", "sort", and "no-sort" are ignored when nx.json has targetDefaults defined. See https://lerna.js.org/docs/lerna6-obsolete-options for details.
`);
});
Expand Down Expand Up @@ -298,7 +298,7 @@ describe("lerna-run-nx-incompatible-options", () => {
lerna notice cli v999.9.9-e2e.0
lerna notice filter including dependencies
lerna info run Using the "include-dependencies" option when nx.json has targetDefaults defined will include both task dependencies detected by Nx and project dependencies detected by Lerna. See https://lerna.js.org/docs/recipes/using-lerna-powered-by-nx-to-run-tasks#--include-dependencies for details.
lerna info run Using the "include-dependencies" option when nx.json has targetDefaults defined will include both task dependencies detected by Nx and project dependencies detected by Lerna. See https://lerna.js.org/docs/lerna6-obsolete-options#--include-dependencies for details.
`);
});
Expand Down
2 changes: 0 additions & 2 deletions website/docs/api-reference/configuration.md
Expand Up @@ -69,8 +69,6 @@ Find the available options in [the API docs](/docs/api-reference/commands).

# Nx.json

This configuration is only relevant if you have `useNx: true` in your `lerna.json`.

> NOTE: "{projectRoot}" and "{workspaceRoot}" are special syntax supported by the task-runner, which will be appropriately interpolated internally when the command runs. You should therefore not replace "{projectRoot}" or "{workspaceRoot}" with fixed paths as this makes your configuration less flexible.
```json title="nx.json"
Expand Down
32 changes: 2 additions & 30 deletions website/docs/concepts/task-pipeline-configuration.md
Expand Up @@ -11,7 +11,7 @@ configure how Nx does it.

:::tip

If you don't have `nx.json`, run `npx nx init`.
If you don't have `nx.json`, run `npx lerna add-caching`.

:::

Expand Down Expand Up @@ -40,37 +40,9 @@ Note, you can also change the default in `nx.json`, like this:
}
```


## Allow Tasks to Run in Any Order

To run the `test` script for each of the projects, run the following:

```bash
npx lerna run test --no-sort
```

You should see the following output:

```bash title="Terminal Output"
✔ footer:test (1s)
✔ header:test (1s)
✔ remixapp:test (236ms)

——————————————————————————————————————————————————————————————————————————————

> Lerna (powered by Nx) Successfully ran target test for 3 projects (1s)
```

Note that we are passing `--no-sort` to tell Lerna that tasks can run in any order.

## Define Task Dependencies (aka Task Pipelines)

Without our help Lerna cannot know what targets (scripts) require order and which don't. That's why you can
pass `--sort` and `--no-sort`, but this isn't the best way to go about it.

If builds have to run in the topological order, they **always** have to run in that order; otherwise things will be broken. On the other hand, if tests can run in any order, it never make sense to run them in topological order. That would only make them slower.

A better way to do it is to tell Lerna how targets relate. Add the following to `nx.json`:
Without our help Lerna cannot know what targets (scripts) have prerequisites and which ones don't. You can define task dependencies in the `nx.json` file:

```json title="nx.json"
{
Expand Down
8 changes: 1 addition & 7 deletions website/docs/features/cache-tasks.md
Expand Up @@ -12,20 +12,14 @@ type: recipe
It's costly to rebuild and retest the same code over and over again. Lerna uses a computation cache to never rebuild the
same code twice.

:::info

To use task result caching, you must first enable nx by setting `"useNx": true` in `lerna.json`.

:::

## Setup

Lerna via Nx has the most sophisticated and battle-tested computation caching system. It knows when the task you are
about to run has been executed before, so it can use the cache to restore the results of running that task.

:::tip

If you don't have `nx.json`, run `npx nx init`.
If you don't have `nx.json`, run `npx lerna add-caching`.

:::

Expand Down
81 changes: 51 additions & 30 deletions website/docs/getting-started.md
Expand Up @@ -171,12 +171,7 @@ You should see the following output:
```

Note, `lerna` will run the three `test` npm scripts in the topological order as well. Although we had to do it when
building, it isn't necessary for tests (and it also makes the command slower). We can change this behavior by
adding `--no-sort` to the command.

```bash
npx lerna run test --no-sort
```
building, it isn't necessary for tests (and it also makes the command slower). We can change this behavior by configuring caching.

## Caching

Expand All @@ -186,25 +181,39 @@ a bit of configuration.
First, let's run

```bash
npx nx init
npx lerna add-caching
```

This which will generate a `nx.json` at the root of your workspace:
This will take you through a series of prompts to configure your caching:

```json
{
"tasksRunnerOptions": {
"default": {
"runner": "nx/tasks-runners/default",
"options": {
"cacheableOperations": []
}
}
}
}
```bash
? Which of the following scripts need to be run in deterministic/topoglogical order?
(Press <space> to select, <a> to toggle all, <i> to invert selection, and <enter> to proceed)
❯◉ build
test
◯ dev
◯ start
```

Second, let's mark `build` and `test` to be cacheable operations.
```bash
? Which of the following scripts are cacheable? (Produce the same output given the same input, e.g. build, test and lint usually are, serve and
start are not)
(Press <space> to select, <a> to toggle all, <i> to invert selection, and <enter> to proceed)
◉ build
❯◉ test
◯ dev
◯ start
```

```bash
? Does the "build" script create any outputs? If not, leave blank, otherwise provide a path relative to a project root (e.g. dist, lib, build,
coverage)
dist
? Does the "test" script create any outputs? If not, leave blank, otherwise provide a path relative to a project root (e.g. dist, lib, build,
coverage)
```

This generates an `nx.json` at the root of your workspace:

```json
{
Expand All @@ -215,10 +224,18 @@ Second, let's mark `build` and `test` to be cacheable operations.
"cacheableOperations": ["build", "test"]
}
}
},
"targetDefaults": {
"build": {
"dependsOn": ["^build"],
"outputs": ["{projectRoot}/dist"]
}
}
}
```

This configuration caches `build` and `test` tasks and forces `build` to run in topological order (but `test` will not). Also each project's `dist` folder defaults to being cached as the `build` output.

Now, let's run tests on the header project twice. The second time the operation will be instant:

```bash
Expand Down Expand Up @@ -252,10 +269,7 @@ Ran all test suites.
Lerna (powered by Nx) was able to recognize that the same command has already executed against the same relevant code
and environment, so instead running it Lerna restored the necessary files and replayed the terminal output.

Most of the time Lerna (powered by Nx) is good at recognizing what files need to be cached and restored. In case of
building the Remix app we need to help it by adding the following section to `packages/remixapp/package.json`.

> NOTE: "{projectRoot}" is a special syntax supported by the task-runner, which will be appropriately interpolated internally when the command runs. You should therefore not replace "{projectRoot}" with a fixed path as this makes your configuration less flexible.
We have specified the default build output as the `dist` folder, which works for `header` and `footer`. However, in the case of the Remix app we need to specify the output folder as `build` by adding the following section to `packages/remixapp/package.json`.

```json
{
Expand All @@ -269,6 +283,8 @@ building the Remix app we need to help it by adding the following section to `pa
}
```

> NOTE: "{projectRoot}" is a special syntax supported by the task-runner, which will be appropriately interpolated internally when the command runs. You should therefore not replace "{projectRoot}" with a fixed path as this makes your configuration less flexible.
Caching not only restores the terminal output logs, but also artifacts that might have been produced.

Build all the projects, then remove the remix build folder and run the build command again.
Expand Down Expand Up @@ -297,14 +313,19 @@ You will see all the files restored from cache and the command executing instant
## Target Dependencies (aka task pipelines)

We have made good progress, but there are two problems left to be solved:
We have made good progress, but there is one problem left to be solved. The following configuration in `nx.json` is incomplete:

1. We need to remember to use `--no-sort` when running tests.
2. We need to remember to build `header` and `footer` before we run `lerna run dev --scope=remixapp`.
```jsonc
{
"targetDefaults": {
"build": {
"dependsOn": ["^build"]
}
}
}
```

Both are the symptoms of the same issue: by default, Lerna doesn't know how different targets (npm scripts) relate to
each other. We can fix that by defining dependencies between targets (also often known as task pipelines) in
the `nx.json`:
This ensures that `build` dependencies are run before any `build` command, but we also need to remember to build `header` and `footer` before we run `lerna run dev --scope=remixapp`. We can fix that by defining dependencies between targets (also known as task pipelines) in the `nx.json`:

```json
{
Expand Down
6 changes: 2 additions & 4 deletions website/docs/lerna-and-nx.md
Expand Up @@ -45,13 +45,11 @@ Free and open source

### Set up

- `npm install nx`
- `npx nx init`
- Set `"useNx": true` in `lerna.json`
- `npx lerna add-caching`
- Continue using Lerna as usual

:::note
When Lerna is set to use Nx and detects `nx.json` with `targetDefaults` in the workspace, it will defer to Nx to detect task dependencies. Some options for `lerna run` will behave differently. See [Using Lerna (Powered by Nx) to Run Tasks](./recipes/using-lerna-powered-by-nx-to-run-tasks) for more details.
When Lerna is set to use Nx and detects `nx.json` with `targetDefaults` in the workspace, it will defer to Nx to detect task dependencies. Some options for `lerna run` will behave differently than older versions of Lerna. See [Using Lerna (Powered by Nx) to Run Tasks](docs/lerna6-obsolete-options.md) for more details.
:::

---
Expand Down
50 changes: 50 additions & 0 deletions website/docs/lerna6-obsolete-options.md
@@ -0,0 +1,50 @@
# Lerna 6: Obsolete Options

Nx and Lerna work together seamlessly in the same workspace.

When `nx.json` is detected in the current workspace, Lerna will respect the `nx.json` configuration during `lerna run`
and delegate to the Nx task runner.

Nx will run tasks in an order and with a concurrency that it determines appropriate based on the task graph that it
creates. For more information,
see [Nx Mental Model: The Task Graph](https://nx.dev/concepts/mental-model#the-task-graph).

**This behavior allows Nx to run tasks in the most efficient way possible, but it also means that some existing options
for `lerna run` become obsolete.
**

## Obsolete Options

### `--sort` and `--no-sort`

When `nx.json` is present, Lerna will always run tasks in the order it deems is correct based on its knowledge of
project and task dependencies, so `--sort` and `--no-sort` have no effect.

### `--parallel`

Lerna will use the task graph to determine which tasks can be run in parallel and do so automatically, so `--parallel`
has no effect.

:::note
If you want to limit the concurrency of tasks, you can still use
the [concurrency global option](https://github.com/lerna/lerna/blob/6cb8ab2d4af7ce25c812e8fb05cd04650105705f/core/global-options/README.md#--concurrency)
to accomplish this.
:::

### `--include-dependencies`

Lerna 6 will automatically run dependent tasks first when necessary, so `--include-dependencies` is obsolete. However,
the flag can still be used to include tasks that are not required (e.g., running the tests of all the dependent
projects).

### `--ignore`

When used with Nx, `--ignore` will never cause `lerna run` to exclude any tasks that are deemed to be
required [task graph](https://nx.dev/concepts/mental-model#the-task-graph).

:::tip

The effects on the options above will only apply if `nx.json` exists in the root with the `targetDefaults` property
defined. Otherwise, they will behave just as they would with Lerna's base task runner (if `useNx` is `false`).

:::

0 comments on commit 62b1fc8

Please sign in to comment.