Experiment: Replace lerna with yarn2 workspaces

[tylerbutler]

Note: closely related to PR #4593.

This draft PR is an experiment of replacing our custom fluid-build tool and lerna with yarn2
workspaces and lage.

Note: using lage here is incidental. Fluid-build already paralellizes things efficiently, but I didn't have time to
make code changes to fluid-build.

We can decide on package manager vs "task runner" independently, including updating fluid-build as needed.

I do not intend to merge this in; I am sharing it in case anyone else is investigating speeding up the repo init
experience.

Goals of the experiment:

• Speed up installation/bootstrapping of a new Fluid repo clone.
• Speed up experimentation with new packages or mass package upgrades (no lengthy lerna bootstrap step to change deps).
• Save disk space when using multiple Fluid repo clones.
• Determine what, if anything, doesn't work in this setup.

How to try it

1. Install yarn globally: npm i -g yarn.
2. Run yarn install.
3. Use yarn lage l-build to run a build using lage.

Note that the build will fail. Likely with an error like this:

ERR! [@fluidframework/container-loader tsc] ERROR DETECTED
ERR! started
ERR! Running C:\Users\tylerbu\AppData\Local\Temp\xfs-a2e3f87a\yarn.EXE run tsc
ERR! src/container.ts:1210:26 - error TS2554: Expected 9 arguments, but got 8.
ERR!
ERR! 1210         const protocol = new ProtocolOpHandler(
ERR!                               ~~~~~~~~~~~~~~~~~~~~~~
ERR! 1211             attributes.minimumSequenceNumber,
ERR!      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERR!  ...
ERR! 1217             (key, value) => this.submitMessage(MessageType.Propose, { key, value }),
ERR!      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERR! 1218             (sequenceNumber) => this.submitMessage(MessageType.Reject, sequenceNumber));
ERR!      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERR!
ERR!   ../../../server/routerlicious/packages/protocol-base/dist/protocol.d.ts:24:299
ERR!     24     constructor(branchId: string, minimumSequenceNumber: number, sequenceNumber: number, term: number | undefined, members: [string, ISequencedClient][], proposals: [number, ISequencedProposal, string[]][], values: [string, ICommittedProposal][], sendProposal: (key: string, value: any) => number, sendReject: (sequenceNumber: number) => void);
ERR!


       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERR!     An argument for 'sendReject' was not provided.
ERR!
ERR!
ERR! Found 1 error.
ERR!
ERR! failed

That's the current state.

I got much further in the compilation progress, but things regressed somehow over the past few days. One thing to try is using yarn to build just the server packages first:

cd server/routerlicious
yarn workspaces foreach --topological-dev --parallel --verbose --interlaced run build:compile

Then go back to the root and try yarn lage l-build again. It might get further. You can also try building using yarn itself from the root of the repo:

yarn workspaces foreach --topological-dev --parallel --verbose --interlaced run tsc

Workspace structure

Yarn supports nested workspaces to some extent, so it actually supports our repo structure pretty well already. Here's
the root workspace config:

  "workspaces": [
    "packages/*/*",
    "examples/**",
    "server/historian/",
    "server/routerlicious/",
    "common/"
  ]

The last three entries are also worktrees (more at
https://yarnpkg.com/features/workspaces#how-to-declare-a-worktree), and define subpaths that they encompass.

You can list all the workspaces using yarn workspaces list.

Features

Yarn v2 seems particularly well suited to monorepo management. It will determine, based on the version numbers in
package dependencies, whether to install a package from npm or use the one in the repo. You can also explicitly
define these relationships and say I always want the workspace version.

It also has some very nice interactive package management features. Try going to a non-React package in the repo and
typing yarn add react-dom. You can select from versions influenced by what is already used in the repo. You can also
try yarn upgrade-interactive.

It can help deduplicate packages within the repo: yarn dedupe --check

There are also things I haven't fully explored, like the release workflows and constraints, but they look very
useful from a monorepo management perspective.

Known issues and caveats

Caching doesn't work. See microsoft/lage#129 and microsoft/workspace-tools#36.