Create full build pipeline in Azure DevOps

[Daniel15]

Following on from #6495, it would be good to configure the full build pipeline in Azure DevOps so we consider using it exclusively. It should simplify releases a bit as we wouldn't have to wait for both CircleCI and AppVeyor to finish the build, and won't encounter issues where one passes while the other fials.

Parts of the build:

• Restore packages (yarn install --frozen-lockfile)
• For non-release builds, set the version number (./scripts/set-dev-version.js)
• Build dist (yarn build-dist)
• Run tests
• Build Windows installer (yarn build-win-installer). Must be ran on Windows
• Build Debian and RPM packages (./scripts/build-deb.sh). Must be ran on Linux
• For master builds:
  • Save the artifacts to https://yarnpkg.com/en/docs/nightly. Currently done as a webhook call to a remote server that downloads the files via AppVeyor/CircleCI APIs
• For tagged version builds:
  • Publish to npm
  • Publish to everything else (GPG sign, push to GitHub releases, update Debian repo, update RPM repo). Currently done as a webhook

The current Azure DevOps pipeline runs some acceptance tests, and builds Yarn for each config (OS + Node.js version). I think we actually just need to restore packages and build Yarn once, and then can reuse those built artifacts for each test job (rather than having multiple jobs that each build then test Yarn).

One potential approach I can see for this is:

• Build job that runs on Linux - Restores packages, builds dist + the Debian and RPM packages, saves pipeline artifacts
• Windows Build job - Restores pipeline artifacts from build job, builds the Windows installer, saves artifact (.msi file)
• Test jobs for each combination we support
• Publish job to publish everything, for tagged releases, only if the build fails

Need to figure out whether Azure DevOps supports webhooks for deployment (either post the artifacts to the webhook, or allow retrieving them via an API).

cc @arcanis

[btholt]

Please @ me if you need me to chase anything down on the Microsoft side for you.

[Daniel15]

Thanks @btholt! I think the main thing for me is finding time to work out how everything works 😛 The docs seem pretty good though. Will try to take a look soon.

[Daniel15]

@btholt - Is it possible to install prerequisites on the build system? For example, our Windows build requires WiX (http://wixtoolset.org/). On Linux I'd create a Docker image with everything installed, but I'm not sure if something similar could be done with Windows? I could pprobably make it a step in the build process (download the MSI and install/extract it), but that would slow down the builds.

[btholt]

Let me ping the DevOps team and get back to you!

[hross]

Our image already has Wix preinstalled (see here), so if you use the base image you shouldn't need a container. If you want to you could create a container with everything but then you would have to download the container as part of the build process.