Install ChromeDriver 404 error with install-browsers flag

[johnkim-det]

The orb does not work as expected

• what version of the orb are you currently using? 3.3.0
• paste the circle.yml file if possible or at least the relevant portion

version: 2.1

orbs:
  cypress: cypress-io/cypress@3.3.0

workflows:
  main:
    jobs:
      - cypress/run:
          install-browsers: true
          cypress-command: 'npx cypress run --component --browser chrome --headed'

• describe what you think should happen
  Should install
• describe what happens
  Error at Install ChromeDriver step:

Chrome version major is 122
Installed version of Google Chrome is 122.0.6261.94 
404
Matching Chrome Driver Version 404'd, falling back to first matching major version.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   937  100   937    0     0  13573      0 --:--:-- --:--:-- --:--:-- 13779
New ChromeDriver version to be installed: 122.0.6261.94
122.0.6261.94 will be installed
curl: (22) The requested URL returned error: 404

Exited with code exit status 22

Tried multiple versions (3.3.0, 3.2.1, 3.2.0, 3.1.4, 3.1.3), all with the same error.

[jennifer-shehane]

This was caused by an issue with browser-tools orb which is a dependency of the cypress orb. It was fixed here: CircleCI-Public/browser-tools-orb#105

We set browser-tools: 1 as the version, so my expectation is that this would update to latest underneath. I'm not completely sure how the versioning works for the orbs with interdependent orbs.

[MikeMcC399]

@jennifer-shehane

This is impacting https://github.com/cypress-io/cypress-example-kitchensink again. (See cypress-io/cypress-example-kitchensink#797).

[MikeMcC399]

@jennifer-shehane

This was caused by an issue with browser-tools orb which is a dependency of the cypress orb. It was fixed here: CircleCI-Public/browser-tools-orb#105

We set browser-tools: 1 as the version, so my expectation is that this would update to latest underneath. I'm not completely sure how the versioning works for the orbs with interdependent orbs.

CircleCI documentation states that published orbs are immutable which ties in with the observation that the old version circleci/browser-tools@1.4.6 is being used by cypress-io/cypress@3.3.0. This was the current version of circleci/browser-tools at the time that cypress-io/cypress@3.3.0 was published.

Suggestion

Bump the version of circleci/browser-tools to circleci/browser-tools@1.4.8 in

circleci-orb/src/@orb.yml

Line 13 in a5e724b

browser-tools: circleci/browser-tools@1

replacing the semver @1 with an explicit version @1.4.8

and republish the orb.

It seems that the version of circleci/browser-tools is only re-evaluated when the cypress-io/cypress orb is republished and not at run-time when the orb is used. Listing @1 is confusing since in that case neither GitHub nor CircleCI apparently documents exactly which version of circleci/browser-tools has been pulled in.

In comparison to GitHub workflows this is counter-intuitive. Nested workflows of JavaScript actions calling other JavaScript actions evaluate versions to be used at run-time.

cc: @DaleGardner

[DaleGardner]

That makes sense, great explanation Mike!

[MikeMcC399]

@jennifer-shehane

https://discuss.circleci.com/t/orb-to-orb-dependency-resolution/29012 from 2019 says

"In short—orbs are designed to be immutable, so when you import one orb in another orb, you import that orb at a specific version, which will never change unless you manually change it."

however that does not seem to be particularly clear reading the documentation

semantic versioning

[DaleGardner]

PR opened to fix this issue. @jennifer-shehane If you can review, it would be much appreciated! Then hopefully could get a new version published and resolve the error!
Thank you @MikeMcC399 for pointing this out and all the research!!

[MikeMcC399]

@DaleGardner

• After reading Orb publishing does not work for contributor PRs #451 I thought it would be better to leave the change to the core Cypress.io team, since I am external and work from forks.

[jennifer-shehane]

@MikeMcC399 Annoying. Thanks for diving into this and assessing the true state of how it works. I was confused by the semver allowance also, yet it doesn't abide by it.

[jennifer-shehane]

Please update to cypress' 3.3.1 orb to get this fix: https://github.com/cypress-io/circleci-orb/releases/tag/v3.3.1