fix: issue with array environment variables not being handled correct…

[dmoini]

…ly (#6810)

Signed-off-by: Donovan Moini donovanmoini@gmail.com

• Closes Array in environment variables CYPRESS_BLACKLIST_HOSTS, CYPRESS_testFiles not handled correctly #6810

User facing changelog

Fixes an issue where Cypress environment variables intended to be arrays were saved as strings.

Additional details

When setting Cypress environment variables as arrays, they are stored as a string of the array instead of an array. By default, all environment variables are stored as strings, but some environment variables are intended to be numbers or booleans. That issue gets resolved in packages/server/lib/util/coerce.js by checking if any Cypress environment variables are meant to be a number or boolean, and if so converting them. But there is no check to see if a Cypress environment variable is meant to be an array.

I originally looked into using strToArray() in packages/server/lib/util/args.js, but that relies on JSON.parse(), which would throw an error for an environment variable formatted as '[bar1,bar2]' due to the array elements not containing their own quotations marks, like so: '["bar1","bar2"]'.

This issue is resolved by added a helper function (toArray()) to check if the Cypress environment variable can be converted into an array and adding toArray() to be checked in the coerce function. One important part in toArray() is updating the toString() method for the returned array. This is because the default array toString() method returns one string containing each array element separated by commas, but without '[' or ']'. If an environment variable is intended to be an array, it will be a string beginning and ending with '[' and ']' respectively. For example, if export CYPRESS_ARR=[1,2,3], then process.env['CYPRESS_ARR']: '[1,2,3]'. But [1,2,3].toString() = '1,2,3', so comparing the two values would return false and not correctly update the Cypress environment variable to be an array. To correctly compare the Cypress environment variable with the returned array from toArray(), the returned array's toString() method must be updated to include '[' and ']'.

How has the user experience changed?

Now when setting Cypress environment variables as arrays, they are correctly converted to arrays. Below are examples from #6810.

export CYPRESS_BLACKLIST_HOSTS=["www.google-analytics.com","*.hotjar.com"]
export CYPRESS_testFiles="[\"**/196*\", \"**/475*\", \"**/476*\"]"

Before:

After:

The example with CYPRESS_testFiles was originally intended as a workaround for #6810 before this pull request. While CYPRESS_testFiles is technically parsed correctly, I doubt that a user would want the array elements to contain quotation marks within their strings and have an element such as " "**/475*"" instead of "**/475*". To avoid this, make sure to set array Cypress environment variables with each element in quotation marks and each element delimited by a single comma without a whitespace (i.e. CYPRESS_testFiles=["**/196*","**/475*","**/476*"].

This also works when setting Cypress environment variables via --env.

cypress run --env foo="[bar1,bar2,bar3]"

PR Tasks

• Have tests been added/updated?
  • Note: I have added tests for adding environment variables through the command line using --env, but I am stumped how to add tests by adding environment variables via exporting them in the terminal (i.e. export CYPRESS_VAR=123). I did however find a test here that checks if the Cypress environment variable blacklistHosts can accept arrays successfully, and this test was passing when arrays passed in were just strings. Is there possibly an issue with how arrays being passed in as environment variables were being tested?

[cypress-bot]

Thanks for taking the time to open a PR!

• Create a Draft Pull Request if your PR is not ready for review. Mark the PR as Ready for Review when you're ready for a Cypress team member to review the PR.
• Become familiar with the Code Review Checklist for guidelines on coding standards and what needs to be done before a PR can be merged.

[jennifer-shehane]

@dmoini I could probably argue that this coerce method is complex enough to warrant some unit tests. So, you could make a cypress/packages/server/test/unit/coerce_spec.js file and add some tests there for input/output values.

But yah it would be nice to have some more integration tests - I think you can just set process.env.CYPRESS_testFiles = .... similar to how this test is for HTTP_PROXY env vars (making sure to clean them up in between). https://github.com/cypress-io/cypress/blob/develop/packages/server/test/unit/args_spec.js#L491:L491

[jennifer-shehane]

@dmoini Will you have time to add some more tests as described above in my comment?

[dmoini]

@jennifer-shehane Yes I have some time tomorrow to work on adding tests for the coerce function. Is the args_spec.js file that you listed above a good file to look at to get an idea of how to make tests for coerce?

[jennifer-shehane]

@dmoini Yes, this one is maybe a simpler example where you can just see how we structure unit tests in general: https://github.com/cypress-io/cypress/blob/develop/packages/server/test/unit/duration_spec.js

[dmoini]

@jennifer-shehane Just added unit tests for coerce. Check them out whenever you have the chance.

[jennifer-shehane]

Just some suggestions for more clarity in reading the code. Looks good otherwise.

[dmoini]

@jennifer-shehane Just added your suggestions.

[cypress-bot]

Released in 5.0.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v5.0.0, please open a new issue.

[jennifer-shehane]

This seems to have introduced a regression: #8818