feat: add test results output #949

mihaisee · 2023-06-21T14:55:31Z

Added a testResults output that contains a strigified JSON object with the test results.

Usage

...
  steps:
    - name: Parse results
      uses: actions/github-script@v6
      with:
        script: |
          const testResults = JSON.parse('${{ steps.cypress-run.outputs.testResults }}')
          console.log(testResults.totalPassed)
          ...

Revert back to needs.check-record-key.outputs.record-key-exists == 'true'

cypress-app-bot · 2023-06-21T14:55:48Z

Create a Draft Pull Request if your PR is not ready for review. Mark the PR as Ready for Review when you're ready for a Cypress team member to review the PR.

mihaisee · 2023-06-21T14:57:29Z

resolves #810

.gitignore

- remove .idea excluded directory

CLAassistant · 2023-06-26T08:36:37Z

All committers have signed the CLA.

Dale-777 · 2023-07-07T17:43:07Z

please can this be approved

Dale-777 · 2023-08-01T10:46:30Z

@mihaisee can you add me as a reviewer? seems this is stuck now for some time, I would like to make use of these changes.

Dale-777 · 2023-08-01T10:47:01Z

@mihaisee can you add me as a reviewer? seems this is stuck now for some time, I would like to make use of these changes.

mihaisee · 2023-08-01T11:21:17Z

@mihaisee can you add me as a reviewer? seems this is stuck now for some time, I would like to make use of these changes.

@Dale-777 sorry, but I can't add/remove reviewers

novirael · 2023-09-13T16:23:04Z

Looking forward to see that change approved!

JenoDK · 2023-09-29T09:49:05Z

Would love to see this as well +1, my use case is using these results in a different step to send an e-mail and ms teams message.

emmanuelnk · 2023-10-03T21:21:01Z

Any progress on getting this reviewed?

index.js

.github/workflows/example-recording.yml

README.md

Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com>

mschile · 2023-10-12T22:56:02Z

action.yml

@@ -96,6 +96,8 @@ outputs:
    description: 'Cypress Cloud URL if the run was recorded (deprecated)'
  resultsUrl:
    description: 'Cypress Cloud URL if the run was recorded'
+  testResults:


Thinking about this some more, we should probably name this runResults.

@mschile

The documentation currently uses the term "test results"
https://docs.cypress.io/guides/guides/module-api#Results

so if the term is changed here, it should be changed in the Module API documentation as well.

action.yml

emmanuelnk · 2023-10-17T14:23:16Z

Can I provide any help to get this moving @mihaisee ?
@mschile @MikeMcC399 any conclusion on the runResults vs testResults ?

update testResults description Co-authored-by: Matt Schile <mschile@gmail.com>

MikeMcC399 · 2023-10-23T18:31:59Z

@mschile

You need to sign the CLA

emilyrohrbough · 2023-10-26T13:27:38Z

index.js

@@ -816,6 +816,8 @@ const runTests = async () => {
  debug(`Cypress options ${JSON.stringify(cypressOptions)}`)

  const onTestsFinished = (testResults) => {
+    core.setOutput('runResults', testResults)


Discussed with the team. For simplicity, we would like to scope the exposed runResults stringified object to version, totalDuration, totalFailed, totalPassed, totalPending, totalSkipped, totalSuites and totalTests. The results object can get quite large depending on the size of your test suite and if more information is needed, it can be parsed and save in the afterRun hook.

@emilyrohrbough can logging the entire results object be an optional input parameter instead? e.g.

output-entire-results-object: true

This way, by default, a summary is output but still leaves us the option to get the entire object. This way one does not need to go in and change all the test files to log an object (only needs a version update and addition of a parameter)
CC @mihaisee

jennifer-shehane · 2023-11-02T16:24:23Z

Hi @mihaisee, will you have time to make the updates asked as Emily outlined above?

jennifer-shehane · 2023-11-28T14:22:18Z

@mihaisee Just checking in again with the status of this PR

jennifer-shehane · 2024-01-19T18:50:36Z

Closing as stale. Please open a new PR if the work can be pulled out to the smaller results output requested.

mihaisee and others added 8 commits June 13, 2023 15:20

Test summary output

9104c25

Removed debug logs

86ba246

Added some documentation for the testResultsObject output

212e2ff

Test cypress testResultsObject output in example-recording.yml

1e424a0

Test cypress testResultsObject output in example-recording.yml

8188a63

Skip parsing testResultsObject if empty

63fc380

Update example-recording.yml

3c0abeb

Revert back to needs.check-record-key.outputs.record-key-exists == 'true'

Merge branch 'cypress-io:master' into feat/add-test-reults-output

53a3e26

MikeMcC399 reviewed Jun 21, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

mike-plummer self-assigned this Jun 23, 2023

Update .gitignore

f9794b0

- remove .idea excluded directory

mike-plummer requested review from nagash77 and mjhenkes July 7, 2023 19:15

Merge branch 'master' into feat/add-test-reults-output

06567f7

Merge branch 'master' into feat/add-test-reults-output

b116a60

This comment was marked as outdated.

Sign in to view

mike-plummer assigned dkasper-was-taken and unassigned mike-plummer Oct 4, 2023

jennifer-shehane requested changes Oct 4, 2023

View reviewed changes

index.js Outdated Show resolved Hide resolved

jennifer-shehane removed the request for review from nagash77 October 4, 2023 18:21

Updated testResultsObject to testResults default - update workflow

7421400

MikeMcC399 reviewed Oct 11, 2023

View reviewed changes

.github/workflows/example-recording.yml Outdated Show resolved Hide resolved

MikeMcC399 reviewed Oct 11, 2023

View reviewed changes

README.md Outdated Show resolved Hide resolved

mihaisee and others added 3 commits October 12, 2023 08:31

Update .github/workflows/example-recording.yml

84f0d2a

Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com>

Merge branch 'master' into feat/add-test-reults-output

18af1c0

Update README.md

185d2ef

Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com>

mschile reviewed Oct 12, 2023

View reviewed changes

action.yml Outdated Show resolved Hide resolved

mihaisee and others added 4 commits October 20, 2023 11:04

Merge branch 'master' into feat/add-test-reults-output

7b06979

Update action.yml

4f33b71

update testResults description Co-authored-by: Matt Schile <mschile@gmail.com>

updating to runResults

d144906

updating to runResults

6f3e59e

jennifer-shehane requested review from jennifer-shehane and emilyrohrbough October 25, 2023 15:08

mschile assigned emilyrohrbough and unassigned mschile Oct 25, 2023

Merge branch 'master' into feat/add-test-reults-output

efff8c9

emilyrohrbough requested changes Oct 26, 2023

View reviewed changes

MikeMcC399 mentioned this pull request Nov 1, 2023

Request: Set Cypress exit code as step output #583

Open

MikeMcC399 added the type: enhancement New feature or request label Nov 4, 2023

jennifer-shehane unassigned emilyrohrbough Nov 20, 2023

jennifer-shehane removed their request for review December 19, 2023 14:17

MikeMcC399 removed the request for review from davidr-cy December 21, 2023 15:16

jennifer-shehane closed this Jan 19, 2024

MikeMcC399 mentioned this pull request Mar 12, 2024

Save Test Results to File or Action Output #1148

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add test results output #949

feat: add test results output #949

mihaisee commented Jun 21, 2023 •

edited

cypress-app-bot commented Jun 21, 2023

mihaisee commented Jun 21, 2023

CLAassistant commented Jun 26, 2023 •

edited

Dale-777 commented Jul 7, 2023

Dale-777 commented Aug 1, 2023

Dale-777 commented Aug 1, 2023

mihaisee commented Aug 1, 2023

novirael commented Sep 13, 2023

JenoDK commented Sep 29, 2023 •

edited

emmanuelnk commented Oct 3, 2023

This comment was marked as outdated.

mschile Oct 12, 2023

MikeMcC399 Oct 13, 2023

emmanuelnk commented Oct 17, 2023 •

edited

MikeMcC399 commented Oct 23, 2023 •

edited

emilyrohrbough Oct 26, 2023 •

edited

emmanuelnk Nov 9, 2023

jennifer-shehane commented Nov 2, 2023

jennifer-shehane commented Nov 28, 2023

jennifer-shehane commented Jan 19, 2024

feat: add test results output #949

feat: add test results output #949

Conversation

mihaisee commented Jun 21, 2023 • edited

cypress-app-bot commented Jun 21, 2023

mihaisee commented Jun 21, 2023

CLAassistant commented Jun 26, 2023 • edited

Dale-777 commented Jul 7, 2023

Dale-777 commented Aug 1, 2023

Dale-777 commented Aug 1, 2023

mihaisee commented Aug 1, 2023

novirael commented Sep 13, 2023

JenoDK commented Sep 29, 2023 • edited

emmanuelnk commented Oct 3, 2023

This comment was marked as outdated.

mschile Oct 12, 2023

Choose a reason for hiding this comment

MikeMcC399 Oct 13, 2023

Choose a reason for hiding this comment

emmanuelnk commented Oct 17, 2023 • edited

MikeMcC399 commented Oct 23, 2023 • edited

emilyrohrbough Oct 26, 2023 • edited

Choose a reason for hiding this comment

emmanuelnk Nov 9, 2023

Choose a reason for hiding this comment

jennifer-shehane commented Nov 2, 2023

jennifer-shehane commented Nov 28, 2023

jennifer-shehane commented Jan 19, 2024

mihaisee commented Jun 21, 2023 •

edited

CLAassistant commented Jun 26, 2023 •

edited

JenoDK commented Sep 29, 2023 •

edited

emmanuelnk commented Oct 17, 2023 •

edited

MikeMcC399 commented Oct 23, 2023 •

edited

emilyrohrbough Oct 26, 2023 •

edited