Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Recovered Error Handling and Session Recovered Errors #24511

Merged
merged 67 commits into from Nov 10, 2022
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
bd0a0ee
start
emilyrohrbough Sep 16, 2022
159e4ad
[skip ci]
emilyrohrbough Sep 22, 2022
fa44f35
some progress (branch appears out-of-date)
emilyrohrbough Sep 23, 2022
d6dde1c
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Sep 23, 2022
1d3584d
Correctly handle errors thrown from cypress commands in session setup…
emilyrohrbough Sep 23, 2022
4dc9e6b
more granular error tests
emilyrohrbough Sep 23, 2022
0dfbb99
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Oct 11, 2022
b10cdc4
Fix missed merge conflict & code check-in.
emilyrohrbough Oct 12, 2022
c2a1b5c
great time to check some code in! next steps...aligning the stack tra…
emilyrohrbough Oct 13, 2022
342f701
great checkin point -- going to cherry pick the UI changes out into t…
emilyrohrbough Oct 14, 2022
fea09e9
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Oct 14, 2022
c868386
no ci
emilyrohrbough Oct 14, 2022
e25419b
feat: enhance test error UI
emilyrohrbough Oct 14, 2022
f09ba08
fix tests
emilyrohrbough Oct 14, 2022
85a2b07
Break this up a bit more
emilyrohrbough Oct 14, 2022
2b593de
clean up
emilyrohrbough Oct 14, 2022
db28fd1
minor style fixes in alignment & border color
emilyrohrbough Oct 17, 2022
bfa869f
Merge branch 'develop' into reporter-error-ui
emilyrohrbough Oct 17, 2022
9e7a463
Merge remote-tracking branch 'origin/reporter-error-ui' into sessions…
emilyrohrbough Oct 17, 2022
fdda301
minor padding tweaks
emilyrohrbough Oct 17, 2022
56f4c2a
Merge branch 'reporter-error-ui' of https://github.com/cypress-io/cyp…
emilyrohrbough Oct 17, 2022
2cd8cae
Merge remote-tracking branch 'origin/reporter-error-ui' into sessions…
emilyrohrbough Oct 17, 2022
0ab8846
tests for create session setup / validation errors
emilyrohrbough Oct 18, 2022
d51cc95
write test coverage for recreated sessions
emilyrohrbough Oct 20, 2022
cdafd9f
okay !!! I think we have fixed all the recovered states. check in dump
emilyrohrbough Oct 21, 2022
27ce5e4
no only to run in CI
emilyrohrbough Oct 21, 2022
338d3b8
CI doesnt even like .only in comments :(
emilyrohrbough Oct 21, 2022
b904344
okay check in point. Still not there yet
emilyrohrbough Oct 21, 2022
2446db9
:D whoops.
emilyrohrbough Oct 21, 2022
7d4e8e7
fix stack traces
emilyrohrbough Oct 24, 2022
18761c9
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Oct 24, 2022
4902429
cleanup
emilyrohrbough Oct 24, 2022
898cab6
some progress. still not there. need to fix session tests & likely fi…
emilyrohrbough Oct 24, 2022
8d36df5
clean up.
emilyrohrbough Oct 25, 2022
5cf2ad7
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Oct 25, 2022
c41e537
fix utils tests
emilyrohrbough Oct 25, 2022
44727c4
app tests are passing!!!
emilyrohrbough Oct 26, 2022
87ffa23
and this.
emilyrohrbough Oct 26, 2022
125df05
LETS GOOOOOOOOOOO!!!!! all my session tests are passing! lets see if …
emilyrohrbough Oct 26, 2022
c602c34
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Oct 26, 2022
200b7f7
fix a handful of errors
emilyrohrbough Oct 26, 2022
e17ee7d
fix tests !!
emilyrohrbough Oct 27, 2022
7a490ba
and fix.
emilyrohrbough Oct 27, 2022
3719c91
handle multiple command queue errors
emilyrohrbough Nov 1, 2022
3789d5d
fix
emilyrohrbough Nov 1, 2022
0cdadc8
fix tests
emilyrohrbough Nov 1, 2022
7a3ece9
fix tests & some type errors
emilyrohrbough Nov 1, 2022
5b137ec
fix tests
emilyrohrbough Nov 1, 2022
f34fac4
fix types
emilyrohrbough Nov 2, 2022
d6a6f90
and fix code frames
emilyrohrbough Nov 2, 2022
c515ae0
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Nov 3, 2022
28f6add
fix ensure exists to be a cypress error instead of an assertion error
emilyrohrbough Nov 3, 2022
443e311
snapshot cache
emilyrohrbough Nov 3, 2022
ff8deb5
Fix codeframe issue & redundant reported log message & error nesting.
emilyrohrbough Nov 4, 2022
4fe9cf0
address PR feedback
emilyrohrbough Nov 4, 2022
8952a2b
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Nov 7, 2022
2921590
fix tests
emilyrohrbough Nov 8, 2022
3f2e50d
maintain error message parity with current cypress behavior.
emilyrohrbough Nov 8, 2022
1f87967
fix
emilyrohrbough Nov 8, 2022
e113104
fix
emilyrohrbough Nov 8, 2022
d1eee3a
fix test
emilyrohrbough Nov 8, 2022
2a7bb06
correct types
emilyrohrbough Nov 8, 2022
b8e919f
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Nov 8, 2022
1d52b05
fix test for webkit & firefox
emilyrohrbough Nov 8, 2022
8584bda
Merge branch 'sessions-in-test-errors' of https://github.com/cypress-…
emilyrohrbough Nov 8, 2022
919dd4a
fix.
emilyrohrbough Nov 8, 2022
9172359
Merge branch 'develop' into sessions-in-test-errors
emilyrohrbough Nov 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
391 changes: 365 additions & 26 deletions packages/app/cypress/e2e/runner/sessions.ui.cy.ts

Large diffs are not rendered by default.

16 changes: 12 additions & 4 deletions packages/app/src/runner/logger.ts
Expand Up @@ -8,9 +8,11 @@ interface Table {
}

interface Group {
items: any
label: boolean
name: string
items?: any
label?: boolean
expand?: boolean
table?: boolean
}

export const logger = {
Expand Down Expand Up @@ -93,7 +95,12 @@ export const logger = {
const groups = this._getGroups(consoleProps)

_.each(groups, (group) => {
console.groupCollapsed(group.name)
if (group.expand) {
console.group(group.name)
} else {
console.groupCollapsed(group.name)
}

_.each(group.items, (value, key) => {
if (group.label === false) {
this.log(value)
Expand All @@ -102,6 +109,7 @@ export const logger = {
}
})

this._logGroups(group)
console.groupEnd()
})
},
Expand All @@ -112,7 +120,7 @@ export const logger = {
if (!groups) return

return _.map(groups, (group) => {
group.items = this._formatted(group.items)
group.items = this._formatted(group.items || {})

return group
})
Expand Down
31 changes: 13 additions & 18 deletions packages/driver/cypress/e2e/commands/actions/click.cy.js
Expand Up @@ -985,8 +985,8 @@ describe('src/cy/commands/actions/click', () => {
cy.get('#ptrNone').click({ timeout: 300, force: true })
})

it('should error with message about pointer-events', function () {
const onError = cy.stub().callsFake((err) => {
it('should error with message about pointer-events', function (done) {
cy.once('fail', (err) => {
const { lastLog } = this

expect(err.message).to.contain('has CSS `pointer-events: none`')
Expand All @@ -1001,18 +1001,14 @@ describe('src/cy/commands/actions/click', () => {
])

expect(consoleProps['But it has CSS']).to.eq('pointer-events: none')
done()
})

cy.once('fail', onError)

cy.get('#ptrNone').click({ timeout: 300 })
.then(() => {
expect(onError).calledOnce
})
})

it('should error with message about pointer-events and include inheritance', function () {
const onError = cy.stub().callsFake((err) => {
it('should error with message about pointer-events and include inheritance', function (done) {
cy.once('fail', (err) => {
const { lastLog } = this

expect(err.message).to.contain('has CSS `pointer-events: none`, inherited from this element:')
Expand All @@ -1030,14 +1026,10 @@ describe('src/cy/commands/actions/click', () => {
expect(consoleProps['But it has CSS']).to.eq('pointer-events: none')

expect(consoleProps['Inherited From']).to.eq(this.ptrNone.get(0))
done()
})

cy.once('fail', onError)

cy.get('#ptrNoneChild').click({ timeout: 300 })
.then(() => {
expect(onError).calledOnce
})
})
})

Expand Down Expand Up @@ -2096,10 +2088,8 @@ describe('src/cy/commands/actions/click', () => {
cy.on('log:added', (attrs, log) => {
this.lastLog = log

this.logs.push(log)
this.logs?.push(log)
})

null
})

it('throws when not a dom subject', (done) => {
Expand Down Expand Up @@ -2239,7 +2229,7 @@ describe('src/cy/commands/actions/click', () => {
const $btn = $('<button>button covered</button>').attr('id', 'button-covered-in-span').prependTo(cy.$$('body'))
const span = $('<span>span on button</span>').css({ position: 'absolute', left: $btn.offset().left, top: $btn.offset().top, padding: 5, display: 'inline-block', backgroundColor: 'yellow' }).prependTo(cy.$$('body'))

cy.on('fail', (err) => {
cy.once('fail', (err) => {
const { lastLog } = this

// get + click logs
Expand Down Expand Up @@ -4202,6 +4192,7 @@ describe('mouse state', () => {

e.target.removeEventListener('pointerout', pointerout)
}).as('pointerout')

const pointerleave = cy.stub().callsFake((e) => {
const exp = {
altKey: false,
Expand Down Expand Up @@ -4239,6 +4230,7 @@ describe('mouse state', () => {

e.target.removeEventListener('pointerleave', pointerleave)
}).as('pointerleave')

const mouseover = cy.stub().callsFake((e) => {
const exp = {
altKey: false,
Expand Down Expand Up @@ -4276,6 +4268,7 @@ describe('mouse state', () => {

e.target.removeEventListener('mouseover', mouseover)
}).as('mouseover')

const mouseenter = cy.stub().callsFake((e) => {
const exp = {
altKey: false,
Expand Down Expand Up @@ -4313,6 +4306,7 @@ describe('mouse state', () => {

e.target.removeEventListener('mouseenter', mouseenter)
}).as('mouseenter')

const pointerover = cy.stub().callsFake((e) => {
const exp = {
altKey: false,
Expand Down Expand Up @@ -4350,6 +4344,7 @@ describe('mouse state', () => {

e.target.removeEventListener('pointerover', pointerover)
}).as('pointerover')

const pointerenter = cy.stub().callsFake((e) => {
const exp = {
altKey: false,
Expand Down
Expand Up @@ -1309,10 +1309,14 @@ describe('src/cy/commands/actions/type - #type special chars', () => {
const state = log.get('state')

if (state === 'pending') {
log.on('state:changed', (state) => {
return events.push(`${log.get('name')}:log:${state}`)
})
events.push(`${log.get('name')}:log:${state}`)
}
})

cy.on('command:log:changed', (attrs, log) => {
const state = log.get('state')

if (state === 'pending') {
events.push(`${log.get('name')}:log:${state}`)
}
})
Expand All @@ -1323,7 +1327,14 @@ describe('src/cy/commands/actions/type - #type special chars', () => {

cy.get('#single-input input').type('f{enter}').then(() => {
expect(events).to.deep.eq([
'get:start', 'get:log:pending', 'get:end', 'type:start', 'type:log:pending', 'submit', 'type:end', 'then:start',
'get:start',
'get:log:pending',
'get:end',
'type:start',
'type:log:pending',
'submit',
'type:end',
'then:start',
])
})
})
Expand Down
38 changes: 22 additions & 16 deletions packages/driver/cypress/e2e/commands/assertions.cy.js
Expand Up @@ -48,16 +48,16 @@ describe('src/cy/commands/assertions', () => {
this.logs = []

cy.on('log:added', (attrs, log) => {
this.logs.push(log)
this.logs?.push(log)
this.lastLog = log
})

return null
})

it('returns the subject for chainability', () => {
cy
.noop({ foo: 'bar' }).should('deep.eq', { foo: 'bar' })
cy.noop({ foo: 'bar' })
.should('deep.eq', { foo: 'bar' })
.then((obj) => {
expect(testCommands()).to.eql([
{ name: 'visit', snapshots: 1, retries: 0 },
Expand Down Expand Up @@ -368,7 +368,7 @@ describe('src/cy/commands/assertions', () => {

cy.on('log:added', (attrs, log) => {
if (log.get('name') === 'assert') {
logs.push(log)
logs?.push(log)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was seeing a ton of errors for cannot pushed to defined throughout our tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious. How is this.logs undefined when it's defined above?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I seemed to happen it was when I'd change a file or the spec file and Cypress would start to rerun the spec. 🤷🏻‍♀️Maybe the previous cy.on listener is still attached when the page is loading?? Seems a bit odd to me, but was tired of getting uncaught exception logs when debugging 😬

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we drop a comment there to explain this behavior incase someone runs into it again and thinks the optional chainer isn't necessary?


if (logs.length === 3) {
done()
Expand Down Expand Up @@ -416,34 +416,40 @@ describe('src/cy/commands/assertions', () => {
it('does not log extra commands on fail and properly fails command + assertions', function (done) {
cy.on('fail', (err) => {
assertLogLength(this.logs, 6)
expect(err.message).to.eq('You must provide a valid number to a `length` assertion. You passed: `asdf`')

expect(this.logs[3].get('name')).to.eq('get')
expect(this.logs[3].get('state')).to.eq('failed')
expect(this.logs[3].get('error')).to.eq(err)
expect(this.logs[3].get('state')).to.eq('passed')
expect(this.logs[3].get('error')).to.be.undefined

expect(this.logs[4].get('name')).to.eq('assert')
expect(this.logs[4].get('state')).to.eq('failed')
expect(this.logs[4].get('error').name).to.eq('AssertionError')
expect(this.logs[4].get('error').name).to.eq('CypressError')
expect(this.logs[4].get('error')).to.eq(err)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Get .get('button') is no longer marked as failed because it resolved correctly and the assertion was the failure.

Before After
Screen Shot 2022-11-02 at 5 00 15 PM Screen Shot 2022-11-02 at 4 53 54 PM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the assertion fails, I would expect an AssertionError to be thrown here and not a CypressError. What was the rason that changed? Or is it that the argument into the assertion is bad?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


done()
})

cy
.root().should('exist').and('contain', 'foo')
.get('button').should('have.length', 'asdf')
.root()
.should('exist')
.and('contain', 'foo')
.get('button')
.should('have.length', 'asdf')
})

it('finishes failed assertions and does not log extra commands when cy.contains fails', function (done) {
cy.on('fail', (err) => {
assertLogLength(this.logs, 2)

expect(this.logs[0].get('name')).to.eq('contains')
expect(this.logs[0].get('state')).to.eq('failed')
expect(this.logs[0].get('error')).to.eq(err)
expect(this.logs[0].get('state')).to.eq('passed')
expect(this.logs[0].get('error')).to.be.undefined

expect(this.logs[1].get('name')).to.eq('assert')
expect(this.logs[1].get('state')).to.eq('failed')
expect(this.logs[1].get('error').name).to.eq('AssertionError')
expect(this.logs[1].get('error').name).to.eq('CypressError')
expect(this.logs[1].get('error')).to.eq(err)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before After
Screen Shot 2022-11-03 at 8 10 02 AM Screen Shot 2022-11-03 at 8 11 24 AM

done()
})
Expand Down Expand Up @@ -731,7 +737,7 @@ describe('src/cy/commands/assertions', () => {
})

it('does not additionally log when .should is the current command', function (done) {
cy.on('fail', (err) => {
cy.once('fail', (err) => {
const { lastLog } = this

assertLogLength(this.logs, 1)
Expand Down Expand Up @@ -822,7 +828,7 @@ describe('src/cy/commands/assertions', () => {
this.logs = []

cy.on('log:added', (attrs, log) => {
this.logs.push(log)
this.logs?.push(log)

if (attrs.name === 'assert') {
this.lastLog = log
Expand Down Expand Up @@ -1179,7 +1185,7 @@ describe('src/cy/commands/assertions', () => {
this.logs = []

cy.on('log:added', (attrs, log) => {
this.logs.push(log)
this.logs?.push(log)
})

return null
Expand Down Expand Up @@ -1621,7 +1627,7 @@ describe('src/cy/commands/assertions', () => {
}

cy.on('log:added', (attrs, log) => {
this.logs.push(log)
this.logs?.push(log)
})

return null
Expand Down