Skip to content

end_session_on_fail stopped working in version 2.0.4 #3574

@joshua-redmond

Description

@joshua-redmond

Description of the bug/issue

Once a test fails, I'd like to keep the browser open so I can see what went wrong.

I can do this by setting end_session_on_fail to false, which works perfectly on v1.7.13.

Once I upgraded to v2.6.8 (latest) or v2.0.4 (first 2.x version) it stopped working 🤔

end_session_on_fail still exists in the documentation, so I guess it was just overlooked in the 2.x version?

Thanks.

Steps to reproduce

  1. Run: npm init nightwatch (from https://nightwatchjs.org/guide/quickstarts/create-and-run-a-nightwatch-test.html)
  2. Add end_session_on_fail: false, to: nightwatch.conf.js
  3. Put an x in the .titleContains at nightwatch/examples/basic/ecosia.js:7 (so the test fails).
  4. Run: npx nightwatch ./nightwatch/examples/basic/ecosia.js (from the same guide)

Sample test

// Copied from the example `ecosia.js` test.

describe('Ecosia.org Demo', function() {
  before(browser => browser.navigateTo('https://www.ecosia.org/'));

  it('Demo test ecosia.org', function(browser) {
    browser
      .waitForElementVisible('body')
      .assert.titleContains('Wrong title')
  });

  after(browser => browser.end());
});

Command to run

npx nightwatch ./nightwatch/examples/basic/ecosia.js

Verbose Output

[Ecosia.org Demo] Test Suite
────────────────────────────────────────────────────────
⠋ Starting ChromeDriver on port 9515...
 Starting ChromeDriver with server_path=/c/laptop/stuff/mike/junk/node_modules/chromedriver/lib/chromedriver/chromedriver...
   Request POST /session  
   {
     capabilities: {
       firstMatch: [ {} ],
       alwaysMatch: { browserName: 'chrome', 'goog:chromeOptions': {} }
     }
⠸ Starting ChromeDriver on port 9515...
   Response 200 POST /session (253ms)
   {
     value: {
       capabilities: {
         acceptInsecureCerts: false,
         browserName: 'chrome',
         browserVersion: '109.0.5414.74',
         chrome: {
           chromedriverVersion: '109.0.5414.74 (e7c5703604daa9cc128ccf5a5d3e993513758913-refs/branch-heads/5414@{#1172})',
           userDataDir: '/tmp/.com.google.Chrome.a5XRg4'
         },
         'goog:chromeOptions': { debuggerAddress: 'localhost:44841' },
         networkConnectionEnabled: false,
         pageLoadStrategy: 'normal',
         platformName: 'linux',
         proxy: {},
         setWindowRect: true,
         strictFileInteractability: false,
         timeouts: { implicit: 0, pageLoad: 300000, script: 30000 },
         unhandledPromptBehavior: 'dismiss and notify',
         'webauthn:extension:credBlob': true,
         'webauthn:extension:largeBlob': true,
         'webauthn:virtualAuthenticators': true
       },
       sessionId: '448ef5c5e3e8ce7338d108f6803d06c6'
     }
ℹ Connected to ChromeDriver on port 9515 (281ms).
  Using: chrome (109.0.5414.74) on LINUX.

 Received session with ID: 448ef5c5e3e8ce7338d108f6803d06c6

 → Running [before]:
 
 → Running command: navigateTo ('https://www.ecosia.org/')
   Request POST /session/448ef5c5e3e8ce7338d108f6803d06c6/url  
   { url: 'https://www.ecosia.org/' }
   Response 200 POST /session/448ef5c5e3e8ce7338d108f6803d06c6/url (2032ms)
   { value: null }
  → Completed command: navigateTo ('https://www.ecosia.org/') (2034ms)
 → Completed [before].



  Running Demo test ecosia.org:
──────────────────────────────────────────────────────────────────────────────────────────────────────
 → Running [beforeEach]:
 → Completed [beforeEach].
 
 → Running command: waitForElementVisible ('body')
   Request POST /session/448ef5c5e3e8ce7338d108f6803d06c6/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /session/448ef5c5e3e8ce7338d108f6803d06c6/elements (9ms)
   {
     value: [
       {
         'element-6066-11e4-a52e-4f735466cecf': 'e778edd8-78e0-49c6-9c08-77854bae6fb8'
       }
     ]
  }
   Request POST /session/448ef5c5e3e8ce7338d108f6803d06c6/execute/sync  
   {
     script: 'return (function(){return (function(){var k=this||self;function aa(a){return"string"==typeof a}function ba(a,b){a=a.split(".");var c=k;a[0]in c||"undefined"==typeof c.execScript||c.execScript("var "+a... (44027 characters)',
     args: [
       {
         'element-6066-11e4-a52e-4f735466cecf': 'e778edd8-78e0-49c6-9c08-77854bae6fb8',
         ELEMENT: 'e778edd8-78e0-49c6-9c08-77854bae6fb8'
       }
     ]
  }
   Response 200 POST /session/448ef5c5e3e8ce7338d108f6803d06c6/execute/sync (12ms)
   { value: true }
  ✔ Element <body> was visible after 23 milliseconds.
  → Completed command: waitForElementVisible ('body') (25ms)
 
 → Running command: assert.titleContains ('Wrong title')
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (4ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (6ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (5ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (6ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (7ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (8ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (11ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (12ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (17ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (21ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (4ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (5ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (13ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (15ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (12ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (14ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (13ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (15ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (13ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (15ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (6ms)
   { value: 'Ecosia - the search engine that plants trees' }
  ✖ NightwatchAssertError
   Testing if the page title contains 'Wrong title' in 5000ms - expected "contains 'Wrong title'" but got: "Ecosia - the search engine that plants trees" (5149ms)

    Error location:
    /c/laptop/stuff/mike/junk/nightwatch/examples/basic/ecosia.js:
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
     5 |     browser
     6 |       .waitForElementVisible('body')
     7 |       .assert.titleContains('Wrong title') 
     8 |   });
     9 | 
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

  → Completed command: title ([Function]) (11ms)
  → Completed command: assert.titleContains ('Wrong title') (5155ms)
 → Running [afterEach]:
 → Completed [afterEach].

  FAILED: 1 assertions failed and  1 passed (5.234s)
 → Running [after]:
 
 → Running command: end ()
 
 → Running command: session ('delete', [Function])
   Request DELETE /session/448ef5c5e3e8ce7338d108f6803d06c6  

   Response 200 DELETE /session/448ef5c5e3e8ce7338d108f6803d06c6 (52ms)
   { value: null }
  → Completed command: session ('delete', [Function]) (54ms)
 Wrote log file to: /c/laptop/stuff/mike/junk/logs/basic/ecosia_chromedriver.log
  → Completed command: end () (59ms)
 → Completed [after].

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

  ️TEST FAILURE (7.942s):  
   - 1 assertions failed; 1 passed

   ✖ 1) basic/ecosia

   – Demo test ecosia.org (5.234s)

   → ✖ NightwatchAssertError
   Testing if the page title contains 'Wrong title' in 5000ms - expected "contains 'Wrong title'" but got: "Ecosia - the search engine that plants trees" (5149ms)

    Error location:
    /c/laptop/stuff/mike/junk/nightwatch/examples/basic/ecosia.js:
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
     5 |     browser
     6 |       .waitForElementVisible('body')
     7 |       .assert.titleContains('Wrong title') 
     8 |   });
     9 | 
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––


 ChromeDriver process closed.

Nightwatch Configuration

// This whole thing was generated by: `npm init nightwatch`
// The only thing I changed was adding: `end_session_on_fail: false`
module.exports = {
  end_session_on_fail: false,
  // An array of folders (excluding subfolders) where your tests are located;
  // if this is not specified, the test source must be passed as the second argument to the test runner.
  src_folders: ['test','nightwatch/examples'],

  // See https://nightwatchjs.org/guide/concepts/page-object-model.html
  page_objects_path: ['nightwatch/page-objects'],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-custom-commands.html
  custom_commands_path: ['nightwatch/custom-commands'],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-custom-assertions.html
  custom_assertions_path: ['nightwatch/custom-assertions'],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-plugins.html
  plugins: [],
  
  // See https://nightwatchjs.org/guide/concepts/test-globals.html
  globals_path: '',
  
  webdriver: {},

  test_workers: {
    enabled: true
  },

  test_settings: {
    default: {
      disable_error_log: false,
      launch_url: 'http://localhost',

      screenshots: {
        enabled: false,
        path: 'screens',
        on_failure: true
      },

      desiredCapabilities: {
        browserName: 'chrome'
      },
      
      webdriver: {
        start_process: true,
        server_path: ''
      },
      
    },
    
    chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        'goog:chromeOptions': {
          // More info on Chromedriver: https://sites.google.com/a/chromium.org/chromedriver/
          //
          // w3c:false tells Chromedriver to run using the legacy JSONWire protocol (not required in Chrome 78)
          w3c: true,
          args: [
            //'--no-sandbox',
            //'--ignore-certificate-errors',
            //'--allow-insecure-localhost',
            //'--headless'
          ]
        }
      },

      webdriver: {
        start_process: true,
        server_path: '',
        cli_args: [
          // --verbose
        ]
      }
    },
    
  },
  
};

Nightwatch.js Version

2.6.10

Node Version

16.19.0

Browser

Chrome 109.0.5414.74

Operating System

Ubuntu 20.04.5 LTS

Additional Information

Poking around the Nightwatch code a little, I found this:
node_modules/nightwatch/lib/core/client.js:460

  /**
   * @deprecated
   */
  endSessionOnFail(val) {

It has @deprecated... so I guess this option is deprecated? The docs don't say it's deprecated though.

Activity

beatfactor

beatfactor commented on Jan 18, 2023

@beatfactor
Member

Thanks for raising this issue, we'll look into it. The @deprecated message there just means that the method endSessionOnFail is deprecated because it's not used anymore. But the setting itself should work as expected.

self-assigned this
on Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Participants

    @beatfactor@joshua-redmond@gravityvi

    Issue actions

      `end_session_on_fail` stopped working in version 2.0.4 · Issue #3574 · nightwatchjs/nightwatch