Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

end_session_on_fail stopped working in version 2.0.4 #3574

Closed
joshua-redmond opened this issue Jan 17, 2023 · 1 comment · Fixed by #3595
Closed

end_session_on_fail stopped working in version 2.0.4 #3574

joshua-redmond opened this issue Jan 17, 2023 · 1 comment · Fixed by #3595
Assignees
Labels

Comments

@joshua-redmond
Copy link

joshua-redmond commented Jan 17, 2023

Description of the bug/issue

Once a test fails, I'd like to keep the browser open so I can see what went wrong.

I can do this by setting end_session_on_fail to false, which works perfectly on v1.7.13.

Once I upgraded to v2.6.8 (latest) or v2.0.4 (first 2.x version) it stopped working 🤔

end_session_on_fail still exists in the documentation, so I guess it was just overlooked in the 2.x version?

Thanks.

Steps to reproduce

  1. Run: npm init nightwatch (from https://nightwatchjs.org/guide/quickstarts/create-and-run-a-nightwatch-test.html)
  2. Add end_session_on_fail: false, to: nightwatch.conf.js
  3. Put an x in the .titleContains at nightwatch/examples/basic/ecosia.js:7 (so the test fails).
  4. Run: npx nightwatch ./nightwatch/examples/basic/ecosia.js (from the same guide)

Sample test

// Copied from the example `ecosia.js` test.

describe('Ecosia.org Demo', function() {
  before(browser => browser.navigateTo('https://www.ecosia.org/'));

  it('Demo test ecosia.org', function(browser) {
    browser
      .waitForElementVisible('body')
      .assert.titleContains('Wrong title')
  });

  after(browser => browser.end());
});

Command to run

npx nightwatch ./nightwatch/examples/basic/ecosia.js

Verbose Output

[Ecosia.org Demo] Test Suite
────────────────────────────────────────────────────────
⠋ Starting ChromeDriver on port 9515...
 Starting ChromeDriver with server_path=/c/laptop/stuff/mike/junk/node_modules/chromedriver/lib/chromedriver/chromedriver...
   Request POST /session  
   {
     capabilities: {
       firstMatch: [ {} ],
       alwaysMatch: { browserName: 'chrome', 'goog:chromeOptions': {} }
     }
⠸ Starting ChromeDriver on port 9515...
   Response 200 POST /session (253ms)
   {
     value: {
       capabilities: {
         acceptInsecureCerts: false,
         browserName: 'chrome',
         browserVersion: '109.0.5414.74',
         chrome: {
           chromedriverVersion: '109.0.5414.74 (e7c5703604daa9cc128ccf5a5d3e993513758913-refs/branch-heads/5414@{#1172})',
           userDataDir: '/tmp/.com.google.Chrome.a5XRg4'
         },
         'goog:chromeOptions': { debuggerAddress: 'localhost:44841' },
         networkConnectionEnabled: false,
         pageLoadStrategy: 'normal',
         platformName: 'linux',
         proxy: {},
         setWindowRect: true,
         strictFileInteractability: false,
         timeouts: { implicit: 0, pageLoad: 300000, script: 30000 },
         unhandledPromptBehavior: 'dismiss and notify',
         'webauthn:extension:credBlob': true,
         'webauthn:extension:largeBlob': true,
         'webauthn:virtualAuthenticators': true
       },
       sessionId: '448ef5c5e3e8ce7338d108f6803d06c6'
     }
ℹ Connected to ChromeDriver on port 9515 (281ms).
  Using: chrome (109.0.5414.74) on LINUX.

 Received session with ID: 448ef5c5e3e8ce7338d108f6803d06c6

 → Running [before]:
 
 → Running command: navigateTo ('https://www.ecosia.org/')
   Request POST /session/448ef5c5e3e8ce7338d108f6803d06c6/url  
   { url: 'https://www.ecosia.org/' }
   Response 200 POST /session/448ef5c5e3e8ce7338d108f6803d06c6/url (2032ms)
   { value: null }
  → Completed command: navigateTo ('https://www.ecosia.org/') (2034ms)
 → Completed [before].



  Running Demo test ecosia.org:
──────────────────────────────────────────────────────────────────────────────────────────────────────
 → Running [beforeEach]:
 → Completed [beforeEach].
 
 → Running command: waitForElementVisible ('body')
   Request POST /session/448ef5c5e3e8ce7338d108f6803d06c6/elements  
   { using: 'css selector', value: 'body' }
   Response 200 POST /session/448ef5c5e3e8ce7338d108f6803d06c6/elements (9ms)
   {
     value: [
       {
         'element-6066-11e4-a52e-4f735466cecf': 'e778edd8-78e0-49c6-9c08-77854bae6fb8'
       }
     ]
  }
   Request POST /session/448ef5c5e3e8ce7338d108f6803d06c6/execute/sync  
   {
     script: 'return (function(){return (function(){var k=this||self;function aa(a){return"string"==typeof a}function ba(a,b){a=a.split(".");var c=k;a[0]in c||"undefined"==typeof c.execScript||c.execScript("var "+a... (44027 characters)',
     args: [
       {
         'element-6066-11e4-a52e-4f735466cecf': 'e778edd8-78e0-49c6-9c08-77854bae6fb8',
         ELEMENT: 'e778edd8-78e0-49c6-9c08-77854bae6fb8'
       }
     ]
  }
   Response 200 POST /session/448ef5c5e3e8ce7338d108f6803d06c6/execute/sync (12ms)
   { value: true }
  ✔ Element <body> was visible after 23 milliseconds.
  → Completed command: waitForElementVisible ('body') (25ms)
 
 → Running command: assert.titleContains ('Wrong title')
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (4ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (6ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (5ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (6ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (7ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (8ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (11ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (12ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (17ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (21ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (4ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (5ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (13ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (15ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (12ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (14ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (13ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (15ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (13ms)
   { value: 'Ecosia - the search engine that plants trees' }
  → Completed command: title ([Function]) (15ms)
 
 → Running command: title ([Function])
   Request GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title  

   Response 200 GET /session/448ef5c5e3e8ce7338d108f6803d06c6/title (6ms)
   { value: 'Ecosia - the search engine that plants trees' }
  ✖ NightwatchAssertError
   Testing if the page title contains 'Wrong title' in 5000ms - expected "contains 'Wrong title'" but got: "Ecosia - the search engine that plants trees" (5149ms)

    Error location:
    /c/laptop/stuff/mike/junk/nightwatch/examples/basic/ecosia.js:
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
     5 |     browser
     6 |       .waitForElementVisible('body')
     7 |       .assert.titleContains('Wrong title') 
     8 |   });
     9 | 
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

  → Completed command: title ([Function]) (11ms)
  → Completed command: assert.titleContains ('Wrong title') (5155ms)
 → Running [afterEach]:
 → Completed [afterEach].

  FAILED: 1 assertions failed and  1 passed (5.234s)
 → Running [after]:
 
 → Running command: end ()
 
 → Running command: session ('delete', [Function])
   Request DELETE /session/448ef5c5e3e8ce7338d108f6803d06c6  

   Response 200 DELETE /session/448ef5c5e3e8ce7338d108f6803d06c6 (52ms)
   { value: null }
  → Completed command: session ('delete', [Function]) (54ms)
 Wrote log file to: /c/laptop/stuff/mike/junk/logs/basic/ecosia_chromedriver.log
  → Completed command: end () (59ms)
 → Completed [after].

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

  ️TEST FAILURE (7.942s):  
   - 1 assertions failed; 1 passed

   ✖ 1) basic/ecosia

   – Demo test ecosia.org (5.234s)

   → ✖ NightwatchAssertError
   Testing if the page title contains 'Wrong title' in 5000ms - expected "contains 'Wrong title'" but got: "Ecosia - the search engine that plants trees" (5149ms)

    Error location:
    /c/laptop/stuff/mike/junk/nightwatch/examples/basic/ecosia.js:
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
     5 |     browser
     6 |       .waitForElementVisible('body')
     7 |       .assert.titleContains('Wrong title') 
     8 |   });
     9 | 
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––


 ChromeDriver process closed.

Nightwatch Configuration

// This whole thing was generated by: `npm init nightwatch`
// The only thing I changed was adding: `end_session_on_fail: false`
module.exports = {
  end_session_on_fail: false,
  // An array of folders (excluding subfolders) where your tests are located;
  // if this is not specified, the test source must be passed as the second argument to the test runner.
  src_folders: ['test','nightwatch/examples'],

  // See https://nightwatchjs.org/guide/concepts/page-object-model.html
  page_objects_path: ['nightwatch/page-objects'],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-custom-commands.html
  custom_commands_path: ['nightwatch/custom-commands'],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-custom-assertions.html
  custom_assertions_path: ['nightwatch/custom-assertions'],

  // See https://nightwatchjs.org/guide/extending-nightwatch/adding-plugins.html
  plugins: [],
  
  // See https://nightwatchjs.org/guide/concepts/test-globals.html
  globals_path: '',
  
  webdriver: {},

  test_workers: {
    enabled: true
  },

  test_settings: {
    default: {
      disable_error_log: false,
      launch_url: 'http://localhost',

      screenshots: {
        enabled: false,
        path: 'screens',
        on_failure: true
      },

      desiredCapabilities: {
        browserName: 'chrome'
      },
      
      webdriver: {
        start_process: true,
        server_path: ''
      },
      
    },
    
    chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        'goog:chromeOptions': {
          // More info on Chromedriver: https://sites.google.com/a/chromium.org/chromedriver/
          //
          // w3c:false tells Chromedriver to run using the legacy JSONWire protocol (not required in Chrome 78)
          w3c: true,
          args: [
            //'--no-sandbox',
            //'--ignore-certificate-errors',
            //'--allow-insecure-localhost',
            //'--headless'
          ]
        }
      },

      webdriver: {
        start_process: true,
        server_path: '',
        cli_args: [
          // --verbose
        ]
      }
    },
    
  },
  
};

Nightwatch.js Version

2.6.10

Node Version

16.19.0

Browser

Chrome 109.0.5414.74

Operating System

Ubuntu 20.04.5 LTS

Additional Information

Poking around the Nightwatch code a little, I found this:
node_modules/nightwatch/lib/core/client.js:460

  /**
   * @deprecated
   */
  endSessionOnFail(val) {

It has @deprecated... so I guess this option is deprecated? The docs don't say it's deprecated though.

@beatfactor
Copy link
Member

Thanks for raising this issue, we'll look into it. The @deprecated message there just means that the method endSessionOnFail is deprecated because it's not used anymore. But the setting itself should work as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants