Custom commands written in Typescript break e2e tests

[IceCreamYou]

Describe the bug

After upgrading from Nightwatch 1.5.1 to Nightwatch 1.6.0 or 1.6.1, my end-to-end tests will not run. Unit tests are fine. See repro below.

Note, in the repro below I am compiling the TypeScript and electing to run the JS versions. But the same thing happens if I do not compile the TS and instead run it directly. This error does not happen if I compile to TS and then remove the command TS file, regardless of whether I'm executing a JS or TS test.

Sample test

tests/e2e/test.ts

import { NightwatchTests } from 'nightwatch';

export = {
    'Test': function(browser) {
        browser.wait(1000);
        // From https://nightwatchjs.org/guide/using-nightwatch/writing-tests.html
        browser
            .url('https://www.ecosia.org/')
            .waitForElementVisible('body')
            .assert.titleContains('Ecosia')
            .assert.visible('input[type=search]')
            .setValue('input[type=search]', 'nightwatch')
            .assert.visible('button[type=submit]')
            .click('button[type=submit]')
            .assert.containsText('.mainline-results', 'Nightwatch.js')
            .end();
    },
};

tests/commands/wait.ts

import { NightwatchAPI } from 'nightwatch';

export function command(this: NightwatchAPI, timeout = 0): NightwatchAPI {
    return this
        .perform(() => console.log(`Waiting ${timeout}ms`))
        .pause(timeout);
}

declare module 'nightwatch' {
    export interface NightwatchCustomCommands {
        wait: typeof command;
    }
}

Set up with

$ npm install
$ npx selenium-standalone install --version=3.141.59 --drivers.chrome.version="$(curl -fs "https://chromedriver.storage.googleapis.com/LATEST_RELEASE")"

Run with command

$ tsc && npx nightwatch -c ./nightwatch.conf.js --test=tests/e2e/test.js --verbose

Verbose output

debug.log

 Starting ChromeDriver on port 9515...
 ChromeDriver up and running on port 9515 with pid: 29347 (133ms).
_________________________________________________

TEST FAILURE: 1 error during execution; 0 tests failed, 0 passed (242ms)

  Error: There was an error while trying to load the file wait.ts: Cannot use import statement outside a module
       at Array.forEach (<anonymous>)
       at Array.forEach (<anonymous>)


   Error: There was an error while trying to load the file wait.ts: Cannot use import statement outside a module
       at Array.forEach (<anonymous>)
       at Array.forEach (<anonymous>)

 Wrote log file to: /Users/isaac/Code/scratch/nw-repro/tests/chromedriver.log.
 ChromeDriver process closed.

Configuration

nightwatch.conf.js

module.exports = {
    src_folders: ['tests/e2e'],
    exclude: ['**/*.ts'],
    custom_commands_path: 'tests/commands',
    webdriver: {
        start_process: true,
        server_path: './node_modules/selenium-standalone/.selenium/chromedriver/89.0.4389.23-x64-chromedriver',
        port: 9515,
        log_path: 'tests',
        request_timeout_options: { timeout: 30000, retry_attempts: 3 },
    },
    test_settings: {
        default: {
            launch_url: 'http://localhost:4900',
            desiredCapabilities: {
                browserName: 'chrome',
                javascriptEnabled: true,
                webStorageEnabled: true,
                nativeEvents: true,
                chromeOptions: {
                    prefs: {
                        'profile.default_content_setting_values.notifications': 2,
                    },
                },
            },
            skip_testcases_on_fail: false,
            compatible_testcase_support: true,
            output_folder: 'tests/output',
        },
    },
};

package.json

{
  "name": "nw-repro",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "nightwatch": "^1.6.1"
  },
  "devDependencies": {
    "@types/nightwatch": "^1.3.3",
    "selenium-standalone": "^6.23.0"
  }
}

tsconfig.json

{
    "compilerOptions": {
        "esModuleInterop": true,
        "importHelpers": true,
        "lib": ["dom", "es5", "scripthost", "es2015.promise", "es2015.proxy"],
        "module": "commonjs",
        "moduleResolution": "node",
        "outDir": ".",
        "skipLibCheck": true,
        "rootDir": ".",
        "target": "es5",
        "types": [
            "nightwatch",
        ]
    },
    "include": [
        "tests/**/*",
    ],
    "exclude": []
}

Your Environment

Executable	Version
nightwatch --version	1.6.1
npm --version	7.7.5
node --version	v14.16.0

Browser driver	Version
NAME	VERSION
chromedriver	89.0.4389.23-x64-chromedriver

OS	Version
NAME	VERSION
macOS Catalina	10.15.7

[beatfactor]

@lloiser can you suggest a fix?

[lloiser]

The problem is that nightwatch attempts to load both wait.ts and wait.js.
This approach previously (< 1.6.0) worked because nightwatch ignored all files except *.js.

Probably the best option here is to ignore the .ts file if a .js exists.
I think I can prepare a pull request if you @beatfactor agree with this solution.

[beatfactor]

@lloiser sounds good. thanks for looking into it.

[IceCreamYou]

To be clear, this problem still happens if there is no JS file. It sounds like the proposed solution (ignore .ts if .js exists) could still solve my problem, but I don't think it would enable Nightwatch to work with uncompiled TS commands.

[lloiser]

By default, NodeJS does not know how to compile and execute .ts files. Therefore it is up to the user of nightwatch to make sure that all the code is executable.

There are multiple solutions to use typescript:

• use tsc to compile all .ts files to .js before starting nightwatch (as mentioned in this issue)
• use babel - either as a pre-compile step (similar to tsc) or on the fly using @babel/register
• programmatically call ts-node to inject the typescript compiler (done in this test case: testRunnerTypeScript.js)
• (it probably is also possible to run ts-node directly to start nightwatch, but I have never tried it...)

[IceCreamYou]

Ah, I was wondering how this was supposed to work. So you're saying it is not expected for Nightwatch to work with just .ts files out of the box, without additional steps. But the fact that Nightwatch now looks for .ts files means it does not work at all with .ts files present without additional steps. And the proposed solution of ignoring .ts files if there is a corresponding .js file makes complete sense now. Thanks for clarifying.