Skip to content

Releases: codeceptjs/CodeceptJS

3.6.2

12 May 17:33
418245a
Compare
Choose a tag to compare

What's Changed

Full Changelog: 3.6.1...3.6.2

3.6.1

18 Apr 08:28
73f38dc
Compare
Choose a tag to compare

Fixed regression in interactive pause.

3.6.0

11 Apr 09:11
3989763
Compare
Choose a tag to compare

3.6.0

🛩️ Features

  • Introduced healers to improve stability of failed tests. Write functions that can perform actions to fix a failing test:
heal.addRecipe('reloadPageIfModalIsNotVisisble', {
  steps: [
    'click',
  ],
  fn: async ({ error, step }) => {
    // this function will be executed only if test failed with
    // "model is not visible" message
    if (error.message.include('modal is not visible')) return;

    // we return a function that will refresh a page
    // and tries to perform last step again
    return async ({ I }) => {
      I.reloadPage();
      I.wait(1);
      await step.run();
    };
    // if a function succeeds, test continues without an error
  },
});
  • Breaking Change AI features refactored. Read updated AI guide:

    • removed dependency on openai
    • added support for Azure OpenAI, Claude, Mistal, or any AI via custom request function
    • --ai option added to explicitly enable AI features
    • heal plugin decoupled from AI to run custom heal recipes
    • improved healing for async/await scenarios
    • token limits added
    • token calculation introduced
    • OpenAI helper renamed to AI
  • feat(puppeteer): network traffic manipulation. See #4263 by @kobenguyent

    • startRecordingTraffic
    • grabRecordedNetworkTraffics
    • flushNetworkTraffics
    • stopRecordingTraffic
    • seeTraffic
    • dontSeeTraffic
  • feat(Puppeteer): recording WS messages. See #4264 by @kobenguyent

Recording WS messages:

      I.startRecordingWebSocketMessages();
      I.amOnPage('https://websocketstest.com/');
      I.waitForText('Work for You!');
      const wsMessages = I.grabWebSocketMessages();
      expect(wsMessages.length).to.greaterThan(0);

flushing WS messages:

      I.startRecordingWebSocketMessages();
      I.amOnPage('https://websocketstest.com/');
      I.waitForText('Work for You!');
      I.flushWebSocketMessages();
      const wsMessages = I.grabWebSocketMessages();
      expect(wsMessages.length).to.equal(0);

Examples:

// recording traffics and verify the traffic
  I.startRecordingTraffic();
  I.amOnPage('https://codecept.io/');
  I.seeTraffic({ name: 'traffics', url: 'https://codecept.io/img/companies/BC_LogoScreen_C.jpg' });
// check the traffic with advanced params
  I.amOnPage('https://openai.com/blog/chatgpt');
  I.startRecordingTraffic();
  I.seeTraffic({
    name: 'sentry event',
    url: 'https://images.openai.com/blob/cf717bdb-0c8c-428a-b82b-3c3add87a600',
    parameters: {
      width: '1919',
      height: '1138',
    },
  });
  • Introduce the playwright locator: _react, _vue, data-testid attribute. See #4255 by @kobenguyent
Scenario('using playwright locator @Playwright', () => {
  I.amOnPage('https://codecept.io/test-react-calculator/');
  I.click('7');
  I.click({ pw: '_react=t[name = "="]' });
  I.seeElement({ pw: '_react=t[value = "7"]' });
  I.click({ pw: '_react=t[name = "+"]' });
  I.click({ pw: '_react=t[name = "3"]' });
  I.click({ pw: '_react=t[name = "="]' });
  I.seeElement({ pw: '_react=t[value = "10"]' });
});
Scenario('using playwright data-testid attribute @Playwright', () => {
    I.amOnPage('/');
    const webElements = await I.grabWebElements({ pw: '[data-testid="welcome"]' });
    assert.equal(webElements[0]._selector, '[data-testid="welcome"] >> nth=0');
    assert.equal(webElements.length, 1);
});

Network requests & responses can be mocked and modified. Use mockRoute which strictly follows Puppeteer's setRequestInterception API.

I.mockRoute('https://reqres.in/api/comments/1', request => {
  request.respond({
    status: 200,
    headers: { 'Access-Control-Allow-Origin': '*' },
    contentType: 'application/json',
    body: '{"name": "this was mocked" }',
  });
})
I.mockRoute('**/*.{png,jpg,jpeg}', route => route.abort());
// To disable mocking for a route call `stopMockingRoute`
// for previously mocked URL
I.stopMockingRoute('**/*.{png,jpg,jpeg}');

To master request intercepting use HTTPRequest object passed into mock request handler.

🐛 Bug Fixes

New Contributors

Full Changelog: 3.5.15...3.6.0

3.5.15

15 Mar 09:27
96c6bb8
Compare
Choose a tag to compare

3.5.15

❤️ Thanks all to those who contributed to make this release! ❤️

🛩️ Features

  • feat: improve code coverage plugin (#4252) - by KobeNguyenT
    We revamp the coverage plugin to make it easier to use

Once all the tests are completed, codecept will create and store coverage in output/coverage folder, as shown below.

Open index.html in your browser to view the full interactive coverage report.

🐛 Bug Fixes

dry-run command now supports test level grep.

Tests from /Users/t/Desktop/projects/codeceptjs-rest-demo:@jaja

GET tests -- /Users/t/Desktop/projects/codeceptjs-rest-demo/src/GET_test.ts -- 4 tests
  ☐ Verify getting a single user **[jaja](https://github.com/jaja)**
  ☐ Verify getting list of users **[jaja](https://github.com/jaja)**
PUT tests -- /Users/t/Desktop/projects/codeceptjs-rest-demo/src/PUT_test.ts -- 4 tests
  ☐ Verify creating new user **[Jaja](https://github.com/Jaja)**


  Total: 2 suites | 3 tests  

--- DRY MODE: No tests were executed ---
➜  codeceptjs-rest-demo git:(master) ✗ npx codeceptjs dry-run             
Tests from /Users/t/Desktop/projects/codeceptjs-rest-demo:

DELETE tests -- /Users/t/Desktop/projects/codeceptjs-rest-demo/src/DELETE_test.ts -- 4 tests
  ☐ Verify deleting a user
GET tests -- /Users/t/Desktop/projects/codeceptjs-rest-demo/src/GET_test.ts -- 4 tests
  ☐ Verify a successful call
  ☐ Verify a not found call
  ☐ Verify getting a single user **[jaja](https://github.com/jaja)**
  ☐ Verify getting list of users **[jaja](https://github.com/jaja)**
POST tests -- /Users/tDesktop/projects/codeceptjs-rest-demo/src/POST_test.ts -- 4 tests
  ☐ Verify creating new user
  ☐ Verify uploading a file
PUT tests -- /Users/tDesktop/projects/codeceptjs-rest-demo/src/PUT_test.ts -- 4 tests
  ☐ Verify creating new user **[Jaja](https://github.com/Jaja)**


  Total: 4 suites | 8 tests  

--- DRY MODE: No tests were executed ---
  • Several internal fixes and improvements for github workflows

Full Changelog: 3.5.14...3.5.15

3.5.14

19 Feb 13:07
d9623f5
Compare
Choose a tag to compare

What's Changed

Full Changelog: 3.5.13...3.5.14

3.5.13

16 Feb 10:15
1b6b950
Compare
Choose a tag to compare

3.5.13

❤️ Thanks all to those who contributed to make this release! ❤️

🛩️ Features

  • feat: mock server helper (#4155) - by @kobenguyent
    Screenshot 2024-01-25 at 13 47 59
  • feat(webdriver): network traffics manipulation (#4166) - by @kobenguyent
    [Webdriver] Added commands to check network traffics - supported only with devtoolsProtocol
    • startRecordingTraffic
    • grabRecordedNetworkTraffics
    • flushNetworkTraffics
    • stopRecordingTraffic
    • seeTraffic
    • dontSeeTraffic

Examples:

// recording traffics and verify the traffic
  I.startRecordingTraffic();
  I.amOnPage('https://codecept.io/');
  I.seeTraffic({ name: 'traffics', url: 'https://codecept.io/img/companies/BC_LogoScreen_C.jpg' });
// check the traffic with advanced params
  I.amOnPage('https://openai.com/blog/chatgpt');
  I.startRecordingTraffic();
  I.seeTraffic({
    name: 'sentry event',
    url: 'https://images.openai.com/blob/cf717bdb-0c8c-428a-b82b-3c3add87a600',
    parameters: {
      width: '1919',
      height: '1138',
    },
  });
  • feat(webapi): add waitForCookie (#4169) - by @kobenguyent
    Waits for the specified cookie in the cookies.
I.waitForCookie("token");

🐛 Bug Fixes

const limitation = [':nth-of-type', ':first-of-type', ':last-of-type', ':nth-last-child', ':nth-last-of-type', ':checked', ':disabled', ':enabled', ':required', ':lang']; fixes the issue. Then an old conversion way over css-to-xpath is used.

📖 Documentation

🛩️ Several bugfixes and improvements for Codecept-UI

  • Several internal improvements
  • fix: title is not showing when visiting a test
  • fix: handle erros nicely

New Contributors

Full Changelog: 3.5.12...3.5.13

3.5.12

25 Jan 14:06
439364e
Compare
Choose a tag to compare

3.5.12

❤️ Thanks all to those who contributed to make this release! ❤️

🛩️ Features

  • feat: upgrade wdio (#4123) - by @kobenguyent

    🛩️ With the release of WebdriverIO version v8.14.0, and onwards, all driver management hassles are now a thing of the past 🙌. Read more here.
    One of the significant advantages of this update is that you can now get rid of any driver services you previously had to manage, such as
    wdio-chromedriver-service, wdio-geckodriver-service, wdio-edgedriver-service, wdio-safaridriver-service, and even @wdio/selenium-standalone-service.

For those who require custom driver options, fear not; WebDriver Helper allows you to pass in driver options through custom WebDriver configuration.
If you have a custom grid, use a cloud service, or prefer to run your own driver, there's no need to worry since WebDriver Helper will only start a driver when there are no other connection information settings like hostname or port specified.

Example:

{
   helpers: {
     WebDriver : {
       smartWait: 5000,
       browser: "chrome",
       restart: false,
       windowSize: "maximize",
       timeouts: {
         "script": 60000,
         "page load": 10000
       }
     }
   }
}

Testing Chrome locally is now more convenient than ever. You can define a browser channel, and WebDriver Helper will take care of downloading the specified browser version for you.
For example:

{
   helpers: {
     WebDriver : {
       smartWait: 5000,
       browser: "chrome",
       browserVersion: '116.0.5793.0', // or 'stable', 'beta', 'dev' or 'canary'
       restart: false,
       windowSize: "maximize",
       timeouts: {
         "script": 60000,
         "page load": 10000
       }
     }
   }
}

Running with devtools protocol

{
   helpers: {
     WebDriver : {
       url: "http://localhost",
       browser: "chrome",
       devtoolsProtocol: true,
       desiredCapabilities: {
         chromeOptions: {
           args: [ "--headless", "--disable-gpu", "--no-sandbox" ]
         }
       }
     }
   }
}
  • feat: add a locator builder method withTextEquals() (#4100) - by @mirao

Find an element with exact text

locate('button').withTextEquals('Add');

Waits for number of tabs.

I.waitForNumberOfTabs(2);

Currently I.say is not added into the Test.steps array. This PR aims to add this to steps array so that we could use it to print steps in ReportPortal for instance.

Screenshot 2024-01-19 at 15 41 34

🐛 Bug Fixes

Improve the error message for seeElement, dontSeeElement, seeElementInDOM, dontSeeElementInDOM

The current error message doesn't really help when debugging issue also causes some problem described in #4140

Actual

      expected visible elements '[ELEMENT]' to be empty
      + expected - actual

      -[
      -  "ELEMENT"
      -]
      +[]

Updated

     Error: Element "h1" is still visible
      at seeElementError (lib/helper/errors/ElementAssertion.js:9:9)
      at Playwright.dontSeeElement (lib/helper/Playwright.js:1472:7)
  • fix: grabRecordedNetworkTraffics throws error when being called twice (#4143) - by @kobenguyent
  • fix: missing steps of test when running with workers (#4127) - by @kobenguyent
Scenario('Verify getting list of users', async () => {
let res = await I.getUserPerPage(2);
res.data = []; // this line causes the issue
await I.expectEqual(res.data.data[0].id, 7);
});

at this time, res.data.data[0].id would throw undefined error and somehow the test is missing all its steps.

  • fix: process.env.profile when --profile isn't set in run-multiple mode (#4131) - by @mirao

process.env.profile is the string "undefined" instead of type undefined when no --profile is specified in the mode "run-multiple"

Helpers: Playwright
Plugins: screenshotOnFail, tryTo, retryFailedStep, retryTo, eachElement

Repro --
[1]  Starting recording promises
Timeouts:
 [Session] Starting singleton browser session
Reproduce issue
I am on page "https://example.com"
 [Browser:Error] Failed to load resource: the server responded with a status of 404 ()
 [New Context] {}
user1: I am on page "https://example.com"
user1: I execute script () => {
return { width: window.screen.width, height: window.screen.height };
}
sessionScreen is {"width":375,"height":667}
 OK in 1890ms


OK  | 1 passed   // 4s

deprecate some JSON Wire Protocol commands: grabGeoLocation, setGeoLocation

Locator issue due to the lib changes

The locator locate(".ps-menu-button").withText("Authoring").inside(".ps-submenu-root:nth-child(3)") is translated to
3.5.8: //*[contains(concat(' ', normalize-space(./@class), ' '), ' ps-menu-button ')][contains(., 'Authoring')][ancestor::*[(contains(concat(' ', normalize-space(./@class), ' '), ' ps-submenu-root ') and count(preceding-sibling::*) = 2)]] and works well
3.5.11: //*[contains(@class, "ps-menu-button")][contains(., 'Authoring')][ancestor::*[3][contains(@class, "ps-submenu-root")]] and doesn't work (no clickable element found). Even if you test it in browser inspector, it doesn't work.

Full Changelog: 3.5.11...3.5.12

3.5.11

01 Jan 17:24
1a9f3be
Compare
Choose a tag to compare

3.5.11

❤️ Thanks all to those who contributed to make this release! ❤️

🛩️ Features

🐛 Bug Fixes

📖 Documentation

Full Changelog: 3.5.10...3.5.11

3.5.10

07 Dec 13:56
Compare
Choose a tag to compare

❤️ Thanks all to those who contributed to make this release! ❤️

🛩️ Features

Now we expose the WebElements that are returned by the WebHelper and you could make the subsequence actions on them.

// Playwright helper would return the Locator

I.amOnPage('/form/focus_blur_elements');
const webElements = await I.grabWebElements('#button');
webElements[0].click();
Replaying from HAR

 // Replay API requests from HAR.
 // Either use a matching response from the HAR,
 // or abort the request if nothing matches.
   I.replayFromHar('./output/har/something.har', { url: "*/**/api/v1/fruits" });
   I.amOnPage('https://demo.playwright.dev/api-mocking');
   I.see('CodeceptJS');
[Parameters]
harFilePath [string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String) Path to recorded HAR file
opts [object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)? [Options for replaying from HAR](https://playwright.dev/docs/api/class-page#page-route-from-har)
A HAR file is an HTTP Archive file that contains a record of all the network requests that are made when a page is loaded. 
It contains information about the request and response headers, cookies, content, timings, and more. 
You can use HAR files to mock network requests in your tests. HAR will be saved to output/har. 
More info could be found here https://playwright.dev/docs/api/class-browser#browser-new-context-option-record-har.

...
recordHar: {
    mode: 'minimal', // possible values: 'minimal'|'full'.
    content: 'embed' // possible values:  "omit"|"embed"|"attach".
}
...
  • improvement(playwright): support partial string for option (#4016) - by @kobenguyent
await I.amOnPage('/form/select');
await I.selectOption('Select your age', '21-');

🐛 Bug Fixes

Emit the new event: event.workers.result.

CodeceptJS also exposes the env var `process.env.RUNS_WITH_WORKERS` when running tests with run-workers command so that you could handle the events better in your plugins/helpers.

const { event } = require('codeceptjs');

module.exports = function() {
    // this event would trigger the  `_publishResultsToTestrail` when running `run-workers` command
  event.dispatcher.on(event.workers.result, async () => {
    await _publishResultsToTestrail();
  });
  
  // this event would not trigger the  `_publishResultsToTestrail` multiple times when running `run-workers` command
  event.dispatcher.on(event.all.result, async () => {
      // when running `run` command, this env var is undefined
    if (!process.env.RUNS_WITH_WORKERS) await _publishResultsToTestrail();
  });
}
replaced minify library with a modern and more secure fork. Fixes html-minifier@4.0.0 Regular Expression Denial of Service vulnerability #3829
AI class is implemented as singleton
refactored heal.js plugin to work on edge cases
add configuration params on number of fixes performed by ay heal
improved recorder class to add more verbose log
improved recorder class to ignore some of errors
Fixed this error:

locator.isVisible: Unexpected token "s" while parsing selector ":has-text('Were you able to resolve the resident's issue?') >> nth=0"
      at Playwright.waitForText (node_modules\codeceptjs\lib\helper\Playwright.js:2584:79)
Currently inside the _before() of helpers for example Playwright, the retries is set there, however, when retryFailedStep plugin is enabled, the retries of recorder is still using the value from _before() not the value from retryFailedStep plugin.

Fix:

- introduce the process.env.FAILED_STEP_RETIRES which could be access everywhere as the helper won't know anything about the plugin.
- set default retries of Playwright to 3 to be on the same page with Puppeteer.
When test title doesn't have the data in examples:

Feature: Faker examples

  Scenario Outline: Below are the users
    Examples:
      | user   | role |
      | John  | admin |
      | Tim   | client  |

Faker examples --
    [1]  Starting recording promises
    Timeouts: 
  Below are the users {"user":"John","role":"admin"}
  ✔ OK in 4ms

  Below are the users {"user":"Tim","role":"client"}
  ✔ OK in 1ms

When test title includes the data in examples:


Feature: Faker examples

  Scenario Outline: Below are the users - <user> - <role>
    Examples:
      | user   | role |
      | John  | admin |
      | Tim   | client  |


Faker examples --
    [1]  Starting recording promises
    Timeouts: 
  Below are the users - John - admin 
  ✔ OK in 4ms

  Below are the users - Tim - client 
  ✔ OK in 1ms

📖 Documentation

New Contributors

Full Changelog: 3.5.8...3.5.10

3.5.9

07 Dec 13:48
27e8b4c
Compare
Choose a tag to compare

❤️ Thanks all to those who contributed to make this release! ❤️

🛩️ Features

Now we expose the WebElements that are returned by the WebHelper and you could make the subsequence actions on them.

// Playwright helper would return the Locator

I.amOnPage('/form/focus_blur_elements');
const webElements = await I.grabWebElements('#button');
webElements[0].click();
Replaying from HAR

 // Replay API requests from HAR.
 // Either use a matching response from the HAR,
 // or abort the request if nothing matches.
   I.replayFromHar('./output/har/something.har', { url: "*/**/api/v1/fruits" });
   I.amOnPage('https://demo.playwright.dev/api-mocking');
   I.see('CodeceptJS');
[Parameters]
harFilePath [string](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String) Path to recorded HAR file
opts [object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)? [Options for replaying from HAR](https://playwright.dev/docs/api/class-page#page-route-from-har)
A HAR file is an HTTP Archive file that contains a record of all the network requests that are made when a page is loaded. 
It contains information about the request and response headers, cookies, content, timings, and more. 
You can use HAR files to mock network requests in your tests. HAR will be saved to output/har. 
More info could be found here https://playwright.dev/docs/api/class-browser#browser-new-context-option-record-har.

...
recordHar: {
    mode: 'minimal', // possible values: 'minimal'|'full'.
    content: 'embed' // possible values:  "omit"|"embed"|"attach".
}
...
  • improvement(playwright): support partial string for option (#4016) - by @kobenguyent
await I.amOnPage('/form/select');
await I.selectOption('Select your age', '21-');

🐛 Bug Fixes

Emit the new event: event.workers.result.

CodeceptJS also exposes the env var `process.env.RUNS_WITH_WORKERS` when running tests with run-workers command so that you could handle the events better in your plugins/helpers.

const { event } = require('codeceptjs');

module.exports = function() {
    // this event would trigger the  `_publishResultsToTestrail` when running `run-workers` command
  event.dispatcher.on(event.workers.result, async () => {
    await _publishResultsToTestrail();
  });
  
  // this event would not trigger the  `_publishResultsToTestrail` multiple times when running `run-workers` command
  event.dispatcher.on(event.all.result, async () => {
      // when running `run` command, this env var is undefined
    if (!process.env.RUNS_WITH_WORKERS) await _publishResultsToTestrail();
  });
}
replaced minify library with a modern and more secure fork. Fixes html-minifier@4.0.0 Regular Expression Denial of Service vulnerability #3829
AI class is implemented as singleton
refactored heal.js plugin to work on edge cases
add configuration params on number of fixes performed by ay heal
improved recorder class to add more verbose log
improved recorder class to ignore some of errors
Fixed this error:

locator.isVisible: Unexpected token "s" while parsing selector ":has-text('Were you able to resolve the resident's issue?') >> nth=0"
      at Playwright.waitForText (node_modules\codeceptjs\lib\helper\Playwright.js:2584:79)
Currently inside the _before() of helpers for example Playwright, the retries is set there, however, when retryFailedStep plugin is enabled, the retries of recorder is still using the value from _before() not the value from retryFailedStep plugin.

Fix:

- introduce the process.env.FAILED_STEP_RETIRES which could be access everywhere as the helper won't know anything about the plugin.
- set default retries of Playwright to 3 to be on the same page with Puppeteer.
When test title doesn't have the data in examples:

Feature: Faker examples

  Scenario Outline: Below are the users
    Examples:
      | user   | role |
      | John  | admin |
      | Tim   | client  |

Faker examples --
    [1]  Starting recording promises
    Timeouts: 
  Below are the users {"user":"John","role":"admin"}
  ✔ OK in 4ms

  Below are the users {"user":"Tim","role":"client"}
  ✔ OK in 1ms

When test title includes the data in examples:


Feature: Faker examples

  Scenario Outline: Below are the users - <user> - <role>
    Examples:
      | user   | role |
      | John  | admin |
      | Tim   | client  |


Faker examples --
    [1]  Starting recording promises
    Timeouts: 
  Below are the users - John - admin 
  ✔ OK in 4ms

  Below are the users - Tim - client 
  ✔ OK in 1ms

📖 Documentation

New Contributors

Full Changelog: 3.5.8...3.5.9