V9 migrate from got to fetch #11942

tamil777selvan · 2023-12-31T06:11:22Z

Proposed changes

Fixes #11862

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update

Checklist

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)
I have added proper type definitions for new commands (if appropriate)

Further comments

Reviewers: @webdriverio/project-committers

* (internal): drop support for Node.js 16 update engines of package.json files * (internal): drop support for Node.js 16 update CONTRIBUTING.md with Node.js 20 as recommendation * (internal): drop support for Node.js 16 update github workflows * (internal): drop support for Node.js 16 update external packages that dropped the support already * (internal): drop support for Node.js 16 resolve PR feedback * Update packages/webdriver/package.json * Update packages/wdio-cucumber-framework/package.json * Update package.json --------- Co-authored-by: Christian Bromann <git@bromann.dev>

…iverio#11857) * (webdriverio): merge isDisplayed and isDisplayedWithinViewport * allow to specify withinViewport check in waitForDisplayed

* (internal): drop support for Node.js 16 update engines of package.json files * (internal): drop support for Node.js 16 update CONTRIBUTING.md with Node.js 20 as recommendation * (internal): drop support for Node.js 16 update github workflows * (internal): drop support for Node.js 16 update external packages that dropped the support already * (internal): drop support for Node.js 16 resolve PR feedback * Update packages/webdriver/package.json * Update packages/wdio-cucumber-framework/package.json * Update package.json --------- Co-authored-by: Christian Bromann <git@bromann.dev>

…iverio#11857) * (webdriverio): merge isDisplayed and isDisplayedWithinViewport * allow to specify withinViewport check in waitForDisplayed

tamil777selvan · 2024-01-02T03:44:40Z

Progress Update

Tasks associated with this pull request.

Challenges and Blockers

The Nock library currently lacks support for requests made by fetch, as documented in this issue. The ongoing progress on this pull request indicates an expected update to include fetch support in the coming weeks, this currently blocks the smoke tests.
The fetch functionality ignores the passed Host header as per this, leading E2E test failures specifically on Safari. A potential workaround involves adjusting the hostname to localhost (instead of 0.0.0.0) only for the Safari browser.

cc: @christian-bromann

christian-bromann · 2024-01-12T17:08:11Z

A potential workaround involves adjusting the hostname to localhost (instead of 0.0.0.0) only for the Safari browser.

Sounds reasonable to me. @erwinheitzman was working on #11542 to make this happen for all browser but it doesn't seem to be working.

christian-bromann · 2024-02-02T07:15:38Z

@tamil777selvan can we rebase this too?

christian-bromann

Fantastic work 👏 really excited to clean up this mess of multiple request libs. Added some comments to further clean up the code, particularly the request factory is not needed anymore

packages/wdio-browser-runner/src/vite/plugins/testrunner.ts

christian-bromann · 2024-02-04T17:46:35Z

packages/wdio-types/src/Options.ts


 export type Method = 'GET' | 'POST' | 'PUT' | 'PATCH' | 'HEAD' | 'DELETE' | 'OPTIONS' | 'TRACE' | 'get' | 'post' | 'put' | 'patch' | 'head' | 'delete' | 'options' | 'trace'

 export interface RequestLibOptions {
-    agent?: Agents
-    followRedirect?: boolean


Do we need RequestLibOptions at this point? We can just rely on the fetch types since we use it for all requests. I think we had this type to ensure that requests made through got (in Node.js environments) and ky (in browser environments) are compatible. We had this to support https://www.npmjs.com/package/web2driver.

I guess we might need this. After extending RequestInit, below are the additional options which are used internally. Please share your thoughts,

export interface RequestLibOptions extends RequestInit { json?: Record<string, unknown>; searchParams?: Record<string, string | number | boolean | null | undefined> | URLSearchParams; timeout?: number; url?: URL; path?: string; username?: string; password?: string; }

We should update to the RequestInit interface everywhere, e.g.

json: use body: opts.body ?? JSON.stringify(opts.json)

searchParams: use URLSearchParams instead and add it to the url

timeout: use AbortController as mentioned here

url remove as it is passed as second argument

path should not be needed anymore, I think it makes sense to have a helper function that can compose the full url based on connection credential options

username and password: use headers.set('Authorization', 'Basic ' + btoa(username + ":" + password))

What do you think? Can we introduce these further cleanups? I am not sure how many times we pass along RequestLibOptions but it would be idea if this is all using cross platform features.

Certainly, the cleanup sounds logical. I'll get started on it.

As suggested, I've made the cleanup on RequestLibOptions. Regarding moving timeout to AbortController, I have not done because we're using it in the request.ts file where we handle timeout clearance. I'm open to your thoughts on this.

Can we get rid of it completely?

packages/webdriver/src/request/factory.ts

packages/webdriver/src/request/request.ts

erwinheitzman · 2024-02-08T23:18:33Z

This is some amazing work! Well done 👏

packages/wdio-crossbrowsertesting-service/src/service.ts

packages/wdio-browserstack-service/src/util.ts

christian-bromann · 2024-02-13T01:23:22Z

packages/wdio-cucumber-framework/src/index.ts

        headers: {
            Authorization: `Bearer ${token}`,
        }
    })

-    const { location } = headers
+    const location = response.headers.get('location')


Is there ever a chance that location is undefined?

Suggested change

const location = response.headers.get('location')

const location = response.headers.get('location') as string

So far, I've only observed the location as a string and not any other type.

packages/wdio-testingbot-service/src/service.ts

christian-bromann · 2024-02-13T01:25:59Z

packages/wdio-types/src/Options.ts


 export type Method = 'GET' | 'POST' | 'PUT' | 'PATCH' | 'HEAD' | 'DELETE' | 'OPTIONS' | 'TRACE' | 'get' | 'post' | 'put' | 'patch' | 'head' | 'delete' | 'options' | 'trace'

 export interface RequestLibOptions {
-    agent?: Agents
-    followRedirect?: boolean


Can we get rid of it completely?

packages/webdriver/src/request/index.ts

christian-bromann

As @erwinheitzman pointed out, excellent work. Thanks a lot! This is a big milestone for v9!

* V9 drop support for Node.js 16 (#11493) * (internal): drop support for Node.js 16 update engines of package.json files * (internal): drop support for Node.js 16 update CONTRIBUTING.md with Node.js 20 as recommendation * (internal): drop support for Node.js 16 update github workflows * (internal): drop support for Node.js 16 update external packages that dropped the support already * (internal): drop support for Node.js 16 resolve PR feedback * Update packages/webdriver/package.json * Update packages/wdio-cucumber-framework/package.json * Update package.json --------- Co-authored-by: Christian Bromann <git@bromann.dev> * (webdriverio): merge isDisplayed and isDisplayedWithinViewport (#11857) * (webdriverio): merge isDisplayed and isDisplayedWithinViewport * allow to specify withinViewport check in waitForDisplayed * V9 drop support for Node.js 16 (#11493) * (internal): drop support for Node.js 16 update engines of package.json files * (internal): drop support for Node.js 16 update CONTRIBUTING.md with Node.js 20 as recommendation * (internal): drop support for Node.js 16 update github workflows * (internal): drop support for Node.js 16 update external packages that dropped the support already * (internal): drop support for Node.js 16 resolve PR feedback * Update packages/webdriver/package.json * Update packages/wdio-cucumber-framework/package.json * Update package.json --------- Co-authored-by: Christian Bromann <git@bromann.dev> * (webdriverio): merge isDisplayed and isDisplayedWithinViewport (#11857) * (webdriverio): merge isDisplayed and isDisplayedWithinViewport * allow to specify withinViewport check in waitForDisplayed * Migrating from got to fetch * Migrate got to fetch in packages * migrate got to fetch & update unit test * doc update for proxy setup * rebase and migrate got to fetch in browserstack * Migrate hostname from 0.0.0.0 to localhost * fix unit test by removing headers in safari driver * fix smoke test: update nock to beta * fix launch test for firefox * fix unit test for startGeckodriver * fix firefox executions in node 18 * Incorporating review comments * fix component tests * cleaning up RequestLibOptions * Removing RequestLibOptions & incorporating other review comments * use process from globalThis --------- Co-authored-by: Erwin Heitzman <15839059+erwinheitzman@users.noreply.github.com> Co-authored-by: Christian Bromann <git@bromann.dev>

erwinheitzman and others added 10 commits December 11, 2023 22:48

(webdriverio): merge isDisplayed and isDisplayedWithinViewport (webdr…

855e25e

…iverio#11857) * (webdriverio): merge isDisplayed and isDisplayedWithinViewport * allow to specify withinViewport check in waitForDisplayed

(webdriverio): merge isDisplayed and isDisplayedWithinViewport (webdr…

0862c25

…iverio#11857) * (webdriverio): merge isDisplayed and isDisplayedWithinViewport * allow to specify withinViewport check in waitForDisplayed

Migrating from got to fetch

0655168

Merge branch 'main' into v9-fetch

e463012

Merge branch 'webdriverio:v9' into v9

d4ef01f

Rebase with V9 branch

60edbde

Migrate got to fetch in packages

35508b7

migrate got to fetch & update unit test

8ac35fc

doc update for proxy setup

ab97712

christian-bromann added the v9 label Jan 5, 2024

christian-bromann added this to the v9 milestone Jan 16, 2024

christian-bromann mentioned this pull request Jan 20, 2024

(internal): change host from 0.0.0.0 to localhost #12000

Closed

christian-bromann force-pushed the v9 branch 2 times, most recently from 3d5670f to 1d661ee Compare January 28, 2024 18:21

tamil777selvan added 5 commits February 2, 2024 13:59

Rebase and resolve conflicts

1a154af

rebase and migrate got to fetch in browserstack

1418152

Migrate hostname from 0.0.0.0 to localhost

341a8e9

fix unit test by removing headers in safari driver

46a773d

fix smoke test: update nock to beta

c3ff1ef

tamil777selvan marked this pull request as ready for review February 4, 2024 05:33

tamil777selvan added 3 commits February 4, 2024 11:27

fix launch test for firefox

44858d6

fix unit test for startGeckodriver

f475195

fix firefox executions in node 18

203c73f

christian-bromann reviewed Feb 4, 2024

View reviewed changes

Incorporating review comments

4ea5c14

fix component tests

0293308

cleaning up RequestLibOptions

3d1cc5b

christian-bromann reviewed Feb 13, 2024

View reviewed changes

tamil777selvan added 2 commits February 13, 2024 12:46

Removing RequestLibOptions & incorporating other review comments

20e7f3b

use process from globalThis

f661e83

christian-bromann approved these changes Feb 14, 2024

View reviewed changes

christian-bromann merged commit bec68b3 into webdriverio:v9 Feb 14, 2024
6 of 8 checks passed

tamil777selvan deleted the v9-fetch branch February 14, 2024 06:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

V9 migrate from got to fetch #11942

V9 migrate from got to fetch #11942

tamil777selvan commented Dec 31, 2023

tamil777selvan commented Jan 2, 2024 •

edited

christian-bromann commented Jan 12, 2024

christian-bromann commented Feb 2, 2024

christian-bromann left a comment

christian-bromann Feb 4, 2024

tamil777selvan Feb 6, 2024

christian-bromann Feb 8, 2024

tamil777selvan Feb 10, 2024

tamil777selvan Feb 11, 2024

christian-bromann Feb 13, 2024

erwinheitzman commented Feb 8, 2024

christian-bromann Feb 13, 2024

tamil777selvan Feb 13, 2024

christian-bromann Feb 13, 2024

christian-bromann left a comment

	const location = response.headers.get('location')
	const location = response.headers.get('location') as string

V9 migrate from got to fetch #11942

V9 migrate from got to fetch #11942

Conversation

tamil777selvan commented Dec 31, 2023

Proposed changes

Types of changes

Checklist

Further comments

Reviewers: @webdriverio/project-committers

tamil777selvan commented Jan 2, 2024 • edited

Progress Update

Challenges and Blockers

christian-bromann commented Jan 12, 2024

christian-bromann commented Feb 2, 2024

christian-bromann left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

erwinheitzman commented Feb 8, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

christian-bromann left a comment

Choose a reason for hiding this comment

tamil777selvan commented Jan 2, 2024 •

edited