Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(fetch): re-add support for node v16.8.0+ #1534

Merged
merged 4 commits into from Jul 10, 2022

Conversation

KhafraDev
Copy link
Member

@KhafraDev KhafraDev commented Jul 8, 2022

Fetch support incorrectly claimed it would work on node >= v16.5.0.

  • Data urls would not work until v16.7.0 (requires URL.createObjectURL).
  • Body could be infinitely consumed until v16.8.0 (requires stream.isDisturbed).
  • DOMException has some issues where it would extend a TypeError (or at least its name property would be "TypeError")? No idea when this was fixed.

See #1526 (specifically this comment) for more context.

Also adds node v16.8.0 to the test matrix.

@codecov-commenter
Copy link

codecov-commenter commented Jul 8, 2022

Codecov Report

Merging #1534 (cce41f1) into main (c1a0490) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1534   +/-   ##
=======================================
  Coverage   94.90%   94.90%           
=======================================
  Files          50       50           
  Lines        4608     4609    +1     
=======================================
+ Hits         4373     4374    +1     
  Misses        235      235           
Impacted Files Coverage Δ
lib/fetch/constants.js 100.00% <ø> (ø)
index.js 100.00% <100.00%> (ø)
lib/core/request.js 100.00% <100.00%> (ø)
lib/fetch/body.js 98.51% <100.00%> (-0.02%) ⬇️
lib/fetch/util.js 82.35% <100.00%> (+0.17%) ⬆️
lib/fetch/webidl.js 98.29% <100.00%> (+<0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c1a0490...cce41f1. Read the comment docs.

@KhafraDev
Copy link
Member Author

KhafraDev commented Jul 8, 2022

@ronag - isDisturbed doesn't seem to work on node v16.5 (stream.isDisturbed doesn't yet exist)

function isDisturbed (body) {

edit: minimal repro (this should throw a TypeError)

import { fetch } from './index.js'

const controller = new AbortController()
const res = await fetch('https://www.google.com', {
  signal: controller.signal
})

controller.abort()

await res.text()

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Member

mcollina commented Jul 8, 2022

I don't think we can do without isDisturbed, have you tried lifting the code from core? If not we'd have to support a version that at least has that.

@KhafraDev
Copy link
Member Author

KhafraDev commented Jul 8, 2022

The code is mostly similar to core other than core checks if a symbol is true on the object and does a stream.readableAborted check (which didn't fix the issue). This is a really niche issue afaict, and only happens on versions that don't have stream.isDisturbed.

https://github.com/nodejs/node/blob/5fad0b93667ffc6e4def52996b9529ac99b26319/lib/internal/streams/utils.js#L245-L250

undici/lib/core/util.js

Lines 295 to 304 in c1a0490

function isDisturbed (body) {
return !!(body && (
stream.isDisturbed
? stream.isDisturbed(body) || body[kBodyUsed] // TODO (fix): Why is body[kBodyUsed] needed?
: body[kBodyUsed] ||
body.readableDidRead ||
(body._readableState && body._readableState.dataEmitted) ||
isReadableAborted(body)
))
}

@mcollina
Copy link
Member

mcollina commented Jul 9, 2022

That's why we added it in core. You can't implement fetch() with only the public API from WHATWG streams. That Symbol expose some internal state.

@KhafraDev KhafraDev changed the title fix(fetch): re-add support for node v16.5.0+ fix(fetch): re-add support for node v16.7.0+ Jul 9, 2022
@KhafraDev
Copy link
Member Author

KhafraDev commented Jul 9, 2022

There were a few existing issues with node v16.5 that make fetch largely incompatible.

  1. URL.createObjectURL was added in 16.7. Without it, fetching any date: url would fail.
  2. When a request was aborted, it would throw a TypeError instead of a DOMException (?).
  3. The isDisturbed util didn't work properly on this version.

edit: node-fetch test is still failing. Didn't experience this issue locally...

@KhafraDev KhafraDev closed this Jul 9, 2022
@mcollina
Copy link
Member

mcollina commented Jul 9, 2022

Why did you close this?

@KhafraDev
Copy link
Member Author

Didn't realize lol. Need to bump the min version for fetch to 16.8.

@KhafraDev KhafraDev reopened this Jul 9, 2022
@KhafraDev KhafraDev changed the title fix(fetch): re-add support for node v16.7.0+ fix(fetch): re-add support for node v16.8.0+ Jul 9, 2022
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants