Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚧 Gregor's code review and refactor #2252

Open
wants to merge 93 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 83 commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
267807d
`lib/common.js` separate native node modules from npm dependencies, a…
gr2m Nov 14, 2021
094091a
refactor: comment format
gr2m Nov 14, 2021
c986a22
refactor: improve readability of `index.js` and add type checks
gr2m Nov 15, 2021
94c8fd3
refactor: comment formatting
gr2m Nov 15, 2021
3d7d26d
WIP
gr2m Nov 15, 2021
ed8a352
WIP - make things work without error
gr2m Nov 17, 2021
64b4ddb
WIP
gr2m Nov 17, 2021
568a061
First successful intercept
gr2m Nov 17, 2021
fcae3f6
move response logic into `onIntercept` callback passed to `intercept(…
gr2m Nov 21, 2021
768fbb3
add hello world example
gr2m Nov 21, 2021
e459734
add example for conditional intercept
gr2m Nov 21, 2021
de2ebad
first passing integration test: `should intercept a basic GET request`
gr2m Nov 22, 2021
7b98be9
remove no longer used `InterceptedRequestRouter`
gr2m Nov 22, 2021
95e0cf4
refactor: rename `modules/node-intercept` to `modules/intercept-node-…
gr2m Nov 28, 2021
495b13d
test: remove `.only`
gr2m Nov 28, 2021
8131b72
refactor: rename `request.sendRealRequest` to `request.nockSendRealRe…
gr2m Nov 28, 2021
cd072e4
refactor: rename `proto` to `protocol`
gr2m Nov 28, 2021
4abcba2
fix: make all `test_intercpet.js` tests pass again
gr2m Nov 28, 2021
374db48
test: make existing test more resilient without changing its logic
gr2m Dec 12, 2021
688d961
fix: thow error if nocks `http.ClientRequest()` override is called w…
gr2m Dec 12, 2021
0c14fea
Revert "test: make existing test more resilient without changing its …
gr2m Dec 12, 2021
63764bf
test: call `.end()` to send request as it cannot be intercepted other…
gr2m Dec 12, 2021
b2a6827
fix: emit "error"
gr2m Dec 12, 2021
8707d26
fix: `default` host to `localhost`
gr2m Dec 12, 2021
50499e1
fix: do not emit finish event twice
gr2m Dec 12, 2021
f56b62a
fix: centralize check for valid request options
gr2m Dec 12, 2021
92e1249
refactor: give overwritten `http.{get,request}` methods better names
gr2m Dec 12, 2021
7d0ee09
test: skipping tests checking that the original `http.{get,request}` …
gr2m Dec 12, 2021
78acbb3
fix: emit `"finish"` event after intercept logic run
gr2m Dec 12, 2021
035e451
fix: use correct variable name
gr2m Dec 12, 2021
a0b939f
fix: do not emit "no match" until request finished
gr2m Dec 12, 2021
cab4132
fix: do not log error if an error handler was registered
gr2m Dec 12, 2021
dbb2af6
test: remove usage of fake timers where it is not needed
gr2m Dec 12, 2021
c42a9fb
test: do not call done until request finished
gr2m Dec 12, 2021
c611d18
fix: do not re-trigger `socket` event
gr2m Dec 12, 2021
191204e
test: adapt error message when overwritten `new http.ClientRequest()`…
gr2m Dec 13, 2021
3662d9e
test: adapt test for internal `normalizeRequestOptions()` method. We …
gr2m Dec 13, 2021
a9fd400
test: adapt test for internal `stringifyRequest()` method. We no long…
gr2m Dec 13, 2021
108195c
test: skip remaining failing tests in preparation of #59
gr2m Dec 13, 2021
793ad1f
test: `nock.removeInterceptor()` no longer accepts `proto` option. Ex…
gr2m Dec 13, 2021
a6f13de
fix: import `setImmediate` from `timers`
gr2m Dec 13, 2021
c340aa0
test: `tests/test_fake_timer.js` pass
gr2m Dec 13, 2021
e32b498
Revert "test: remove usage of fake timers where it is not needed"
gr2m Dec 13, 2021
cc5e3dc
fix: match the last duplicate request header
gr2m Dec 13, 2021
3a5590c
test: `tests/test_header_matching.js` pass
gr2m Dec 13, 2021
e1a0f3c
fix: `isActive()`
gr2m Dec 13, 2021
17ac6b1
fix: removes a half-consumed mock
gr2m Dec 13, 2021
51f7332
test: `removes a half-consumed mock`
gr2m Dec 13, 2021
7b87e6f
test: prevents the request from completing
gr2m Dec 13, 2021
8443aea
fix: prevents the request from completing
gr2m Dec 13, 2021
70fc395
test: should be safe to call in the middle of a request
gr2m Dec 13, 2021
0f912fb
refactor: set internal `.intercepted`
gr2m Dec 13, 2021
3ef1407
test: tests/test_socket.js
gr2m Dec 13, 2021
522fa27
test: skip failing tests from tests/test_net_connect.js
gr2m Dec 13, 2021
b3f66e9
test: tests/test_net_connect.js
gr2m Dec 13, 2021
bd4eaab
fix: avoid multiple error events
gr2m Dec 13, 2021
fd926b8
test: tests/test_back.js
gr2m Dec 13, 2021
6c8d591
test: `tests/test_back_filters.js`
gr2m Dec 19, 2021
437b431
fix: recorder & protocol usage
gr2m Dec 19, 2021
b1f68cc
test: tests/test_recorder.js
gr2m Dec 19, 2021
4db4099
fix: recorder
gr2m Dec 19, 2021
fac8952
style: make eslint happy
gr2m Dec 19, 2021
d8e6cca
increase coverage
gr2m Dec 19, 2021
f5f8ede
update intercept usage examples
gr2m Dec 19, 2021
d9b7a85
fix: return new request from `.nockSendRealRequest()`
gr2m Dec 19, 2021
4535a01
record & replay example
gr2m Dec 19, 2021
0abe3bf
fix: when bypassing the intercept and sending the real request instea…
gr2m Dec 19, 2021
86c4a37
rename `.nockGetRequestBodyAsBuffer()` to `.nockGetRequestBodyChunks()`
gr2m Dec 19, 2021
6f3bbd6
recorder: use intercept-node-http module
gr2m Dec 26, 2021
96d0c2a
skip all tests that remain failing in prepraration for https://github…
gr2m Dec 26, 2021
7f77a9c
turn off nock interception (backward compatibility behavior)
gr2m Dec 26, 2021
3a65e34
recorder: checks that data is specified
gr2m Dec 26, 2021
72fc5f3
make sure `response` event handler in `modules/intercept-node-http` i…
gr2m Dec 26, 2021
6366906
test: records https correctly
gr2m Dec 26, 2021
f9367db
recorder: records request headers correctly as an object
gr2m Dec 26, 2021
ea2c986
test: remove `.skip` from recorder tests that are now passing
gr2m Dec 26, 2021
8d4f57a
test: when encoding is set during recording, body is still recorded c…
gr2m Dec 26, 2021
b04e6b6
recorder: do not consume response when recording response body
gr2m Dec 26, 2021
58d8754
test(recorder): all tests passing
gr2m Dec 26, 2021
f0b56fb
test(back): two failing tests remaining
gr2m Dec 26, 2021
9355665
all tests passing \o/
gr2m Dec 26, 2021
9cb849d
remove no longer used `normalizeClientRequestArgs` method
gr2m Dec 26, 2021
1320fee
style: eslint
gr2m Dec 26, 2021
92dab98
test: undo changes to "should not have TLSSocket attributes for HTTP …
gr2m Dec 26, 2021
81cb9e6
refactor: remove no longer needed code to set request host header
gr2m Dec 26, 2021
ae99575
recover support for `nock.removeInterceptor({ proto })
gr2m Dec 26, 2021
0c0df4c
mark "should be safe to call in the middle of a request" test as TODO
gr2m Dec 26, 2021
866e2e0
add more context to skipped test
gr2m Dec 26, 2021
3e3df37
refactor: remove code that is now take care of in the new intercept m…
gr2m Dec 26, 2021
bf08676
actuaully skip TODO test
gr2m Dec 26, 2021
6834394
delete notes
gr2m Dec 26, 2021
64fe949
minor refactorings
gr2m Dec 28, 2021
ec44b74
fix jsdoc tags
gr2m Dec 28, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 0 additions & 1 deletion .eslintrc.yml
Expand Up @@ -28,7 +28,6 @@ rules:
node/no-deprecated-api: 'off' # we still make heavy use of url.parse

# Nock additions.
strict: ['error', 'safe']
no-console: 'error'
no-loop-func: 'error'
no-var: 'error'
Expand Down
2 changes: 2 additions & 0 deletions gregors-code-review-notes
@@ -0,0 +1,2 @@
- when importing `nock`, `back.setMode("dryrun")` is run by default, which runs `nock.activate()`.
- `nock.activate()` disables all net requests by default. `back.setMode("dryrun")` enables unmocked net requests explicitly.
18 changes: 11 additions & 7 deletions index.js
@@ -1,9 +1,9 @@
'use strict'
// @ts-check

const back = require('./lib/back')
const emitter = require('./lib/global_emitter')
const {
activate,
restore,
isActive,
isDone,
isOn,
Expand All @@ -15,12 +15,14 @@ const {
removeAll,
abortPendingRequests,
} = require('./lib/intercept')

const recorder = require('./lib/recorder')
const { Scope, load, loadDefs, define } = require('./lib/scope')
const back = require('./lib/back')

module.exports = (basePath, options) => new Scope(basePath, options)
const { Scope, load, loadDefs, define } = require('./lib/scope')

Object.assign(module.exports, {
const defaultNockFunction = (basePath, options) => new Scope(basePath, options)
const API = {
activate,
isActive,
isDone,
Expand All @@ -40,9 +42,11 @@ Object.assign(module.exports, {
clear: recorder.clear,
play: recorder.outputs,
},
restore: recorder.restore,
restore,
back,
})
}

module.exports = Object.assign(defaultNockFunction, API)

// We always activate Nock on import, overriding the globals.
// Setting the Back mode "activates" Nock by overriding the global entries in the `http/s` modules.
Expand Down
6 changes: 6 additions & 0 deletions lib/back.js
Expand Up @@ -4,6 +4,7 @@ const assert = require('assert')
const recorder = require('./recorder')
const {
activate,
restore,
disableNetConnect,
enableNetConnect,
removeAll: cleanAll,
Expand Down Expand Up @@ -95,6 +96,7 @@ const wild = {
setup: function () {
cleanAll()
recorder.restore()
restore()
activate()
enableNetConnect()
},
Expand All @@ -112,6 +114,7 @@ const dryrun = {
setup: function () {
recorder.restore()
cleanAll()
restore()
activate()
// We have to explicitly enable net connectivity as by default it's off.
enableNetConnect()
Expand All @@ -134,6 +137,7 @@ const record = {
recorder.restore()
recorder.clear()
cleanAll()
restore()
activate()
disableNetConnect()
},
Expand Down Expand Up @@ -180,6 +184,7 @@ const update = {
recorder.restore()
recorder.clear()
cleanAll()
restore()
activate()
disableNetConnect()
},
Expand Down Expand Up @@ -221,6 +226,7 @@ const lockdown = {
recorder.restore()
recorder.clear()
cleanAll()
restore()
activate()
disableNetConnect()
},
Expand Down
173 changes: 16 additions & 157 deletions lib/common.js
@@ -1,19 +1,19 @@
'use strict'
// @ts-check

const util = require('util')

const debug = require('debug')('nock.common')
const set = require('lodash.set')
const timers = require('timers')
const url = require('url')
const util = require('util')

/**
* Normalizes the request options so that it always has `host` property.
*
* @param {Object} options - a parsed options object of the request
*/
function normalizeRequestOptions(options) {
options.proto = options.proto || 'http'
options.port = options.port || (options.proto === 'http' ? 80 : 443)
options.protocol = options.protocol || 'http:'
options.port = options.port || (options.protocol === 'http:' ? 80 : 443)
if (options.host) {
debug('options.host:', options.host)
if (!options.hostname) {
Expand Down Expand Up @@ -51,105 +51,26 @@ function isUtf8Representable(buffer) {
return reconstructedBuffer.equals(buffer)
}

// Array where all information about all the overridden requests are held.
let requestOverrides = {}

/**
* Overrides the current `request` function of `http` and `https` modules with
* our own version which intercepts issues HTTP/HTTPS requests and forwards them
* to the given `newRequest` function.
*
* @param {Function} newRequest - a function handling requests; it accepts four arguments:
* - proto - a string with the overridden module's protocol name (either `http` or `https`)
* - overriddenRequest - the overridden module's request function already bound to module's object
* - options - the options of the issued request
* - callback - the callback of the issued request
*/
function overrideRequests(newRequest) {
debug('overriding requests')
;['http', 'https'].forEach(function (proto) {
debug('- overriding request for', proto)

const moduleName = proto // 1 to 1 match of protocol and module is fortunate :)
const module = {
http: require('http'),
https: require('https'),
}[moduleName]
const overriddenRequest = module.request
const overriddenGet = module.get

if (requestOverrides[moduleName]) {
throw new Error(
`Module's request already overridden for ${moduleName} protocol.`
)
}

// Store the properties of the overridden request so that it can be restored later on.
requestOverrides[moduleName] = {
module,
request: overriddenRequest,
get: overriddenGet,
}
// https://nodejs.org/api/http.html#http_http_request_url_options_callback
module.request = function (input, options, callback) {
return newRequest(proto, overriddenRequest.bind(module), [
input,
options,
callback,
])
}
// https://nodejs.org/api/http.html#http_http_get_options_callback
module.get = function (input, options, callback) {
const req = newRequest(proto, overriddenGet.bind(module), [
input,
options,
callback,
])
req.end()
return req
}

debug('- overridden request for', proto)
})
}

/**
* Restores `request` function of `http` and `https` modules to values they
* held before they were overridden by us.
*/
function restoreOverriddenRequests() {
debug('restoring requests')
Object.entries(requestOverrides).forEach(
([proto, { module, request, get }]) => {
debug('- restoring request for', proto)
module.request = request
module.get = get
debug('- restored request for', proto)
}
)
requestOverrides = {}
}

/**
* In WHATWG URL vernacular, this returns the origin portion of a URL.
* However, the port is not included if it's standard and not already present on the host.
*/
function normalizeOrigin(proto, host, port) {
function normalizeOrigin(protocol, host, port) {
const hostHasPort = host.includes(':')
const portIsStandard =
(proto === 'http' && (port === 80 || port === '80')) ||
(proto === 'https' && (port === 443 || port === '443'))
(protocol === 'http:' && (port === 80 || port === '80')) ||
(protocol === 'https:' && (port === 443 || port === '443'))
const portStr = hostHasPort || portIsStandard ? '' : `:${port}`

return `${proto}://${host}${portStr}`
return `${protocol}//${host}${portStr}`
}

/**
* Get high level information about request as string
* @param {Object} options
* @param {string} options.method
* @param {number|string} options.port
* @param {string} options.proto Set internally. always http or https
* @param {string} options.protocol Set internally. always http: or https:
* @param {string} options.hostname
* @param {string} options.path
* @param {Object} options.headers
Expand All @@ -158,7 +79,7 @@ function normalizeOrigin(proto, host, port) {
*/
function stringifyRequest(options, body) {
const { method = 'GET', path = '', port } = options
const origin = normalizeOrigin(options.proto, options.hostname, port)
const origin = normalizeOrigin(options.protocol, options.hostname, port)

const log = {
method,
Expand Down Expand Up @@ -488,65 +409,6 @@ function isStream(obj) {
)
}

/**
* Converts the arguments from the various signatures of http[s].request into a standard
* options object and an optional callback function.
*
* https://nodejs.org/api/http.html#http_http_request_url_options_callback
*
* Taken from the beginning of the native `ClientRequest`.
* https://github.com/nodejs/node/blob/908292cf1f551c614a733d858528ffb13fb3a524/lib/_http_client.js#L68
*/
function normalizeClientRequestArgs(input, options, cb) {
if (typeof input === 'string') {
input = urlToOptions(new url.URL(input))
} else if (input instanceof url.URL) {
input = urlToOptions(input)
} else {
cb = options
options = input
input = null
}

if (typeof options === 'function') {
cb = options
options = input || {}
} else {
options = Object.assign(input || {}, options)
}

return { options, callback: cb }
}

/**
* Utility function that converts a URL object into an ordinary
* options object as expected by the http.request and https.request APIs.
*
* This was copied from Node's source
* https://github.com/nodejs/node/blob/908292cf1f551c614a733d858528ffb13fb3a524/lib/internal/url.js#L1257
*/
function urlToOptions(url) {
const options = {
protocol: url.protocol,
hostname:
typeof url.hostname === 'string' && url.hostname.startsWith('[')
? url.hostname.slice(1, -1)
: url.hostname,
hash: url.hash,
search: url.search,
pathname: url.pathname,
path: `${url.pathname}${url.search || ''}`,
href: url.href,
}
if (url.port !== '') {
options.port = Number(url.port)
}
if (url.username || url.password) {
options.auth = `${url.username}:${url.password}`
}
return options
}

/**
* Determines if request data matches the expected schema.
*
Expand Down Expand Up @@ -625,11 +487,11 @@ function isPlainObject(value) {
if (Object.getPrototypeOf(value) === null) {
return true
}
let proto = value
while (Object.getPrototypeOf(proto) !== null) {
proto = Object.getPrototypeOf(proto)
let prototype = value
while (Object.getPrototypeOf(prototype) !== null) {
prototype = Object.getPrototypeOf(prototype)
}
return Object.getPrototypeOf(value) === proto
return Object.getPrototypeOf(value) === prototype
}

/**
Expand Down Expand Up @@ -698,7 +560,7 @@ function removeAllTimers() {
* - `request.destroy()`: Added in: v0.3.0
* - `request.destroyed`: Added in: v14.1.0, v13.14.0
*
* @param {ClientRequest} req
* @param {import("http").ClientRequest} req
* @returns {boolean}
*/
function isRequestDestroyed(req) {
Expand Down Expand Up @@ -727,14 +589,11 @@ module.exports = {
isUtf8Representable,
mapValue,
matchStringOrRegexp,
normalizeClientRequestArgs,
normalizeOrigin,
normalizeRequestOptions,
overrideRequests,
percentDecode,
percentEncode,
removeAllTimers,
restoreOverriddenRequests,
setImmediate,
setInterval,
setTimeout,
Expand Down