Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix patching behavior for require-in-the-middle #4298

Merged
merged 10 commits into from
May 22, 2024

Conversation

duncanpharvey
Copy link
Contributor

@duncanpharvey duncanpharvey commented May 13, 2024

What does this PR do?

Applies a fix made to the require-in-the-middle package so monkey patched modules can be required correctly.

Motivation

https://datadoghq.atlassian.net/browse/SVLS-4805

This is needed to use require-in-the-middle with azure-functions-nodejs-worker, which patches require to support require('@azure/functions-core'). @azure/functions adds these modules that otherwise would not be handled by require. So from dd-trace's perspective these modules don't exist and it errors out when trying to require them.

Screenshot 2024-05-13 at 4 23 17 PM

Plugin Checklist

Additional Notes

Confirmed unit tests fails before fix to ritm.js is added.

  Ritm
    ✓ should shim util
    ✓ should handle module load cycles
    ✓ should fall back to monkey patched module


  3 passing (140.468ms)

Mimic require-in-the-middle by setting _origRequire inside the Hook function. Our version of ritm sets origRequire outside of Hook and prevented the unit test from passing. I tried refactoring all of the references to origRequire but it resulted in unit tests in other packages failing. I took this approach to prevent regressions.

https://github.com/elastic/require-in-the-middle/blob/826dbde2898ef4a3a4ddb0fe6d44287694238b5b/index.js#L124

@duncanpharvey duncanpharvey requested a review from a team as a code owner May 13, 2024 20:24
@duncanpharvey duncanpharvey requested a review from a team May 13, 2024 20:24
Copy link

github-actions bot commented May 13, 2024

Overall package size

Self size: 6.48 MB
Deduped: 60.62 MB
No deduping: 60.9 MB

Dependency sizes

name version self size total size
@datadog/native-iast-taint-tracking 2.1.0 14.91 MB 14.92 MB
@datadog/native-appsec 7.1.1 14.39 MB 14.4 MB
@datadog/pprof 5.3.0 9.85 MB 10.22 MB
protobufjs 7.2.5 2.77 MB 6.56 MB
@datadog/native-iast-rewriter 2.3.1 2.15 MB 2.24 MB
@opentelemetry/core 1.14.0 872.87 kB 1.47 MB
@datadog/native-metrics 2.0.0 898.77 kB 1.3 MB
@opentelemetry/api 1.8.0 1.21 MB 1.21 MB
import-in-the-middle 1.7.4 70.19 kB 739.86 kB
msgpack-lite 0.1.26 201.16 kB 281.59 kB
opentracing 0.14.7 194.81 kB 194.81 kB
semver 7.5.4 93.4 kB 123.8 kB
pprof-format 2.1.0 111.69 kB 111.69 kB
@datadog/sketches-js 2.1.0 109.9 kB 109.9 kB
lodash.sortby 4.7.0 75.76 kB 75.76 kB
lru-cache 7.14.0 74.95 kB 74.95 kB
ipaddr.js 2.1.0 60.23 kB 60.23 kB
ignore 5.2.4 51.22 kB 51.22 kB
int64-buffer 0.1.10 49.18 kB 49.18 kB
shell-quote 1.8.1 44.96 kB 44.96 kB
istanbul-lib-coverage 3.2.0 29.34 kB 29.34 kB
tlhunter-sorted-set 0.1.0 24.94 kB 24.94 kB
limiter 1.1.5 23.17 kB 23.17 kB
dc-polyfill 0.1.4 23.1 kB 23.1 kB
retry 0.13.1 18.85 kB 18.85 kB
node-abort-controller 3.1.1 16.89 kB 16.89 kB
jest-docblock 29.7.0 8.99 kB 12.76 kB
crypto-randomuuid 1.0.0 11.18 kB 11.18 kB
path-to-regexp 0.1.7 6.78 kB 6.78 kB
koalas 1.0.2 6.47 kB 6.47 kB
methods 1.1.2 5.29 kB 5.29 kB
module-details-from-path 1.0.3 4.47 kB 4.47 kB

🤖 This report was automatically generated by heaviest-objects-in-the-universe

Qard
Qard previously approved these changes May 14, 2024
@tlhunter
Copy link
Member

Here's an alternative approach that hard codes module names to bypass RITM: #4305

@rochdev
Copy link
Member

rochdev commented May 15, 2024

Turns out the reason is that they add additional resolving capabilities to require, which makes it possible to require modules that would otherwise not be found by require.resolve, and thus are not found by require-in-the-middle either since it resolves using the built-in functionality and not the patched version. This fix avoids that, but ideally we would add either a more descriptive comment of the actual issue than what is in the require-in-the-middle repo, or even better a test or two.

@pr-commenter
Copy link

pr-commenter bot commented May 15, 2024

Benchmarks

Benchmark execution time: 2024-05-21 21:18:45

Comparing candidate commit 08257a8 in PR branch duncan-harvey/azure-functions-require-in-the-middle-fix with baseline commit 79f0d64 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 263 metrics, 3 unstable metrics.

Copy link

codecov bot commented May 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.31%. Comparing base (7000ae5) to head (300a069).
Report is 28 commits behind head on master.

Current head 300a069 differs from pull request most recent head 08257a8

Please upload reports for the commit 08257a8 to get more accurate results.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4298      +/-   ##
==========================================
- Coverage   69.19%   68.31%   -0.88%     
==========================================
  Files           1       13      +12     
  Lines         198      972     +774     
  Branches       33       33              
==========================================
+ Hits          137      664     +527     
- Misses         61      308     +247     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@duncanpharvey duncanpharvey marked this pull request as draft May 15, 2024 20:40
@duncanpharvey duncanpharvey marked this pull request as ready for review May 16, 2024 12:40
'a failing `require(...)` can still throw as expected'
)

httpHook.unhook()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should go in an afterEach otherwise it won't be executed if the test fails.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added set up and tear down functions to the tests to avoid this issue

packages/dd-trace/src/ritm.js Outdated Show resolved Hide resolved
@astuyve
Copy link
Collaborator

astuyve commented May 22, 2024

LGTM, please test this in Lambda as well as Azure to confirm things still work. (unit tests look good for coverage, but still worth manually checking)

@@ -50,8 +50,20 @@ function Hook (modules, options, onrequire) {

if (patchedRequire) return

const _origRequire = Module.prototype.require
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might be unrelated to this PR, but if we are using this _origRequire, then can we remove the origRequire above and use this one in other places too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @joeyzhao2018! I address that in this comment: #4298 (comment)

@duncanpharvey
Copy link
Contributor Author

/merge

@dd-devflow
Copy link

dd-devflow bot commented May 22, 2024

🚂 MergeQueue

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

Use /merge -c to cancel this operation!

@bengl bengl merged commit 77a1ebc into master May 22, 2024
114 checks passed
@bengl bengl deleted the duncan-harvey/azure-functions-require-in-the-middle-fix branch May 22, 2024 17:34
@dd-devflow
Copy link

dd-devflow bot commented May 22, 2024

🚂 MergeQueue

This pull request was merged directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants