Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: protobufjs load fix & definitions update script #70

Merged
merged 11 commits into from Feb 13, 2024

Conversation

david-luna
Copy link
Member

Release of protobufjs@7.2.6 now throws an exception if a .proto file imports another and its not found in the filesystem relative to the file doing the import. This makes our mockotlpserver crash since opentelemetry proto files have absolute imports. Ref: protobufjs/protobuf.js#1971

This PR includes

  • patch to fix loading issue of protobufjs
  • proto files synced with the ones in opentelemetry-js repo (current files were a bit younger)
  • types for the services/messages defined in proto files
  • script to update the files and its type definitions

Script usage:

node ./scripts/update-protos.js

The result will be

  • proto files from https://github.com/open-telemetry/opentelementry-proto.git copied to packages/mockotlpserver/opentelemetry at the specified version in the script
  • packages/mockotlpserver/opentelemetry/proto/collector/README.md file updated with a note specifying which version was downloaded.
  • packages/mockotlpserver/lib/types-proto.d.ts is updated with the latest changes

@@ -38,6 +38,9 @@ const statusCodeEnumFromVal = {
* to a value for converting 'attributes' to a simpler object, e.g.:
* { 'telemetry.sdk.version': '1.19.0',
* 'process.pid': 19667 }
*
* @param {import('./types-proto').opentelemetry.proto.common.v1.IAnyValue} v
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note for reviewer: this is a sample of usage of the new types being generated. I've tried to make an alias like

/** @typedef {import('./types-proto').opentelemetry.proto.common} OTelCommon */

but JsDoc seems to not be able to do that with namespaces

const prefix = resolve(__dirname, '..');
const paths = [
'/opentelemetry/proto/common/v1/common.proto',
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note for reviewer: protobufjs tried to load these files since they were in the import graph but it was failing silently. That's why having it here made possible for this to work.

Now that the loading process is fixed it is not necessary anymore

@@ -1,5 +1,3 @@
import type EventEmitter from 'events';
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left over from a previous PR

3. `metrics` package contains the Metrics Service protos.
4. `logs` package contains the Logs Service protos.

### NOTE from Elastic Observability
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note fort reviewer: this file was not included the 1st time we copied the files. Now we're keeping it and adding a note on the version they files belong to

@@ -55,9 +55,6 @@ message ResourceLogs {
// A list of ScopeLogs that originate from a resource.
repeated ScopeLogs scope_logs = 2;

// The Schema URL, if known. This is the identifier of the Schema that the resource data
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note for reviewer: the script did remove changed that were done after the v0.20.0 tag

@david-luna
Copy link
Member Author

we are starting to have spurious tests with the following error

not ok 11 "system.cpu.utilization" data points have a value between 0-1
  ---
    operator: ok
    expected: true
    actual:   false
    at: <anonymous> (/home/runner/work/elastic-otel-node/elastic-otel-node/packages/opentelemetry-node/test/host-metrics.test.js:47:19)
    stack: |-
      Error: "system.cpu.utilization" data points have a value between 0-1
          at Test.assert [as _assert] (/home/runner/work/elastic-otel-node/elastic-otel-node/node_modules/tape/lib/test.js:479:48)
          at Test.assert (/home/runner/work/elastic-otel-node/elastic-otel-node/node_modules/tape/lib/test.js:612:7)
          at /home/runner/work/elastic-otel-node/elastic-otel-node/packages/opentelemetry-node/test/host-metrics.test.js:47:19
          at Array.forEach (<anonymous>)
          at Object.checkTelemetry (/home/runner/work/elastic-otel-node/elastic-otel-node/packages/opentelemetry-node/test/host-metrics.test.js:42:35)
          at done (/home/runner/work/elastic-otel-node/elastic-otel-node/packages/opentelemetry-node/test/testutils.js:504:42)
          at ChildProcess.exithandler (node:child_process:414:7)
          at ChildProcess.emit (node:events:515:28)
          at maybeClose (node:internal/child_process:1105:16)
          at ChildProcess._handle.onexit (node:internal/child_process:[30](https://github.com/elastic/elastic-otel-node/actions/runs/7871378313/job/21474553428?pr=70#step:6:31)5:5)

we need to check if @opentelemetry/host-metrics is reporting a wrong idle` time/utilization

Comment on lines 55 to 56
const tempPath = tmpdir();
const checkoutPath = `${tempPath}/${REPO_NAME}`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Might be preferable to use fs.mkdtemp() for a guaranteed unique temp dir. Then won't need to the if (existsSync(checkoutPath)) { rmSync(checkoutPath, ... below.

@trentm
Copy link
Member

trentm commented Feb 13, 2024

we are starting to have spurious tests with the following error

I opened #73 for this. I've hit it a couple times.

@david-luna david-luna merged commit adb10c1 into main Feb 13, 2024
11 checks passed
@david-luna david-luna deleted the dluna/chore-generate-protos-types branch February 13, 2024 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants