[Question] Pino@7 transports and Webpack

[sdc395]

Hi all

Having chosen to use PNPM to manage the packages within my monorepo, I turned to Webpack 5 to create bundles for each of my Node apps (Webpack follows PNPM's links and creates single-file bundles that include each app's dependencies). Now, I want to add Pino to my apps and send my logs to LogDNA (using pino-logdna) and Sentry (using pino-sentry). I love that Pino@7 supports multiple transports using web workers. Unfortunately, I've no idea how to get Webpack to include Pino's worker.js, or how to persuade Node to then load the script from somewhere.

How must I configure Webpack 5 to include Pino's worker.js?

I've create a minimal repo (here) that I hope to make work with some help.

Thanks
Simon

[jsumners]

If you are able to figure it out, we will welcome a pull request to add documentation (or other). But none of the maintainers of Pino transpile/"pack" their server-side code.

[hmorey3]

I have this same issue with webpack and am eager to see it resolved

[ShogunPanda]

Hello, we just have landed the support for this in #1209 and we also now created the pino-webpack-plugin which should solve your need. Note that you need to update to pino 7.2.0 to have this working.

Can you please let us know if it worked?

[hmorey3]

@ShogunPanda Thanks for the update! I'm running into some issues, but it might be an issue with my setup. Webpack compiles with no errors, but when I try to try to run node dist/main.js I get an error.

Error:

Error: Cannot find module 'C:\projects\<insert-my-path-here>\dist\.        hread-stream-worker.js'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:885:15)
    at Function.Module._load (internal/modules/cjs/loader.js:730:27)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at MessagePort.<anonymous> (internal/main/worker_thread.js:177:24)
    at MessagePort.[nodejs.internal.kHybridDispatch] (internal/event_target.js:398:41)
    at MessagePort.exports.emitMessage (internal/per_context/messageport.js:18:26)
Emitted 'error' event on process instance at:
    at emitUnhandledRejectionOrErr (internal/event_target.js:577:11)
    at MessagePort.[nodejs.internal.kHybridDispatch] (internal/event_target.js:402:9)
    at MessagePort.exports.emitMessage (internal/per_context/messageport.js:18:26) {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

This is my webpack config:

const TerserPlugin = require('terser-webpack-plugin');
const { PinoWebpackPlugin } = require('pino-webpack-plugin')

module.exports = {
  mode: 'production',
  optimization: {
    minimize: true,
    minimizer: [
      new TerserPlugin({
        extractComments: true,
        terserOptions: {
          ecma: 'es2020',
          mangle: false,
          keep_classnames: true,
          keep_fnames: true,
        },
      }),
    ],
  },
  target: 'node',
  entry: './src/main.ts',
  resolve: {
    extensions: ['.ts', '.js', '.json'],
  },
  output: {
    libraryTarget: 'commonjs2',
    filename: '[name].js',
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        exclude: /node_modules/,

        use: [
          {
            loader: 'cache-loader',
            options: {
              cacheDirectory: '.webpackCache',
            },
          },
          {
            loader: 'ts-loader',
          },
        ],
      },
    ],
  },
  plugins: [new PinoWebpackPlugin({ transports: ['pino-pretty'] })],
};

[ShogunPanda]

@hmorey3 I've tried on my system and it works fine. Note that I had to remove cache-loader since it's not supported/neeaded for webpack 5.

Here's my webpack.config.js:

const TerserPlugin = require('terser-webpack-plugin')
const { PinoWebpackPlugin } = require('pino-webpack-plugin')

module.exports = {
  mode: 'production',
  optimization: {
    minimize: true,
    minimizer: [
      new TerserPlugin({
        extractComments: true,
        terserOptions: {
          ecma: 'es2020',
          mangle: false,
          keep_classnames: true,
          keep_fnames: true
        }
      })
    ]
  },
  target: 'node',
  entry: './src/main.ts',
  resolve: {
    extensions: ['.ts', '.js', '.json']
  },
  output: {
    libraryTarget: 'commonjs2',
    filename: '[name].js'
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        exclude: /node_modules/,
        use: [
          {
            loader: 'ts-loader'
          }
        ]
      }
    ]
  },
  plugins: [new PinoWebpackPlugin({ transports: ['pino-pretty'] })]
}

And here a sample src/main.ts I created:

import pino from 'pino'

const logger = pino({ transport: { target: 'pino-pretty' } })
logger.info("We're cool!")

Do you mind sharing your package.json and tsconfig.json?
Also, can you do a ls dist after compilation is done?

[hmorey3]

@ShogunPanda Thanks for the quick reply. I removed cache loader as well but am still getting the same error.

My tsconfig

{
  "compilerOptions": {
    "strict": true,
    "lib": ["es2020"],
    "module": "commonjs",
    "target": "es2020",
    "outDir": "lib",
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "allowJs": true,
    "declaration": true,
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true,
    "strictPropertyInitialization": false
  },
  "include": ["src/**/*.ts"],
  "exclude": ["node_modules", "__mocks__"]
}

My package.json

{
  "name": <omitted>,
  "version": "1.0.0",
  "description": <omitted>,
  "author": {
    "name": <omitted>,
    "email": <omitted>
  },
  "license": <omitted>,
  "scripts": <omitted>,
  "devDependencies": {
    "@types/aws-lambda": "8.10.83",
    "@types/jest": "27.0.1",
    "@types/luxon": "2.0.3",
    "@types/node": "14.14.25",
    "@typescript-eslint/eslint-plugin": "4.29.3",
    "@typescript-eslint/parser": "4.29.3",
    "@vanguard/dexter-cli": "^4.58.0",
    "cross-env": "^7.0.3",
    "dotenv": "^8.2.0",
    "dotenv-safe": "^8.2.0",
    "jest": "^26.6.3",
    "jest-extended": "^0.11.5",
    "jest-html-reporter": "^3.3.0",
    "jest-junit": "^12.0.0",
    "jest-mock": "^26.6.2",
    "mocked-env": "^1.3.5",
    "ts-jest": "^26.5.2",
    "ts-node": "^8.4.1",
    "ts-loader": "^9.2.6",
    "typescript": "4.3.5",
    "webpack": "^5.63.0",
    "webpack-cli": "^4.9.1"
  },
  "dependencies": {
    "@aws-sdk/client-secrets-manager": "^3.36.0",
    "@aws-sdk/client-sns": "^3.38.0",
    "@aws-sdk/client-sqs": "^3.38.0",
    "@aws-sdk/client-sts": "^3.36.0",
    "@aws-sdk/credential-provider-node": "^3.36.0",
    "@aws-sdk/node-http-handler": "^3.38.0",
    "@aws/dynamodb-data-mapper": "^0.7.3",
    "@aws/dynamodb-data-mapper-annotations": "^0.7.3",
    "aws-sdk": "2.987.0",
    "got": "^11.8.2",
    "https-proxy-agent": "^5.0.0",
    "luxon": "2.0.2",
    "nock": "^13.1.4",
    "pino": "^7.2.0",
    "pino-pretty": "^7.2.0",
    "pino-webpack-plugin": "^1.0.0",
    "proxy-agent": "^4.0.1",
    "vg-authentication": "^2.0.0",
    "vg-secrets": "^7.0.1"
  }
}

My pino logger setup

import pino from 'pino'
...
const pinoLogger = pino({
      transport: {
        target: 'pino-pretty'
      }
    });

My dist folder

$ ls dist
main.js              pino-file.js              pino-pipeline-worker.js  pino-pretty.js.LICENSE.txt  pino-worker.js.LICENSE.txt
main.js.LICENSE.txt  pino-file.js.LICENSE.txt  pino-pretty.js           pino-worker.js              thread-stream-worker.js

[ShogunPanda]

So...everything seems to be there.
Your dist/main.js should have a content like this at the beginning:

function pinoWebpackAbsolutePath(p){try{return require("path").join(__dirname,p)}catch(e){return new Function("p","return new URL(p, import.meta.url).pathname")(p)}}globalThis.__bundlerPathsOverrides={"pino/file":pinoWebpackAbsolutePath("./pino-file.js"),"pino-worker":pinoWebpackAbsolutePath("./pino-worker.js"),"pino-pipeline-worker":pinoWebpackAbsolutePath("./pino-pipeline-worker.js"),"pino-pretty":pinoWebpackAbsolutePath("./pino-pretty.js"),"thread-stream-worker":pinoWebpackAbsolutePath("./thread-stream-worker.js")},

(which might differ slightly since you are on Windows)

Can you please copy it here?

[hmorey3]

Interesting, some slashes in my paths seem to be missing or backwards slashes. I re-added them, and made everything forward slashes, and it worked! Here is the original output though:

function pinoWebpackAbsolutePath(e){try{return require("path").join(__dirname,e)}catch(t){return new Function("p","return new URL(p, import.meta.url).pathname")(e)}}globalThis.__bundlerPathsOverrides={"pino/file":pinoWebpackAbsolutePath(".pino-file.js"),"pino-worker":pinoWebpackAbsolutePath(".pino-worker.js"),"pino-pipeline-worker":pinoWebpackAbsolutePath(".pino-pipeline-worker.js"),"pino-pretty":pinoWebpackAbsolutePath(".pino-pretty.js"),"thread-stream-worker":pinoWebpackAbsolutePath(".\thread-stream-worker.js")},

[ShogunPanda]

That's very weird. Can you please execute: console.log(require('path').sep) in a node shell and tell me if it outputs something?

[hmorey3]

@ShogunPanda it outputs a backwards slash

[ShogunPanda]

Which seems correct.
Can you provide a minimal version of your src/main.ts? If you include pino from another file, please provide both, respecting the folder structure.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.