New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding support for mix&match pipelines #1954
base: main
Are you sure you want to change the base?
Conversation
I think this is too complex, would avoid the "naming" altogether. |
One of the advantages that I saw in adopting named pipeline would have been enabling the implementation of one input, multiple outputs, similarly to On the user side, this simplifies the configuration since the pipeline requires to be defined only once. Besides the technical aspects, I also understand that it also comes down to how common such use case is so, I don't feel like pushing this idea too much. |
I feel most devs won't understand that. |
Pity but fair enough. I'll revert the changes related to the named pipelines in favor to a configuration like the following one, looks intuitive enough to me. const pino = require('pino')
const transport = pino.transport({
targets: [{
level: 'info',
target: 'pino-pretty' // must be installed separately
}, {
level: 'trace',
target: 'pino/file',
options: { destination: '/path/to/store/logs' }
}, {
pipeline: [{
target: 'pino-syslog' // must be installed separately
}, {
target: 'pino-socket' // must be installed separately
}]
}
]
})
pino(transport) |
- Implemented support for mixed target&pipeline definitions within `targets` in `transport.js` - Merged logic from both `worker.js` and `worker-pipeline.js` into `worker.js` - Fixed `pipeline.test.js` - Fixed docs to reflect changes above TODO: - Remove `worker-pipeline.js` - Fix `transport.js` to use only `worker.js` - Fix related docs - Fix UTs
- Updated docs to remove mentions of `worker-pipeline.js` - Fixed failing UTs - Fixed `transport.js` to use only `worker.js` also when `pipeline` is defined - Fixed `worker.js` to work properly when only `pipeline` is defined
Removing the Draft mark as the code is now in a more decent shape for some feedback. Besides the implementation of the support for pipelines illustrated above, I also thought to get rid of The following schema illustrates how the streams in
There're probably some naïve decisions here with impacts I don't see, for example pipelines defined via |
I would recommend:
|
Yes, that's something I wanted follow up a bit as I wasn't sure about the implications of logging level vs pipelines in this context. I'll read more about this. Thanks for the heads up. About the recommended changes:
There is actually one layer of
I drew the wrong schema, my bad.
Makes sense, will do. |
This PR wants to address the use-case of mix matching pipelines and targets, see discussion #1302.
Currently only
pipeline
ortargets
can be defined.As a stretch, the current prototype proposes to introduce support for named pipelines that can be referred within the definition of a target.
More details will follow
Update
The idea of named pipelines was abandoned.
This PR extends
targets
to support mixing in pipeline definitions within the array of transport configurationsExample: