Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filter_parser: add error event for multiple parsed results #4478

Merged

Conversation

daipom
Copy link
Contributor

@daipom daipom commented Apr 30, 2024

Which issue(s) this PR fixes:

What this PR does / why we need it:
The following problem of filter_parser remains in #4474 :

However, we have a remaining problem with filter_parser.
filter_parser could return Array based on this wrong behavior.
This PR disables it, so it can not return multiple parsed results anymore.
Even though it was wrong to begin with, it's possible that this change in specifications would be unacceptable.
We need to consider this change carefully.
I explain some examples in detail below.

At least, it would not be happy if some events were lost without any warnings or errors.
So, this PR adds an error handling for that.

Docs Changes:
TODO

Release Note:
Same as the title.

It would be not happy if some events are lost without any
warnings or errors.

Signed-off-by: Daijiro Fukuda <fukuda@clear-code.com>
@daipom
Copy link
Contributor Author

daipom commented Apr 30, 2024

Sample:

<source>
  @type sample
  tag test.array
  sample {"message": "[{\"k\":\"v\"}, {\"k2\":\"v2\"}]"}
</source>

<filter test.**>
  @type parser
  key_name message
  <parse>
    @type json
  </parse>
</filter>

<match test.**>
  @type stdout
</match>
2024-04-30 14:35:58 +0900 [warn]: #0 dump an error event: error_class=Fluent::Plugin::Parser::ParserError error="Could not emit the event. The parser returned multiple results, but currently filter_parser plugin only returns the first parsed result. Raw data: '[{\"k\":\"v\"}, {\"k2\":\"v2\"}]'" location=nil tag="test.array" time=2024-04-30 14:35:58.020965248 +0900 record={"k2"=>"v2"}
2024-04-30 14:35:58.020948041 +0900 test.array: {"k":"v"}

The warning message:

2024-04-30 14:35:58 +0900 [warn]: #0 dump an error event: error_class=Fluent::Plugin::Parser::ParserError
 error="Could not emit the event. The parser returned multiple results,
 but currently filter_parser plugin only returns the first parsed result.
 Raw data: '[{\"k\":\"v\"}, {\"k2\":\"v2\"}]'"
 location=nil tag="test.array" time=2024-04-30 14:35:58.020965248 +0900 record={"k2"=>"v2"}

Copy link
Member

@ashie ashie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ashie ashie merged commit a916014 into master Apr 30, 2024
15 of 16 checks passed
@ashie ashie deleted the filter-parser-add-error-event-for-multiple-parsed-results branch April 30, 2024 07:06
@ashie ashie added this to the v1.17.0 milestone Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants