Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider some message scrubbing before parsing JSON payloads #2237

Open
akkomar opened this issue Jan 3, 2023 · 0 comments
Open

Consider some message scrubbing before parsing JSON payloads #2237

akkomar opened this issue Jan 3, 2023 · 0 comments

Comments

@akkomar
Copy link
Contributor

akkomar commented Jan 3, 2023

Currently messages are scrubbed after the payload is parsed. While looking into mozdata.monitoring.payload_bytes_error_structured I noticed JSON parse exceptions for some pings that are ignored.
Since some parts of the scrubbing process do not require message to be parsed (e.g. we have document namespace and type available beforehand), we could split and run them before attempting to parse to avoid doing some unnecessary work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant