Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent the Agent from storing raw SQL queries in memory when "record_sql" is "off" or "obfuscated" #461

Open
kford-newrelic opened this issue Sep 29, 2021 · 5 comments · May be fixed by #559
Labels
enhancement New feature or request

Comments

@kford-newrelic
Copy link
Contributor

kford-newrelic commented Sep 29, 2021

It looks like the Agent will currently store any full, raw SQL query that it detects until the end of the Tracer. At that point it checks the Agent's "record_sql" configuration and either obfuscates the query or drops it entirely. Unfortunately we've seen a few instances where customers have a huge number of large SQL queries being executed which causes the Agent to take up massive amounts of heap space over short periods.

Since the "record_sql" configuration doesn't prevent the raw queries from being stored in memory we don't have any good options of resolving the issue except by disabling the database instrumentation entirely.
Allowing the "record_sql" configuration to immediately affect the data that gets stored in memory, either by obfuscating that data first or dropping it entirely(based on the exact configuration), would give Support a path forward to prevent raw SQL query data from using memory space, if needed.

(Migrate to Jira)

@kford-newrelic kford-newrelic added the enhancement New feature or request label Sep 29, 2021
@kford-newrelic kford-newrelic added this to Triage in Java Engineering Board via automation Sep 29, 2021
@kford-newrelic kford-newrelic moved this from Triage to In progress in Java Engineering Board Sep 29, 2021
@kford-newrelic
Copy link
Contributor Author

Comment added by Kevyn Ford in Aha! - View

Snapshot agent build provided to customer for feedback. Moving this MMF to Nov to compensate for the feedback delay.

@kford-newrelic kford-newrelic linked a pull request Jan 5, 2022 that will close this issue
@kford-newrelic
Copy link
Contributor Author

Current approach in #559 has concerns, including breaking an underlying public API.

Going to have to re-prioritize this for a future round of engineering work.

@kford-newrelic kford-newrelic moved this from In progress to Triage in Java Engineering Board Jan 13, 2022
@kford-newrelic
Copy link
Contributor Author

Comment added by Kevyn Ford in Aha! - View

Could not make the 7.5.0 release due to feature conflict concerns. Will review for a potential future release.

@kford-newrelic
Copy link
Contributor Author

Will likely need another ~2 weeks of engineering time

@kford-newrelic kford-newrelic added Q3 SC and removed Q3 SC labels Sep 8, 2022
@workato-integration
Copy link

@kford-newrelic kford-newrelic removed this from Triage in Java Engineering Board Sep 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants