Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a DLQ for Server to collect failures #309

Open
0xdade opened this issue Jun 26, 2020 · 0 comments
Open

Create a DLQ for Server to collect failures #309

0xdade opened this issue Jun 26, 2020 · 0 comments
Labels
enhancement New feature or request server affecting natlas-server

Comments

@0xdade
Copy link
Member

0xdade commented Jun 26, 2020

Is your feature request related to a problem?
Right now if an agent submits a scan that the server can't parse, the server just throws a 500 and doesn't make any attempt to save the troublesome data. Instead, the agent can optionally (and not default) save the failed scans. Unfortunately having the agent save these scans means that to troubleshoot, the server owner has to control the agent that experienced the failure.

Describe the feature you'd like
The server should have a dead letter queue (DLQ) where scans that fail to parse for one reason or another get dropped for further debugging. Sentry.io captures some data around exceptions if the deployment is configured to use it, but will often trim out a bunch of the larger fields (and rightfully so). By having the server save these failures, the agents don't ever have a need to save them and their logic can be further simplified.

Have you considered alternative ways to get this feature
I don't have specific details of how to implement this. Just that the server should do it instead of the agents independently being responsible for saving failures.

@0xdade 0xdade added enhancement New feature or request server affecting natlas-server labels Jun 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request server affecting natlas-server
Projects
None yet
Development

No branches or pull requests

1 participant