Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unified detailed error logging #2250

Merged

Conversation

alexeevit
Copy link
Contributor

@alexeevit alexeevit commented May 4, 2020

Description

Hey, it's my humble try to make a unified debug logger for the whole project.
The work isn't finished and I'm not sure if I'm doing what expected so I'm going to just put it here for further discussion.
Closes #2235

Your checklist for this pull request

  • I have reviewed the guidelines for contributing to this repository.
  • I have added an entry to History.md if this PR fixes a bug or adds a feature. If it doesn't need an entry to HISTORY.md, I have added [changelog skip] the pull request title.
  • I have added appropriate tests if this PR fixes a bug or adds a feature.
  • My pull request is 100 lines added/removed or less so that it can be easily reviewed.
  • If this PR doesn't need tests (docs change), I added [ci skip] to the title of the PR.
  • If this closes any issues, I have added "Closes #issue" to the PR description or my commit messages.
  • I have updated the documentation accordingly.
  • All new and existing tests passed, including Rubocop.

@acmh
Copy link
Contributor

acmh commented May 4, 2020

Please, can you fix the rubocop offenses to the CI run the tests?

@alexeevit
Copy link
Contributor Author

@nateberkopec hey, I've got a couple of questions:

  • WDYT about the solution?
  • Do I have to put logging to each rescue in the project?
  • What about the logging format? Do we need to support different formatters?
  • Do we have to print headers & body of http requests?

lib/puma/events.rb Outdated Show resolved Hide resolved
lib/puma/events.rb Outdated Show resolved Hide resolved
lib/puma/server.rb Outdated Show resolved Hide resolved
lib/puma/server.rb Outdated Show resolved Hide resolved
lib/puma/server.rb Outdated Show resolved Hide resolved
@nateberkopec
Copy link
Member

Left some comments

Do I have to put logging to each rescue in the project?

Definitely not, because not all errors are worth logging. Also, I feel like there's a higher-level rescue in Puma somewhere where you can catch most of the exceptions that are re-raised. Instead, concentrate on where we are logging errors today, usually to stderr but maybe in other ways, and replacing those with this interface

Do we have to print headers & body of http requests?

I think there should be a global option to dump the rack env. So, if this option is on, we dump the rack env to the output every time the error interface is called.

@nateberkopec nateberkopec added the waiting-for-changes Waiting on changes from the requestor label May 8, 2020
@alexeevit
Copy link
Contributor Author

Well, the CI is weird

@alexeevit
Copy link
Contributor Author

alexeevit commented May 9, 2020

I removed the server argument from logging methods (because it doesn't get invoked anywhere) in the Events class and unified them with the single order of arguments

"#{error.inspect}" \
"\n---\n"
def parse_error(error, env)
@debug_logger.error_dump(error: error, env: env, text: 'HTTP parse error, malformed request', force: true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This is more a comment on the concept and not this specific line)

"debug logger" and the "force: true" reads a bit strange to me... what about using different levels instead?

Just throwing this out as an idea, not demanding anything, this is already great work, thanks for doing it 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you're definitely right. It's strange and needs to be simplified, I'll have a look what I can do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the MR with the new name and the new interface of the logger class.
In my opinion, now it looks better, but there is a couple of things that confuse me:

  • info and debug methods prints headers and body of the request. As for me, this is ok for debug, but I'm not sure about info
  • Since we replaced all $stderr invokes with the new interface, log, write, and debug use $stdout and have the same format, but error uses the new interface and have a different format

@nateberkopec what do you think about it?

@dentarg
Copy link
Member

dentarg commented May 9, 2020

Well, the CI is weird

It's because some whitespace snuck in the gemspec recently (@schneems noticed this and fixed it #2253 but if a maintainer can fix it directly in master I think that would be good).

Don't know about the latest failures here though

@nateberkopec
Copy link
Member

This looks great! I think the approach is now right-on-target. I think I'll wait for you to add the request information, and then I'll add a final round of review.

@nateberkopec nateberkopec added waiting-for-review Waiting on review from anyone and removed waiting-for-changes Waiting on changes from the requestor labels May 15, 2020
Copy link
Member

@nateberkopec nateberkopec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to try this out locally to see what it looks like. One thing I'm concerned about is whether or not we're dumping info about the request when we did not do so before. We've had security issues in the past re: HTTP parse error specifically where logging request info put sensitive user info into the logs. Basically nothing about the request should ever be logged unless the user opted in via PUMA_DEBUG.

require 'puma/const'

module Puma
# The implementation of a logging in debug mode.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment here is out of date now - I think it is uneccessary anyway and can just be deleted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically nothing about the request should ever be logged unless the user opted in via PUMA_DEBUG.

Agree, so I made the interface to have 2 methods: debug and info. Obviously, info prints only non-sensitive info and debug prints only in debug mode.

@nateberkopec
Copy link
Member

BTW I think the design now is really top-notch, exactly what it should be.

@nateberkopec
Copy link
Member

nateberkopec commented Jun 7, 2020

Example output without debug on, opening an ssl connection to a non-SSL configured Puma:

2020-06-07 09:57:49 +0900 HTTP parse error, malformed request (" " - (-)): #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>

@nateberkopec
Copy link
Member

Not a huge fan of how that "malformed request (" " - (-))" looks.

@alexeevit alexeevit changed the title add debug_logger Add unified detailed error logging Jun 7, 2020
@alexeevit
Copy link
Contributor Author

alexeevit commented Jun 7, 2020

Not a huge fan of how that "malformed request (" " - (-))" looks.

Yup, I totally agree, so I fixed it. Now it looks like this:

2020-06-07 16:21:54 +0300 HTTP parse error, malformed request: #<Puma::HttpParserError: Invalid HTTP format, parsing fails. Are you trying to open an SSL connection to a non-SSL Puma?>

@nateberkopec
Copy link
Member

Looks good. I think we have the infrastructure now to improve specific errors/error messages where needed.

This was really impressive work, doing a refactor like this for your first ever contribution to a project is great! I think you can be very proud of what you've done here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature maintenance waiting-for-review Waiting on review from anyone
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Debug logging for 4xx/5xx
4 participants