Add validation of HTTP status line, header keys and values

[franekmagiera]

What do these changes do?

HTTP status line, header keys and values are checked in order to prevent custom header injection. If any of those contain a carriage return or newline character a ValueError is raised. The checks are done in _serialize_headers.

Changes where made and tested for pure python version and cython version.

Also imported the _serialize_headers function to test_web_response. I wanted to prevent code duplication and make sure that future changes to the way the headers are created are reflected in the tests. Let me know what you think about it.

The documentation was not updated since I could not find a suitable place to find describe the changes made. Suggestions are welcome.

Are there changes in behavior for the user?

No significant changes, except an exception may be raised.

Related issue number

#4818

Checklist

• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes
• If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • The format is <Name> <Surname>.
  • Please keep alphabetical order, the file is sorted by names.
• Add a new news fragment into the CHANGES folder
  • name it <issue_id>.<type> for example (588.bugfix)
  • if you don't have an issue_id change it to the pr id after creating the pr
  • ensure type is one of the following:
    • .feature: Signifying a new feature.
    • .bugfix: Signifying a bug fix.
    • .doc: Signifying a documentation improvement.
    • .removal: Signifying a deprecation or removal of public API.
    • .misc: A ticket has been closed, but it is not of interest to users.
  • Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

[codecov]

Codecov Report

Merging #5452 (66ec8d5) into master (10a51ae) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #5452   +/-   ##
=======================================
  Coverage   96.75%   96.75%           
=======================================
  Files          44       44           
  Lines        9852     9857    +5     
  Branches     1591     1592    +1     
=======================================
+ Hits         9532     9537    +5     
  Misses        182      182           
  Partials      138      138           

Flag	Coverage Δ
unit	96.65% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
aiohttp/http_writer.py	99.15% <100.00%> (+0.03%)	⬆️
aiohttp/web_protocol.py	86.41% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 10a51ae...66ec8d5. Read the comment docs.

[franekmagiera]

@asvetlov @webknjaz what do you think about this?

[franekmagiera]

@asvetlov @webknjaz would you be interested in merging something like this?

[Dreamsorcerer]

Tests without extensions are stuck. Looks like they get stuck on tests/test_client_functional.py::test_keepalive_two_requests_success[pyloop]

[franekmagiera]

Sorry, missed that, forgot to run the code without extensions. There should be two \r\n signs after the headers.

[Dreamsorcerer]

Sorry, missed that, forgot to run the code without extensions. There should be two \r\n signs after the headers.

Ah, yes, I missed that change.

[Dreamsorcerer]

@webknjaz Unless you've got an idea on a different place to include the check, I think this should be merged.

[Dreamsorcerer]

If there's no response from @webknjaz, I'll merge this at the weekend.

[patchback]

Backport to 3.8: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.8/a1158c5389854f9885c30e08b0020e2d7d8a290f/pr-5452

Backported as #5984

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.