New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 Check Content-Type request header before assuming JSON #2118
Conversation
Codecov Report
@@ Coverage Diff @@
## master #2118 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 246 246
Lines 7556 7591 +35
=========================================
+ Hits 7556 7591 +35
Continue to review full report at Codecov.
|
📝 Docs preview for commit fa54629 at: https://5f76a9473885317380dee0a8--fastapi.netlify.app |
"msg": "Expecting value: line 1 column 1 (char 0)", | ||
"type": "value_error.jsondecode", | ||
"loc": ["body"], | ||
"msg": "value is not a valid dict", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be considered a breaking change. Think even my work API has a test similar to this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Find the formatting a bit odd but if that's how flake8(?) did it then okay.
📝 Docs preview for commit 49e5945 at: https://5f77437a90b2834b8fa103c0--fastapi.netlify.app |
I think this needs to be thought about a bit more - there are other mimetypes that are JSON but which don't use the application/json mimetype. application/hal+json is one such common example of this |
📝 Docs preview for commit 47a1d9f97c0e187b9ba151a8315549c0bbe2aa93 at: https://5f775f837658b75dc0ecf011--fastapi.netlify.app |
Another thing - you're also breaking content-type headers with charset values, such as |
Perhaps some new tests are called-for at this point. |
📝 Docs preview for commit 332cbd47c607765bf144ca0fa1536810f22f1713 at: https://5f78bf70b71464b14bcbcfc0--fastapi.netlify.app |
How do we like conflicts to be handled? I can rebase and force-push to my feature branch... |
Hi @patrickkwang. Are you planning to merge this? |
Don't try to convert to JSON unless Content-Type == application/json.
Per RFC 7231, the "Content-Type" header is not strictly required, and if it is not present, "application/octet-stream" should be assumed.
📝 Docs preview for commit d752efd at: https://6070fdb40418353f9ff17368--fastapi.netlify.app |
I've rebased and everything is green, but I do not have the necessary permissions to merge. |
Hi @tiangolo, sorry to disturb you, but can you have a look at this? We've been using FastAPI, so far has been an excellent framework, but only because of strict json application type, we are facing troubles in documenting the examples. |
📝 Docs preview for commit 29257d0 at: https://60bdf67acdd67667c44ad652--fastapi.netlify.app |
📝 Docs preview for commit 5729e8a at: https://60bdf86fcdd6766d184ad619--fastapi.netlify.app |
Thanks @patrickkwang ! 🚀 🤓 And thanks for the thorough reviews @ArcLightSlavik and @Mause 🙇 🍰 I updated the tests to include extra corner cases and updated the implementation to parse the content-type headers using Python's standard library |
For what it's worth this appears to be a pretty big breaking change for clients who were previously not sending
I understand the original default was incorrect but this fix breaks functional code and I believe the CVE would be resolved by assuming json in any case that is not exempted from CORS preflight (so don't assume json when the content-type is explicitly set to something not |
This just cost me two days of debugging. Not so pleased right now. I fully see and appreciate the reasoning behind stricter checks, mind you. Go ahead with that! Is it possible to implement an exception that's more to the point here? I have the feeling I'm not the only one tearing their hair out over this... Sorry for the venting -- FastAPI is and remains my favourite framework. Just feeling a bit betrayed by that error message? |
experiencing a similar problem right now (content-type is plain text). Bumping Fast API to 0.65.2 to breaks their integration, and we are not able to update all clients right away. So we either need to hold off on this and future FastAPI updates, or find a server side fix to go around it. |
FastAPI introduced Content-Type checking in v0.65.2 (see tiangolo/fastapi#2118)
That's a very good point @jolynch! Thanks for idea. I just released version This still provides protection from CSRF because browsers automatically set Content-Type headers. And if an attacker removes the header explicitly, it triggers the CORS preflight, which provides the main protection using CORS. So, you should all be able to upgrade to |
🐛 Check Content-Type request header before assuming JSON
Related to #1018.
Edit by @tiangolo: Updated tests to include extra corner cases, and updated implementation to parse content-type headers using Python's standard library
email
module to cover more corner cases.