Add default user agent header

[ghost]

For #1296

No tests fail that didn't already fail when I ran them locally (which isn't all that hot, I know; not sure what's going on there)

Should I try to add this behaviour to the docs as well, or it okay being implicit?

[ghost]

Well, that travis build clearly isn't good. Let me look into that tomorrow

[ghost]

Hopefully that one does it; lacking access to a non-windows machine at the moment, which is my guess for why I'm seeing tens of inconsistent failures locally.

[codecov-io]

Codecov Report

Merging #1750 into master will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master     #1750   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           23        23           
  Lines         2029      2043   +14     
=========================================
+ Hits          2029      2043   +14     

Flag	Coverage Δ
#unittests	99.55% <100.00%> (+<0.01%)	⬆️

Impacted Files	Coverage Δ
src/urllib3/util/__init__.py	100.00% <ø> (ø)
src/urllib3/connection.py	100.00% <100.00%> (ø)
src/urllib3/util/request.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 444f044...f61b547. Read the comment docs.

[sethmlarson]

Thanks for opening this PR! 🎉 I have a few comments before it can be merged.

[ghost]

Whelp, I'll figure that one out tomorrow.

[ghost]

I think the mysteries of the app engine is a story for another day.
Also, I just now realized the builds only show if my request is open, and then only the most recent one, so I look like I'm talking to myself. That's okay, I guess.

[pquentin]

@DonaCthulhuote We do follow your progress with attention! And are happy to help out if you're stuck.

[ghost]

Oh, just realized what that comment sounded like; shouldn't write things other people can see when I'm tired. I simply thought it was funny that the context for my statements was disappearing.

And not stuck (yet); just discovering that writing code for multiple python versions is a skill in and of itself.

[ghost]

I'm certain the tests will all pass now. Prove me wrong, travis!

[pquentin]

Just a quick note to mention that the remaining failures here are not in your code! So if you think this is ready for review, then it is ready indeed :)

[ghost]

Have at it, if you will.

[pquentin]

@DonaCthulhuote Sorry I'm going to ask one more thing first -- can you please rebase your commit on top of master? We only merge pull requests that pass CI, and the only way to do that here is to rebase.

[ghost]

I'm hoping those should still be green; my local for some reason decided it'd have trouble running tests after rebasing. If travis discovers something, I'll take a deeper look tonight.

[pquentin]

@DonaCthulhuote we need to review this, sorry that we haven't be able to do it yet. You no longer need to rebase your commit on top of master since the tests pass and there are no conflicts!

[hodbn]

I've rebased the work @DonaCthulhuote did on top of master.
Although I think a default user agent is the correct behavior, I think we should let users a way to restore old, no user agent behavior. I've added a new constant named urllib3.util.SUPPRESS_USER_AGENT to allow this.

I still need to figure out where and how to document the new user agent choosing logic (default and suppressing).

Regarding @user234683's comment, I'm also leaning towards removing the version from the user agent. @sethmlarson @pquentin your thoughts?

[ghost]

I'd use None for the suppress headers, if it's being added only to be removed, but I don't have a good justification other than that it feels cleaner in my head.

Thanks for picking this up; I got run over by work last week, and hadn't had a chance to take another look at this.

[hodbn]

@DonaCthulhuote You’re right, it is cleaner and it was my first attempt. I’ve reverted it because it didn’t work with urllib3.make_request.

The same problem will probably rise for higher-level APIs (i.e. requests) that will expose user agent as an optional argument.

What do you think?

[ghost]

I think I clearly missed that case, and that you've clearly put more thought into the problem then I have :) . I'll defer to you and the maintainers on the matter.

[pquentin]

I don't have a strong opinion here. requests sends python-requests/2.23.0, but I'm happy with just urllib3 too.

@hodbn What's the problem with make_request?

[hodbn]

My bad 🙃 not make_request but make_headers:

urllib3/src/urllib3/util/request.py

Lines 22 to 29 in 4901ec7

	def make_headers(
	keep_alive=None,
	accept_encoding=None,
	user_agent=None,
	basic_auth=None,
	proxy_basic_auth=None,
	disable_cache=None,
	):

Callers will not be able to suppress the default behavior. I guess there might be other callers that will have the same problem.
I'm not that happy with the constant I've introduced, do you think a sentinel object will be cleaner?