Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: using None checking cookies instead of bool(#2875) #2894

Closed
wants to merge 0 commits into from

Conversation

Tunglies
Copy link

@Tunglies Tunglies commented Oct 14, 2023

Summary

Performance improve. Related #2875
Before

         16002280 function calls (14002219 primitive calls) in 10.641 seconds

   Ordered by: internal time
   List reduced from 225 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  2000002    2.431    0.000    2.898    0.000 cookiejar.py:1679(set_cookie)
  1000001    2.243    0.000    6.808    0.000 _models.py:1057(set)
  1000001    1.289    0.000    1.599    0.000 copy.py:66(copy)
  1000001    1.219    0.000    2.914    0.000 cookiejar.py:761(__init__)
3000009/1000003    1.089    0.000    1.089    0.000 cookiejar.py:1227(deepvalues)
        1    0.532    0.532    0.532    0.532 perf_test.py:11(<dictcomp>)
        1    0.386    0.386   10.056   10.056 _models.py:1147(update)
        3    0.381    0.127    7.189    2.396 _models.py:1024(__init__)
  2000003    0.277    0.000    0.277    0.000 {method 'acquire' of '_thread.RLock' objects}
  1000001    0.207    0.000    0.207    0.000 {method 'copy' of 'dict' objects}

Now

         2245 function calls (2190 primitive calls) in 0.539 seconds

   Ordered by: internal time
   List reduced from 221 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.505    0.505    0.505    0.505 perf_test.py:11(<dictcomp>)
        3    0.029    0.010    0.029    0.010 {method 'load_verify_locations' of '_ssl._SSLContext' objects}
       14    0.001    0.000    0.001    0.000 {built-in method __new__ of type object at 0x00007FF9B50B8F90}
        3    0.001    0.000    0.001    0.000 {built-in method nt.stat}
        3    0.000    0.000    0.000    0.000 {method 'set_ciphers' of '_ssl._SSLContext' objects}
        6    0.000    0.000    0.000    0.000 enum.py:1374(_missing_)
        1    0.000    0.000    0.000    0.000 {built-in method winreg.OpenKey}
        5    0.000    0.000    0.000    0.000 _urlparse.py:149(urlparse)
      126    0.000    0.000    0.000    0.000 <frozen os>:674(__getitem__)
        3    0.000    0.000    0.032    0.011 default.py:119(__init__)

Benchmark

import io
import pstats
import cProfile
import httpx

merge_cookies_times = 1_000_000

pr = cProfile.Profile()
pr.enable()

httpx.Client(cookies={"default":"none"})._merge_cookies(cookies={f"{k}":f"{v}" for k,v in zip(range(merge_cookies_times), range(merge_cookies_times))})

pr.disable()
s = io.StringIO()
ps = (pstats.Stats(pr, stream=s)
      .strip_dirs()
      .sort_stats("tottime"))
ps.print_stats(10)
print(s.getvalue())

Github Action Workflow test all passed.

Checklist

  • I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.

httpx/_client.py Outdated Show resolved Hide resolved
httpx/_client.py Outdated
@@ -397,7 +397,7 @@ def _merge_cookies(
Merge a cookies argument together with any cookies on the client,
to create the cookies used for the outgoing request.
"""
if cookies is None or self.cookies is None:
if not (cookies is None or self.cookies is None):
Copy link
Contributor

@zanieb zanieb Oct 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the same either... not (x or y) is not the same is not x or not y

For example, x is true and y is false then not (x or y) is false and not x or not y is true.

Additionally, self.cookies is always not None so this will not work as intended

self._cookies = Cookies(cookies)

httpx/_client.py Outdated
@@ -397,7 +397,7 @@ def _merge_cookies(
Merge a cookies argument together with any cookies on the client,
to create the cookies used for the outgoing request.
"""
if cookies or self.cookies:
if not (cookies is None or self.cookies is None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if not (cookies is None or self.cookies is None):
if not (cookies is None and self.cookies is None):

Then I think we can merge it after changelog added.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we cannot. self.cookies is always not None.

Copy link
Contributor

@zanieb zanieb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://github.com/encode/httpx/pull/2894/files#r1360817089

Marking this to avoid confusion re the review status.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants