Description
Long story short
Using a ClientSession where the server sends cookies with an expires
date in the past results in the cookies remaining in the cookie_jar that are then sent in subsequent requests.
Expected behaviour
If a server sends an expired cookie, it should remove that entry from the cookie_jar if it exists.
Actual behaviour
The expired cookie remains in the cookie_jar and is sent in the header of the next response.
Steps to reproduce
Session creation code
def _create_session(self, force=False) -> None:
if not self._session or force:
# define session headers
self._headers = {
'User-Agent': ('Mozilla/5.0 (Windows NT 6.3; Win64; x64) '
'AppleWebKit/537.36 (KHTML, like Gecko) '
'Chrome/68.0.3440.106 Safari/537.36'),
'Accept': ('text/html,application/xhtml+xml, '
'application/xml;q=0.9,*/*;q=0.8'),
'Accept-Language': '*'
}
# initiate session
self._session = aiohttp.ClientSession(headers=self._headers)
Get/post example code
resp = await self._session.get(site)
post_resp = await self._session.post(site,
data=self._data)
Providing debug logs from my use case as the actual code flow is a bit complex. Hopefully the debug logs can explain the issue. I will annotate below.
Get request and headers:
2019-09-16 01:19:25 DEBUG (MainThread) [alexapy.alexalogin] GET:
https://www.amazon.es/ap/signin?showRmrMe=1&openid.return_to=https://alexa.amazon.es/&openid.identity=http://specs.openid.net/auth/2.0/identifier_select&openid.assoc_handle=amzn_dp_project_dee_es&o enid.mode=checkid_setup&openid.claimed_id=http://specs.openid.net/auth/2.0/identifier_select&openid.ns=http://specs.openid.net/auth/2.0& with
<CIMultiDictProxy('Host': 'www.amazon.es', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36', 'Accept': 'text/html,a plication/xhtml+xml, application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': '*', 'Accept-Encoding': 'gzip, deflate')>
Response from server:
Please notice the cookies for the .www.amazon.es
domain that expire in 2007
with value of -
. They will be sent in the next post.
returned 200:OK with response <CIMultiDictProxy('Content-Type': 'text/html;charset=UTF-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Mon, 16 Sep 2019 08:19:24 GMT',
'Set-Cookie': 'session-id=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT',
'Set-Cookie': 'session-id-time=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT',
'Set-Cookie': 'session-token=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT', 'Set-Cookie': 'ubid-acbes=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT',
'Set-Cookie': 'at-acbes=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT',
'Set-Cookie': 'lc-acbes=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT',
'Set-Cookie': 'x-acbes=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT',
'Set-Cookie': 'x-wl-uid=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT',
'Set-Cookie': 'sess-at-acbes=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT', 'Set-Cookie': 'UserPref=-; path=/; domain=.www.amazon.es; expires=Sun, 16-Sep-2007 08:19:24 GMT',
'Strict-Transport-Security': 'max-age=47474747; includeSubDomains; preload', 'X-XSS-Protection': '1', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'SAMEORIGIN', 'x-ua-compatible': 'IE=edge', 'Pragma': 'No-cache', 'Cache-Control': 'max-age=0, no-cache, no-store, must-revalidate', 'Expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'Set-Cookie': 'ap-fid=""; Domain=.amazon.es; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/ap/; Secure', 'Set-Cookie': 'session-id=258-6333802-9738236; Domain=.amazon.es; Expires=Sun, 11-Sep-2039 08:19:24 GMT; Path=/', 'Set-Cookie': 'session-id-time=2199341964l; Domain=.amazon.es; Expires=Sun, 11-Sep-2039 08:19:24 GMT; Path=/', 'Vary': 'Accept-Encoding,X-Amzn-CDN-Cache,X-Amzn-AX-Treatment,User-Agent', 'Content-Encoding': 'gzip', 'x-amz-rid': 'R4N02XVMR834WTEP455G', 'X-Cache': 'Miss from cloudfront', 'Via': '1.1 9d99e2b9dedf1fed654116378ecfcced.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LAX3-C4', 'X-Amz-Cf-Id': 'N0FOhqW GvziQdNCKTLfAc09oRUe6neT-n4KSWsO7HHkttSThCa_5w==')>
Subsequent post within the same session; notice the cookies with value -
sent in the header
2019-09-16 01:28:40 DEBUG (MainThread) [alexapy.alexalogin] POST: redirected from
https://www.amazon.es/ap/signin/262-6418664-6885408
2019-09-16 01:28:40 DEBUG (MainThread) [alexapy.alexalogin] GET:
https://www.amazon.es/ap/mfa?ie=UTF8&arb=8f80c249-16f7-4b95-a8ca-71638e3a34c7&mfa.arb.value=8f80c249-16f7-4b95-a8ca-71638e3a34c7&mfa.arb.key=arb with
<CIMultiDictProxy('Host': 'www.amazon.es', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml, application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': '*', 'Referer': 'https://www.amazon.es/ap/signin?showRmrMe=1&openid.return_to=https://alexa.amazon.es/&openid.identity=http://specs.openid.net/auth/2.0/identifier_select&openid.assoc_handle=amzn_dp_project_dee_es&openid.mode=checkid_setup&openid.claimed_id=http://specs.openid.net/auth/2.0/identifier_select&openid.ns=http://specs.openid.net/auth/2.0&', 'Accept-Encoding': 'gzip, deflate',
'Cookie': 'UserPref=-; at-acbes=-; lc-acbes=-; sess-at-acbes=-; session-id=262-6418664-6885408; session-id-time=2199342519l; sess on-token=-; ubid-acbes=-; x-acbes=-; x-wl-uid=-')>
Your environment
aiohttp 3.5.4
client
Activity
asvetlov commentedon Sep 16, 2019
Thanks for the report.
To go further we need a functional test that reproduces the bug.
Something like the following but with proper cookie headers set:
alandtse commentedon Sep 17, 2019
@asvetlov Understood on the test suite. Actually found a potential fix but it breaks the existing test suite which I think may have an incorrect assumption. Before I do anything further I want to make sure I was aligned on the proper behavior. Please see my comment in the PR.
asvetlov commentedon Sep 28, 2019
Fixed by #4066