Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement new style cookies. #1312

Merged
merged 7 commits into from Mar 7, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 14 additions & 0 deletions SECURITY.md
@@ -0,0 +1,14 @@
# Security policy

## Reporting a vulnerability

When you identify a vulnerability in HTTPie, please report it privately using one of the following channels:

- Email to [`security@httpie.io`](mailto:security@httpie.io)
- Report on [huntr.dev](https://huntr.dev/)

In addition to the description of the vulnerability, include the following information:

- A short reproducer to verify it (it can be a small HTTP server, shell script, docker image, etc.)
- Your deemed severity level of the vulnerability (`LOW`/`MEDIUM`/`HIGH`/`CRITICAL`)
- [CWE](https://cwe.mitre.org/) ID, if available.
157 changes: 124 additions & 33 deletions docs/README.md
Expand Up @@ -2157,38 +2157,88 @@ $ http --session-read-only=./ro-session.json pie.dev/headers Custom-Header:orig-
$ http --session-read-only=./ro-session.json pie.dev/headers Custom-Header:new-value
```

### Cookie Storage Behavior
### Host-based cookie policy

**TL;DR:** Cookie storage priority: Server response > Command line request > Session file
Cookies persisted in sessions files have a `domain` field. This _binds_ them to a specified hostname. For example:

To set a cookie within a Session there are three options:
```json
{
"cookies": [
{
"domain": "pie.dev",
"name": "pie",
"value": "apple"
},
{
"domain": "httpbin.org",
"name": "bin",
"value": "http"
}
]
}
```

Using this session file, we include `Cookie: pie=apple` only in requests against `pie.dev` and subdomains (e.g., `foo.pie.dev` or `foo.bar.pie.dev`):

```bash
$ http --session=./session.json pie.dev/cookies
```

```json
{
"cookies": {
"pie": "apple"
}
}
```

To make a cookie domain _unbound_ (i.e., to make it available to all hosts, including throughout a cross-domain redirect chain), you can set the `domain` field to `null` in the session file:

1. Get a `Set-Cookie` header in a response from a server
```json
{
"cookies": [
{
"domain": null,
"name": "unbound-cookie",
"value": "send-me-to-any-host"
}
]
}
```

```bash
$ http --session=./session.json pie.dev/cookie/set?foo=bar
```
```bash
$ http --session=./session.json pie.dev/cookies
```

```json
{
"cookies": {
"unbound-cookie": "send-me-to-any-host"
}
}
```

2. Set the cookie name and value through the command line as seen in [cookies](#cookies)

```bash
$ http --session=./session.json pie.dev/headers Cookie:foo=bar
```
### Cookie storage behavior

3. Manually set cookie parameters in the JSON file of the session
There are three possible sources of persisted cookies within a session. They have the following storage priority: 1—response; 2—command line; 3—session file.

```json
{
"__meta__": {
"about": "HTTPie session file",
"help": "https://httpie.org/doc#sessions",
"httpie": "2.2.0-dev"
},
"auth": {
"password": null,
"type": null,
"username": null
},
1. Receive a response with a `Set-Cookie` header:

```bash
$ http --session=./session.json pie.dev/cookie/set?foo=bar
```

2. Send a cookie specified on the command line as seen in [cookies](#cookies):

```bash
$ http --session=./session.json pie.dev/headers Cookie:foo=bar
```

3. Manually set cookie parameters in the session file:

```json
{
"cookies": {
"foo": {
"expires": null,
Expand All @@ -2197,16 +2247,53 @@ To set a cookie within a Session there are three options:
"value": "bar"
}
}
}
```
}
```

In summary:

Cookies will be set in the session file with the priority specified above.
For example, a cookie set through the command line will overwrite a cookie of the same name stored in the session file.
If the server returns a `Set-Cookie` header with a cookie of the same name, the returned cookie will overwrite the preexisting cookie.
- Cookies set via the CLI overwrite cookies of the same name inside session files.
- Server-sent `Set-Cookie` header cookies overwrite any pre-existing ones with the same name.

Expired cookies are never stored.
If a cookie in a session file expires, it will be removed before sending a new request.
If the server expires an existing cookie, it will also be removed from the session file.
Cookie expiration handling:

- When the server expires an existing cookie, HTTPie removes it from the session file.
- When a cookie in a session file expires, HTTPie removes it before sending a new request.

### Upgrading sessions

HTTPie may introduce changes in the session file format. When HTTPie detects an obsolete format, it shows a warning. You can upgrade your session files using the following commands:

Upgrade all existing [named sessions](#named-sessions) inside the `sessions` subfolder of your [config directory](https://httpie.io/docs/cli/config-file-directory):

```bash
$ httpie cli sessions upgrade-all
Upgraded 'api_auth' @ 'pie.dev' to v3.1.0
Upgraded 'login_cookies' @ 'httpie.io' to v3.1.0
```

Upgrading individual sessions requires you to specify the session's hostname. That allows HTTPie to find the correct file in the case of name sessions. Additionally, it allows it to correctly bind cookies when upgrading with [`--bind-cookies`](#session-upgrade-options).

Upgrade a single [named session](#named-sessions):

```bash
$ httpie cli sessions upgrade pie.dev api_auth
Upgraded 'api_auth' @ 'pie.dev' to v3.1.0
```

Upgrade a single [anonymous session](#anonymous-sessions) using a file path:

```bash
$ httpie cli sessions upgrade pie.dev ./session.json
Upgraded 'session.json' @ 'pie.dev' to v3.1.0
```

#### Session upgrade options

These flags are available for both `sessions upgrade` and `sessions upgrade-all`:

------------------|------------------------------------------
`--bind-cookies` | Bind all previously [unbound cookies](#host-based-cookie-policy) to the session’s host.

## Config

Expand All @@ -2219,7 +2306,7 @@ To see the exact location for your installation, run `http --debug` and look for

The default location of the configuration file on most platforms is `$XDG_CONFIG_HOME/httpie/config.json` (defaulting to `~/.config/httpie/config.json`).

For backwards compatibility, if the directory `~/.httpie` exists, the configuration file there will be used instead.
For backward compatibility, if the directory `~/.httpie` exists, the configuration file there will be used instead.

On Windows, the config file is located at `%APPDATA%\httpie\config.json`.

Expand Down Expand Up @@ -2445,6 +2532,10 @@ Helpers to convert from other client tools:

See [CONTRIBUTING](https://github.com/httpie/httpie/blob/master/CONTRIBUTING.md).

### Security policy

See [github.com/httpie/httpie/security/policy](https://github.com/httpie/httpie/security/policy).

### Change log

See [CHANGELOG](https://github.com/httpie/httpie/blob/master/CHANGELOG.md).
Expand Down
6 changes: 2 additions & 4 deletions httpie/client.py
Expand Up @@ -44,6 +44,7 @@ def collect_messages(
httpie_session_headers = None
if args.session or args.session_read_only:
httpie_session = get_httpie_session(
env=env,
config_dir=env.config.directory,
session_name=args.session or args.session_read_only,
host=args.headers.get('Host'),
Expand Down Expand Up @@ -130,10 +131,7 @@ def collect_messages(
if httpie_session:
if httpie_session.is_new() or not args.session_read_only:
httpie_session.cookies = requests_session.cookies
httpie_session.remove_cookies(
# TODO: take path & domain into account?
cookie['name'] for cookie in expired_cookies
)
httpie_session.remove_cookies(expired_cookies)
httpie_session.save()


Expand Down
60 changes: 40 additions & 20 deletions httpie/config.py
@@ -1,7 +1,7 @@
import json
import os
from pathlib import Path
from typing import Union
from typing import Any, Dict, Union

from . import __version__
from .compat import is_windows
Expand Down Expand Up @@ -62,6 +62,21 @@ class ConfigFileError(Exception):
pass


def read_raw_config(config_type: str, path: Path) -> Dict[str, Any]:
try:
with path.open(encoding=UTF8) as f:
try:
return json.load(f)
except ValueError as e:
raise ConfigFileError(
f'invalid {config_type} file: {e} [{path}]'
)
except FileNotFoundError:
pass
except OSError as e:
raise ConfigFileError(f'cannot read {config_type} file: {e}')


class BaseConfigDict(dict):
name = None
helpurl = None
Expand All @@ -77,26 +92,25 @@ def ensure_directory(self):
def is_new(self) -> bool:
return not self.path.exists()

def pre_process_data(self, data: Dict[str, Any]) -> Dict[str, Any]:
"""Hook for processing the incoming config data."""
return data

def post_process_data(self, data: Dict[str, Any]) -> Dict[str, Any]:
"""Hook for processing the outgoing config data."""
return data

def load(self):
config_type = type(self).__name__.lower()
try:
with self.path.open(encoding=UTF8) as f:
try:
data = json.load(f)
except ValueError as e:
raise ConfigFileError(
f'invalid {config_type} file: {e} [{self.path}]'
)
self.update(data)
except FileNotFoundError:
pass
except OSError as e:
raise ConfigFileError(f'cannot read {config_type} file: {e}')

def save(self):
self['__meta__'] = {
'httpie': __version__
}
data = read_raw_config(config_type, self.path)
if data is not None:
data = self.pre_process_data(data)
self.update(data)

def save(self, *, bump_version: bool = False):
self.setdefault('__meta__', {})
if bump_version or 'httpie' not in self['__meta__']:
self['__meta__']['httpie'] = __version__
if self.helpurl:
self['__meta__']['help'] = self.helpurl

Expand All @@ -106,13 +120,19 @@ def save(self):
self.ensure_directory()

json_string = json.dumps(
obj=self,
obj=self.post_process_data(self),
indent=4,
sort_keys=True,
ensure_ascii=True,
)
self.path.write_text(json_string + '\n', encoding=UTF8)

@property
def version(self):
return self.get(
'__meta__', {}
).get('httpie', __version__)


class Config(BaseConfigDict):
FILENAME = 'config.json'
Expand Down
Empty file added httpie/legacy/__init__.py
Empty file.