diff --git a/CHANGELOG.md b/CHANGELOG.md index 03871f2a98..d7cd8ba27d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,73 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). +## 1.0.0.beta0 + +The 1.0 pre-release adds an integrated command-line client, and also includes some +design changes. The most notable of these is that redirect responses are no longer +automatically followed, unless specifically requested. + +This design decision prioritises a more explicit approach to redirects, in order +to avoid code that unintentionally issues multiple requests as a result of +misconfigured URLs. + +For example, previously a client configured to send requests to `http://api.github.com/` +would end up sending every API request twice, as each request would be redirected to `https://api.github.com/`. + +If you do want auto-redirect behaviour, you can enable this either by configuring +the client instance with `Client(follow_redirects=True)`, or on a per-request +basis, with `.get(..., follow_redirects=True)`. + +This change is a classic trade-off between convenience and precision, with no "right" +answer. See [discussion #1785](https://github.com/encode/httpx/discussions/1785) for more +context. + +The other major design change is an update to the Transport API, which is the low-level +interface against which requests are sent. Previously this interface used only primitive +datastructures, like so... + +```python +(status_code, headers, stream, extensions) = transport.handle_request(method, url, headers, stream, extensions) +try + ... +finally: + stream.close() +``` + +Now the interface is much simpler... + +```python +response = transport.handle_request(request) +try + ... +finally: + response.close() +``` + +### Changed + +* The `allow_redirects` flag is now `follow_redirects` and defaults to `False`. +* The `raise_for_status()` method will now raise an exception for any responses + except those with 2xx status codes. Previously only 4xx and 5xx status codes + would result in an exception. +* The low-level transport API changes to the much simpler `response = transport.handle_request(request)`. +* The `client.send()` method no longer accepts a `timeout=...` argument, but the + `client.build_request()` does. This required by the signature change of the + Transport API. The request timeout configuration is now stored on the request + instance, as `request.extensions['timeout']`. + +### Added + +* Added the `httpx` command-line client. +* Response instances now include `.is_informational`, `.is_success`, `.is_redirect`, `.is_client_error`, and `.is_server_error` + properties for checking 1xx, 2xx, 3xx, 4xx, and 5xx response types. Note that the behaviour of `.is_redirect` is slightly different in that it now returns True for all 3xx responses, in order to allow for a consistent set of properties onto the different HTTP status code types. The `response.has_redirect_location` location may be used to determine responses with properly formed URL redirects. + +### Fixed + +* `response.iter_bytes()` no longer raises a ValueError when called on a response with no content. (Pull #1827) +* The `'wsgi.error'` configuration now defaults to `sys.stderr`, and is corrected to be a `TextIO` interface, not a `BytesIO` interface. Additionally, the WSGITransport now accepts a `wsgi_error` confguration. (Pull #1828) +* Follow the WSGI spec by properly closing the iterable returned by the application. (Pull #1830) + ## 0.19.0 (19th August, 2021) ### Added diff --git a/README.md b/README.md index 985359d248..7df358518f 100644 --- a/README.md +++ b/README.md @@ -13,15 +13,21 @@

-HTTPX is a fully featured HTTP client for Python 3, which provides sync and async APIs, and support for both HTTP/1.1 and HTTP/2. +HTTPX is a fully featured HTTP client library for Python 3. It includes **an integrated +command line client**, has support for both **HTTP/1.1 and HTTP/2**, and provides both **sync +and async APIs**. -**Note**: _HTTPX should be considered in beta. We believe we've got the public API to -a stable point now, but would strongly recommend pinning your dependencies to the `0.19.*` -release, so that you're able to properly review [API changes between package updates](https://github.com/encode/httpx/blob/master/CHANGELOG.md). A 1.0 release is expected to be issued sometime in 2021._ +**Note**: *This is the README for the 1.0 pre-release. This release adds support for an integrated command-line client, and also includes a couple of design changes from 0.19. Redirects are no longer followed by default, and the low-level Transport API has been updated. Upgrades from 0.19 will need to see [the CHANGELOG](https://github.com/encode/httpx/blob/version-1.0/CHANGELOG.md) for more details.* --- -Let's get started... +Installing HTTPX. + +```shell +$ pip install httpx --pre +``` + +Now, let's get started... ```pycon >>> import httpx @@ -36,26 +42,32 @@ Let's get started... '\n\n\nExample Domain...' ``` -Or, using the async API... - -_Use [IPython](https://ipython.readthedocs.io/en/stable/) or Python 3.8+ with `python -m asyncio` to try this code interactively._ +Or, using the command-line client. -```pycon ->>> import httpx ->>> async with httpx.AsyncClient() as client: -... r = await client.get('https://www.example.org/') -... ->>> r - +```shell +$ pip install --pre 'httpx[cli]' # The command line client is an optional dependency. ``` +Which now allows us to use HTTPX directly from the command-line... + +

+ httpx --help +

+ +Sending a request... + +

+ httpx http://httpbin.org/json +

+ ## Features HTTPX builds on the well-established usability of `requests`, and gives you: * A broadly [requests-compatible API](https://www.python-httpx.org/compatibility/). -* Standard synchronous interface, but with [async support if you need it](https://www.python-httpx.org/async/). +* An integrated command-line client. * HTTP/1.1 [and HTTP/2 support](https://www.python-httpx.org/http2/). +* Standard synchronous interface, but with [async support if you need it](https://www.python-httpx.org/async/). * Ability to make requests directly to [WSGI applications](https://www.python-httpx.org/advanced/#calling-into-python-web-apps) or [ASGI applications](https://www.python-httpx.org/async/#calling-into-python-web-apps). * Strict timeouts everywhere. * Fully type annotated. diff --git a/docs/compatibility.md b/docs/compatibility.md index 7807339499..5ffa32579d 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -1,29 +1,10 @@ # Requests Compatibility Guide -HTTPX aims to be broadly compatible with the `requests` API. +HTTPX aims to be broadly compatible with the `requests` API, although there are a +few design differences in places. This documentation outlines places where the API differs... -## Client instances - -The HTTPX equivalent of `requests.Session` is `httpx.Client`. - -```python -session = requests.Session(**kwargs) -``` - -is generally equivalent to - -```python -client = httpx.Client(**kwargs) -``` - -## Request URLs - -Accessing `response.url` will return a `URL` instance, rather than a string. - -Use `str(response.url)` if you need a string instance. - ## Redirects Unlike `requests`, HTTPX does **not follow redirects by default**. @@ -44,6 +25,26 @@ Or else instantiate a client, with redirect following enabled by default... client = httpx.Client(follow_redirects=True) ``` +## Client instances + +The HTTPX equivalent of `requests.Session` is `httpx.Client`. + +```python +session = requests.Session(**kwargs) +``` + +is generally equivalent to + +```python +client = httpx.Client(**kwargs) +``` + +## Request URLs + +Accessing `response.url` will return a `URL` instance, rather than a string. + +Use `str(response.url)` if you need a string instance. + ## Determining the next redirect request The `requests` library exposes an attribute `response.next`, which can be used to obtain the next redirect request. @@ -97,8 +98,7 @@ opened in text mode. ## Content encoding HTTPX uses `utf-8` for encoding `str` request bodies. For example, when using `content=` the request body will be encoded to `utf-8` before being sent over the wire. This differs from Requests which uses `latin1`. If you need an explicit encoding, pass encoded bytes explictly, e.g. `content=.encode("latin1")`. - -For response bodies, assuming the server didn't send an explicit encoding then HTTPX will do its best to figure out an appropriate encoding. HTTPX makes a guess at the encoding to use for decoding the response using `charset_normalizer`. Fallback to that or any content with less than 32 octets will be decoded using `utf-8` with the `error="replace"` decoder strategy. +For response bodies, assuming the server didn't send an explicit encoding then HTTPX will do its best to figure out an appropriate encoding. HTTPX makes a guess at the encoding to use for decoding the response using `charset_normalizer`. Fallback to that or any content with less than 32 octets will be decoded using `utf-8` with the `error="replace"` decoder strategy. ## Cookies @@ -133,7 +133,7 @@ HTTPX provides a `.stream()` interface rather than using `stream=True`. This ens For example: ```python -with request.stream("GET", "https://www.example.com") as response: +with httpx.stream("GET", "https://www.example.com") as response: ... ``` @@ -165,13 +165,21 @@ Requests supports `REQUESTS_CA_BUNDLE` which points to either a file or a direct ## Request body on HTTP methods -The HTTP `GET`, `DELETE`, `HEAD`, and `OPTIONS` methods are specified as not supporting a request body. To stay in line with this, the `.get`, `.delete`, `.head` and `.options` functions do not support `files`, `data`, or `json` arguments. +The HTTP `GET`, `DELETE`, `HEAD`, and `OPTIONS` methods are specified as not supporting a request body. To stay in line with this, the `.get`, `.delete`, `.head` and `.options` functions do not support `content`, `files`, `data`, or `json` arguments. If you really do need to send request data using these http methods you should use the generic `.request` function instead. -## Checking for 4xx/5xx responses +```python +httpx.request( + method="DELETE", + url="https://www.example.com/", + content=b'A request body on a DELETE request.' +) +``` + +## Checking for success and failure responses -We don't support `response.is_ok` since the naming is ambiguous there, and might incorrectly imply an equivalence to `response.status_code == codes.OK`. Instead we provide the `response.is_error` property. Use `if not response.is_error:` instead of `if response.is_ok:`. +We don't support `response.is_ok` since the naming is ambiguous there, and might incorrectly imply an equivalence to `response.status_code == codes.OK`. Instead we provide the `response.is_success` property, which can be used to check for a 2xx response. ## Request instantiation diff --git a/docs/img/httpx-help.png b/docs/img/httpx-help.png new file mode 100644 index 0000000000..32b4ad9d90 Binary files /dev/null and b/docs/img/httpx-help.png differ diff --git a/docs/img/httpx-request.png b/docs/img/httpx-request.png new file mode 100644 index 0000000000..2057d010af Binary files /dev/null and b/docs/img/httpx-request.png differ diff --git a/docs/index.md b/docs/index.md index e7f2504374..b302379a3b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -25,15 +25,19 @@ HTTPX is a fully featured HTTP client for Python 3, which provides sync and asyn !!! note - HTTPX should currently be considered in beta. + This is the documentation for the 1.0 pre-release. - We believe we've got the public API to a stable point now, but would strongly recommend pinning your dependencies to the `0.19.*` release, so that you're able to properly review [API changes between package updates](https://github.com/encode/httpx/blob/master/CHANGELOG.md). - - A 1.0 release is expected to be issued sometime in 2021. + This release adds support for an integrated command-line client, and also includes a couple of design changes from 0.19. Redirects are no longer followed by default, and the low-level Transport API has been updated. See [the CHANGELOG](https://github.com/encode/httpx/blob/version-1.0/CHANGELOG.md) for more details. --- -Let's get started... +Installing the HTTPX 1.0 pre-release. + +```shell +$ pip install httpx --pre +``` + +Now, let's get started... ```pycon >>> import httpx @@ -48,23 +52,24 @@ Let's get started... '\n\n\nExample Domain...' ``` -Or, using the async API... - -_Use [IPython](https://ipython.readthedocs.io/en/stable/) or Python 3.8+ with `python -m asyncio` to try this code interactively._ +Or, using the command-line client. -```pycon ->>> import httpx ->>> async with httpx.AsyncClient() as client: -... r = await client.get('https://www.example.org/') -... ->>> r - +```shell +# The command line client is an optional dependency. +$ pip install --pre 'httpx[cli]' ``` +Which now allows us to use HTTPX directly from the command-line... + +![httpx --help](img/httpx-help.png) + +Sending a request... + +![httpx http://httpbin.org/json](img/httpx-request.png) + ## Features -HTTPX is a high performance asynchronous HTTP client, that builds on the -well-established usability of `requests`, and gives you: +HTTPX builds on the well-established usability of `requests`, and gives you: * A broadly [requests-compatible API](compatibility.md). * Standard synchronous interface, but with [async support if you need it](async.md). diff --git a/docs/quickstart.md b/docs/quickstart.md index 23e1765246..e8923f02d7 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -73,9 +73,7 @@ You can inspect what encoding will be used to decode the response. ``` In some cases the response may not contain an explicit encoding, in which case HTTPX -will attempt to automatically determine an encoding to use. This defaults to -UTF-8, but also includes robust fallback behaviour for handling ascii, -iso-8859-1 and windows 1252 encodings. +will attempt to automatically determine an encoding to use. ```pycon >>> r.encoding @@ -84,7 +82,6 @@ None '\n\n\nExample Domain...' ``` - If you need to override the standard behaviour and explicitly set the encoding to use, then you can do that too. @@ -277,7 +274,7 @@ HTTPX also includes an easy shortcut for accessing status codes by their text ph True ``` -We can raise an exception for any Client or Server error responses (4xx or 5xx status codes): +We can raise an exception for any responses which are not a 2xx success code: ```pycon >>> not_found = httpx.get('https://httpbin.org/status/404') diff --git a/httpx/__init__.py b/httpx/__init__.py index bfce57639f..b6303deb3f 100644 --- a/httpx/__init__.py +++ b/httpx/__init__.py @@ -43,6 +43,21 @@ from ._transports.wsgi import WSGITransport from ._types import AsyncByteStream, SyncByteStream +try: + from ._main import main +except ImportError: # pragma: nocover + + def main() -> None: # type: ignore + import sys + + print( + "The httpx command line client could not run because the required " + "dependencies were not installed.\nMake sure you've installed " + "everything with: pip install 'httpx[cli]'" + ) + sys.exit(1) + + __all__ = [ "__description__", "__title__", @@ -76,6 +91,7 @@ "InvalidURL", "Limits", "LocalProtocolError", + "main", "MockTransport", "NetworkError", "options", diff --git a/httpx/__version__.py b/httpx/__version__.py index bab8a1c052..27b0a99f47 100644 --- a/httpx/__version__.py +++ b/httpx/__version__.py @@ -1,3 +1,3 @@ __title__ = "httpx" __description__ = "A next generation HTTP client, for Python 3." -__version__ = "0.19.0" +__version__ = "1.0.0.beta0" diff --git a/httpx/_main.py b/httpx/_main.py new file mode 100644 index 0000000000..20834777e4 --- /dev/null +++ b/httpx/_main.py @@ -0,0 +1,438 @@ +import json +import sys +import typing + +import click +import pygments.lexers +import pygments.util +import rich.console +import rich.progress +import rich.syntax + +from ._client import Client +from ._exceptions import RequestError +from ._models import Request, Response + + +def print_help() -> None: + console = rich.console.Console() + + console.print("[bold]HTTPX :butterfly:", justify="center") + console.print() + console.print("A next generation HTTP client.", justify="center") + console.print() + console.print( + "Usage: [bold]httpx[/bold] [cyan] [OPTIONS][/cyan] ", justify="left" + ) + console.print() + + table = rich.table.Table.grid(padding=1, pad_edge=True) + table.add_column("Parameter", no_wrap=True, justify="left", style="bold") + table.add_column("Description") + table.add_row( + "-m, --method [cyan]METHOD", + "Request method, such as GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD.\n" + "[Default: GET, or POST if a request body is included]", + ) + table.add_row( + "-p, --params [cyan] ...", + "Query parameters to include in the request URL.", + ) + table.add_row( + "-c, --content [cyan]TEXT", "Byte content to include in the request body." + ) + table.add_row( + "-d, --data [cyan] ...", "Form data to include in the request body." + ) + table.add_row( + "-f, --files [cyan] ...", + "Form files to include in the request body.", + ) + table.add_row("-j, --json [cyan]TEXT", "JSON data to include in the request body.") + table.add_row( + "-h, --headers [cyan] ...", + "Include additional HTTP headers in the request.", + ) + table.add_row( + "--cookies [cyan] ...", "Cookies to include in the request." + ) + table.add_row( + "--auth [cyan]", + "Username and password to include in the request. Specify '-' for the password to use " + "a password prompt. Note that using --verbose/-v will expose the Authorization " + "header, including the password encoding in a trivially reverisible format.", + ) + + table.add_row( + "--proxy [cyan]URL", + "Send the request via a proxy. Should be the URL giving the proxy address.", + ) + + table.add_row( + "--timeout [cyan]FLOAT", + "Timeout value to use for network operations, such as establishing the connection, " + "reading some data, etc... [Default: 5.0]", + ) + + table.add_row("--follow-redirects", "Automatically follow redirects.") + table.add_row("--no-verify", "Disable SSL verification.") + table.add_row( + "--http2", "Send the request using HTTP/2, if the remote server supports it." + ) + + table.add_row( + "--download [cyan]FILE", + "Save the response content as a file, rather than displaying it.", + ) + + table.add_row("-v, --verbose", "Verbose output. Show request as well as response.") + table.add_row("--help", "Show this message and exit.") + console.print(table) + + +def get_lexer_for_response(response: Response) -> str: + content_type = response.headers.get("Content-Type") + if content_type is not None: + mime_type, _, _ = content_type.partition(";") + try: + return pygments.lexers.get_lexer_for_mimetype(mime_type.strip()).name + except pygments.util.ClassNotFound: # pragma: nocover + pass + return "" # pragma: nocover + + +def format_request_headers(request: Request) -> str: + target = request.url.raw[-1].decode("ascii") + lines = [f"{request.method} {target} HTTP/1.1"] + [ + f"{name.decode('ascii')}: {value.decode('ascii')}" + for name, value in request.headers.raw + ] + return "\n".join(lines) + + +def format_response_headers(response: Response) -> str: + lines = [ + f"{response.http_version} {response.status_code} {response.reason_phrase}" + ] + [ + f"{name.decode('ascii')}: {value.decode('ascii')}" + for name, value in response.headers.raw + ] + return "\n".join(lines) + + +def print_request_headers(request: Request) -> None: + console = rich.console.Console() + http_text = format_request_headers(request) + syntax = rich.syntax.Syntax(http_text, "http", theme="ansi_dark", word_wrap=True) + console.print(syntax) + syntax = rich.syntax.Syntax("", "http", theme="ansi_dark", word_wrap=True) + console.print(syntax) + + +def print_response_headers(response: Response) -> None: + console = rich.console.Console() + http_text = format_response_headers(response) + syntax = rich.syntax.Syntax(http_text, "http", theme="ansi_dark", word_wrap=True) + console.print(syntax) + + +def print_delimiter() -> None: + console = rich.console.Console() + syntax = rich.syntax.Syntax("", "http", theme="ansi_dark", word_wrap=True) + console.print(syntax) + + +def print_redirects(response: Response) -> None: + if response.has_redirect_location: + response.read() + print_response_headers(response) + print_response(response) + + +def print_response(response: Response) -> None: + console = rich.console.Console() + lexer_name = get_lexer_for_response(response) + if lexer_name: + if lexer_name.lower() == "json": + try: + data = response.json() + text = json.dumps(data, indent=4) + except ValueError: # pragma: nocover + text = response.text + else: + text = response.text + syntax = rich.syntax.Syntax(text, lexer_name, theme="ansi_dark", word_wrap=True) + console.print(syntax) + else: # pragma: nocover + console.print(response.text) + + +def download_response(response: Response, download: typing.BinaryIO) -> None: + console = rich.console.Console() + syntax = rich.syntax.Syntax("", "http", theme="ansi_dark", word_wrap=True) + console.print(syntax) + + content_length = response.headers.get("Content-Length") + kwargs = {"total": int(content_length)} if content_length else {} + with rich.progress.Progress( + "[progress.description]{task.description}", + "[progress.percentage]{task.percentage:>3.0f}%", + rich.progress.BarColumn(bar_width=None), + rich.progress.DownloadColumn(), + rich.progress.TransferSpeedColumn(), + ) as progress: + description = f"Downloading [bold]{download.name}" + download_task = progress.add_task(description, **kwargs) # type: ignore + for chunk in response.iter_bytes(): + download.write(chunk) + progress.update(download_task, completed=response.num_bytes_downloaded) + + +def validate_json( + ctx: click.Context, + param: typing.Union[click.Option, click.Parameter], + value: typing.Any, +) -> typing.Any: + if value is None: + return None + + try: + return json.loads(value) + except json.JSONDecodeError: # pragma: nocover + raise click.BadParameter("Not valid JSON") + + +def validate_auth( + ctx: click.Context, + param: typing.Union[click.Option, click.Parameter], + value: typing.Any, +) -> typing.Any: + if value == (None, None): + return None + + username, password = value + if password == "-": # pragma: nocover + password = click.prompt("Password", hide_input=True) + return (username, password) + + +def handle_help( + ctx: click.Context, + param: typing.Union[click.Option, click.Parameter], + value: typing.Any, +) -> None: + if not value or ctx.resilient_parsing: + return + + print_help() + ctx.exit() + + +@click.command(add_help_option=False) +@click.argument("url", type=str) +@click.option( + "--method", + "-m", + "method", + type=str, + help=( + "Request method, such as GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD. " + "[Default: GET, or POST if a request body is included]" + ), +) +@click.option( + "--params", + "-p", + "params", + type=(str, str), + multiple=True, + help="Query parameters to include in the request URL.", +) +@click.option( + "--content", + "-c", + "content", + type=str, + help="Byte content to include in the request body.", +) +@click.option( + "--data", + "-d", + "data", + type=(str, str), + multiple=True, + help="Form data to include in the request body.", +) +@click.option( + "--files", + "-f", + "files", + type=(str, click.File(mode="rb")), + multiple=True, + help="Form files to include in the request body.", +) +@click.option( + "--json", + "-j", + "json", + type=str, + callback=validate_json, + help="JSON data to include in the request body.", +) +@click.option( + "--headers", + "-h", + "headers", + type=(str, str), + multiple=True, + help="Include additional HTTP headers in the request.", +) +@click.option( + "--cookies", + "cookies", + type=(str, str), + multiple=True, + help="Cookies to include in the request.", +) +@click.option( + "--auth", + "auth", + type=(str, str), + default=(None, None), + callback=validate_auth, + help=( + "Username and password to include in the request. " + "Specify '-' for the password to use a password prompt. " + "Note that using --verbose/-v will expose the Authorization header, " + "including the password encoding in a trivially reverisible format." + ), +) +@click.option( + "--proxies", + "proxies", + type=str, + default=None, + help="Send the request via a proxy. Should be the URL giving the proxy address.", +) +@click.option( + "--timeout", + "timeout", + type=float, + default=5.0, + help=( + "Timeout value to use for network operations, such as establishing the " + "connection, reading some data, etc... [Default: 5.0]" + ), +) +@click.option( + "--follow-redirects", + "follow_redirects", + is_flag=True, + default=False, + help="Automatically follow redirects.", +) +@click.option( + "--no-verify", + "verify", + is_flag=True, + default=True, + help="Disable SSL verification.", +) +@click.option( + "--http2", + "http2", + type=bool, + is_flag=True, + default=False, + help="Send the request using HTTP/2, if the remote server supports it.", +) +@click.option( + "--download", + type=click.File("wb"), + help="Save the response content as a file, rather than displaying it.", +) +@click.option( + "--verbose", + "-v", + type=bool, + is_flag=True, + default=False, + help="Verbose. Show request as well as response.", +) +@click.option( + "--help", + is_flag=True, + is_eager=True, + expose_value=False, + callback=handle_help, + help="Show this message and exit.", +) +def main( + url: str, + method: str, + params: typing.List[typing.Tuple[str, str]], + content: str, + data: typing.List[typing.Tuple[str, str]], + files: typing.List[typing.Tuple[str, click.File]], + json: str, + headers: typing.List[typing.Tuple[str, str]], + cookies: typing.List[typing.Tuple[str, str]], + auth: typing.Optional[typing.Tuple[str, str]], + proxies: str, + timeout: float, + follow_redirects: bool, + verify: bool, + http2: bool, + download: typing.Optional[typing.BinaryIO], + verbose: bool, +) -> None: + """ + An HTTP command line client. + Sends a request and displays the response. + """ + if not method: + method = "POST" if content or data or files or json else "GET" + + event_hooks: typing.Dict[str, typing.List[typing.Callable]] = {} + if verbose: + event_hooks["request"] = [print_request_headers] + if follow_redirects: + event_hooks["response"] = [print_redirects] + + try: + with Client( + proxies=proxies, + timeout=timeout, + verify=verify, + http2=http2, + event_hooks=event_hooks, + ) as client: + with client.stream( + method, + url, + params=list(params), + content=content, + data=dict(data), + files=files, # type: ignore + json=json, + headers=headers, + cookies=dict(cookies), + auth=auth, + follow_redirects=follow_redirects, + ) as response: + print_response_headers(response) + + if download is not None: + download_response(response, download) + else: + response.read() + if response.content: + print_delimiter() + print_response(response) + + except RequestError as exc: + console = rich.console.Console() + console.print(f"{type(exc).__name__}: {exc}") + sys.exit(1) + + sys.exit(0 if response.is_success else 1) diff --git a/requirements.txt b/requirements.txt index f65a5cc2ea..8d92864357 100644 --- a/requirements.txt +++ b/requirements.txt @@ -2,7 +2,7 @@ # On the other hand, we're not pinning package dependencies, because our tests # needs to pass with the latest version of the packages. # Reference: https://github.com/encode/httpx/pull/1721#discussion_r661241588 --e .[http2,brotli] +-e .[cli,http2,brotli] # Documentation mkdocs==1.2.2 diff --git a/setup.py b/setup.py index 243bbd830b..e22afc6f13 100644 --- a/setup.py +++ b/setup.py @@ -69,6 +69,14 @@ def get_packages(package): "brotli; platform_python_implementation == 'CPython'", "brotlicffi; platform_python_implementation != 'CPython'" ], + "cli": [ + "click==8.*", + "rich==10.*", + "pygments==2.*" + ] + }, + entry_points = { + "console_scripts": "httpx=httpx:main" }, classifiers=[ "Development Status :: 4 - Beta", diff --git a/tests/conftest.py b/tests/conftest.py index 1ed87a467a..c40df09720 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -84,6 +84,8 @@ async def app(scope, receive, send): await echo_headers(scope, receive, send) elif scope["path"].startswith("/redirect_301"): await redirect_301(scope, receive, send) + elif scope["path"].startswith("/json"): + await hello_world_json(scope, receive, send) else: await hello_world(scope, receive, send) @@ -99,6 +101,17 @@ async def hello_world(scope, receive, send): await send({"type": "http.response.body", "body": b"Hello, world!"}) +async def hello_world_json(scope, receive, send): + await send( + { + "type": "http.response.start", + "status": 200, + "headers": [[b"content-type", b"application/json"]], + } + ) + await send({"type": "http.response.body", "body": b'{"Hello": "world!"}'}) + + async def slow_response(scope, receive, send): await sleep(1.0) await send( diff --git a/tests/test_main.py b/tests/test_main.py new file mode 100644 index 0000000000..1afd538e3f --- /dev/null +++ b/tests/test_main.py @@ -0,0 +1,165 @@ +import os + +from click.testing import CliRunner + +import httpx + + +def splitlines(output): + return [line.strip() for line in output.splitlines()] + + +def remove_date_header(lines): + return [line for line in lines if not line.startswith("date:")] + + +def test_help(): + runner = CliRunner() + result = runner.invoke(httpx.main, ["--help"]) + assert result.exit_code == 0 + assert "A next generation HTTP client." in result.output + + +def test_get(server): + url = str(server.url) + runner = CliRunner() + result = runner.invoke(httpx.main, [url]) + assert result.exit_code == 0 + assert remove_date_header(splitlines(result.output)) == [ + "HTTP/1.1 200 OK", + "server: uvicorn", + "content-type: text/plain", + "Transfer-Encoding: chunked", + "", + "Hello, world!", + ] + + +def test_json(server): + url = str(server.url.copy_with(path="/json")) + runner = CliRunner() + result = runner.invoke(httpx.main, [url]) + assert result.exit_code == 0 + assert remove_date_header(splitlines(result.output)) == [ + "HTTP/1.1 200 OK", + "server: uvicorn", + "content-type: application/json", + "Transfer-Encoding: chunked", + "", + "{", + '"Hello": "world!"', + "}", + ] + + +def test_redirects(server): + url = str(server.url.copy_with(path="/redirect_301")) + runner = CliRunner() + result = runner.invoke(httpx.main, [url]) + assert result.exit_code == 1 + assert remove_date_header(splitlines(result.output)) == [ + "HTTP/1.1 301 Moved Permanently", + "server: uvicorn", + "location: /", + "Transfer-Encoding: chunked", + ] + + +def test_follow_redirects(server): + url = str(server.url.copy_with(path="/redirect_301")) + runner = CliRunner() + result = runner.invoke(httpx.main, [url, "--follow-redirects"]) + assert result.exit_code == 0 + assert remove_date_header(splitlines(result.output)) == [ + "HTTP/1.1 301 Moved Permanently", + "server: uvicorn", + "location: /", + "Transfer-Encoding: chunked", + "", + "HTTP/1.1 200 OK", + "server: uvicorn", + "content-type: text/plain", + "Transfer-Encoding: chunked", + "", + "Hello, world!", + ] + + +def test_post(server): + url = str(server.url.copy_with(path="/echo_body")) + runner = CliRunner() + result = runner.invoke(httpx.main, [url, "-m", "POST", "-j", '{"hello": "world"}']) + assert result.exit_code == 0 + assert remove_date_header(splitlines(result.output)) == [ + "HTTP/1.1 200 OK", + "server: uvicorn", + "content-type: text/plain", + "Transfer-Encoding: chunked", + "", + '{"hello": "world"}', + ] + + +def test_verbose(server): + url = str(server.url) + runner = CliRunner() + result = runner.invoke(httpx.main, [url, "-v"]) + assert result.exit_code == 0 + assert remove_date_header(splitlines(result.output)) == [ + "GET / HTTP/1.1", + f"Host: {server.url.netloc.decode('ascii')}", + "Accept: */*", + "Accept-Encoding: gzip, deflate, br", + "Connection: keep-alive", + f"User-Agent: python-httpx/{httpx.__version__}", + "", + "HTTP/1.1 200 OK", + "server: uvicorn", + "content-type: text/plain", + "Transfer-Encoding: chunked", + "", + "Hello, world!", + ] + + +def test_auth(server): + url = str(server.url) + runner = CliRunner() + result = runner.invoke(httpx.main, [url, "-v", "--auth", "username", "password"]) + print(result.output) + assert result.exit_code == 0 + assert remove_date_header(splitlines(result.output)) == [ + "GET / HTTP/1.1", + f"Host: {server.url.netloc.decode('ascii')}", + "Accept: */*", + "Accept-Encoding: gzip, deflate, br", + "Connection: keep-alive", + f"User-Agent: python-httpx/{httpx.__version__}", + "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=", + "", + "HTTP/1.1 200 OK", + "server: uvicorn", + "content-type: text/plain", + "Transfer-Encoding: chunked", + "", + "Hello, world!", + ] + + +def test_download(server): + url = str(server.url) + runner = CliRunner() + with runner.isolated_filesystem(): + runner.invoke(httpx.main, [url, "--download", "index.txt"]) + assert os.path.exists("index.txt") + with open("index.txt", "r") as input_file: + assert input_file.read() == "Hello, world!" + + +def test_errors(): + runner = CliRunner() + result = runner.invoke(httpx.main, ["invalid://example.org"]) + assert result.exit_code == 1 + assert splitlines(result.output) == [ + "UnsupportedProtocol: Request URL has an unsupported protocol 'invalid://'.", + ]