Skip to content

Commit

Permalink
Allow for modification of certain components
Browse files Browse the repository at this point in the history
When using the URIBulider, it's plausible that one would want to use a
base URI and then modify components like the path and query string
without replacing them wholesale. Let's add two methods for that as well
as a convenience method to replace the `.finalize().unsplit()` dance.

This updates our docs to be a bit friendlier and adds release notes for
this change.

Closes #29
  • Loading branch information
sigmavirus24 committed Apr 10, 2020
1 parent 3657ec4 commit a38427f
Show file tree
Hide file tree
Showing 6 changed files with 243 additions and 1 deletion.
6 changes: 6 additions & 0 deletions docs/source/api-ref/builder.rst
Expand Up @@ -16,10 +16,16 @@

.. automethod:: rfc3986.builder.URIBuilder.add_path

.. automethod:: rfc3986.builder.URIBuilder.extend_path

.. automethod:: rfc3986.builder.URIBuilder.add_query_from

.. automethod:: rfc3986.builder.URIBuilder.extend_query_with

.. automethod:: rfc3986.builder.URIBuilder.add_query

.. automethod:: rfc3986.builder.URIBuilder.add_fragment

.. automethod:: rfc3986.builder.URIBuilder.finalize

.. automethod:: rfc3986.builder.URIBuilder.geturl
9 changes: 9 additions & 0 deletions docs/source/release-notes/unreleased.rst
Expand Up @@ -6,8 +6,17 @@

See also `GitHub #57`_

- Add :meth:`~rfc3986.builder.URIBuilder.extend_path`,
:meth:`~rfc3986.builder.URIBuilder.extend_query_with`,
:meth:`~rfc3986.builder.URIBuilder.geturl` to
:class:`~rfc3986.builder.URIBuilder`.

See also `GitHub #29`_

.. links
.. _GitHub #29:
https://github.com/python-hyper/rfc3986/issues/29

.. _GitHub #57:
https://github.com/python-hyper/rfc3986/issues/57
59 changes: 58 additions & 1 deletion docs/source/user/building.rst
Expand Up @@ -22,7 +22,12 @@ Example Usage
.. note::

All of the methods on a :class:`~rfc3986.builder.URIBuilder` are
chainable (except :meth:`~rfc3986.builder.URIBuilder.finalize`).
chainable (except :meth:`~rfc3986.builder.URIBuilder.finalize` and
:meth:`~rfc3986.builder.URIBuilder.geturl` as neither returns a
:class:`~rfc3986.builder.URIBuilder`).

Building From Scratch
---------------------

Let's build a basic URL with just a scheme and host. First we create an
instance of :class:`~rfc3986.builder.URIBuilder`. Then we call
Expand All @@ -42,6 +47,10 @@ a :class:`~rfc3986.uri.URIReference` and call
... ).finalize().unsplit())
https://github.com


Replacing Components of a URI
-----------------------------

It is possible to update an existing URI by constructing a builder from an
instance of :class:`~rfc3986.uri.URIReference` or a textual representation:

Expand All @@ -53,6 +62,9 @@ instance of :class:`~rfc3986.uri.URIReference` or a textual representation:
... ).finalize().unsplit())
https://github.com

The Builder is Immutable
------------------------

Each time you invoke a method, you get a new instance of a
:class:`~rfc3986.builder.URIBuilder` class so you can build several different
URLs from one base instance.
Expand All @@ -74,6 +86,32 @@ URLs from one base instance.
... ).finalize().unsplit())
https://api.github.com/repos/sigmavirus24/rfc3986

Convenient Path Management
--------------------------

Because our builder is immutable, one could use the
:class:`~rfc3986.builder.URIBuilder` class to build a class to make HTTP
Requests that used the provided path to extend the original one.

.. doctest::

>>> from rfc3986 import builder
>>> github_builder = builder.URIBuilder().add_scheme(
... 'https'
... ).add_host(
... 'api.github.com'
... ).add_path(
... '/users'
... )
>>> print(github_builder.extend_path("sigmavirus24").geturl())
https://api.github.com/users/sigmavirus24
>>> print(github_builder.extend_path("lukasa").geturl())
https://api.github.com/users/lukasa


Convenient Credential Handling
------------------------------

|rfc3986| makes adding authentication credentials convenient. It takes care of
making the credentials URL safe. There are some characters someone might want
to include in a URL that are not safe for the authority component of a URL.
Expand All @@ -91,6 +129,9 @@ to include in a URL that are not safe for the authority component of a URL.
... ).finalize().unsplit())
https://us3r:p%40ssw0rd@api.github.com

Managing Query String Parameters
--------------------------------

Further, |rfc3986| attempts to simplify the process of adding query parameters
to a URL. For example, if we were using Elasticsearch, we might do something
like:
Expand All @@ -109,6 +150,22 @@ like:
... ).finalize().unsplit())
https://search.example.com/_search?q=repo%3Asigmavirus24%2Frfc3986&sort=created_at%3Aasc

If one also had an existing URL with query string that we merely wanted to
append to, we can also do that with |rfc3986|.

.. doctest::

>>> from rfc3986 import builder
>>> print(builder.URIBuilder().from_uri(
... 'https://search.example.com/_search?q=repo%3Asigmavirus24%2Frfc3986'
... ).extend_query_with(
... [('sort', 'created_at:asc')]
... ).finalize().unsplit())
https://search.example.com/_search?q=repo%3Asigmavirus24%2Frfc3986&sort=created_at%3Aasc

Adding Fragments
----------------

Finally, we provide a way to add a fragment to a URL. Let's build up a URL to
view the section of the RFC that refers to fragments:

Expand Down
60 changes: 60 additions & 0 deletions src/rfc3986/builder.py
Expand Up @@ -234,6 +234,35 @@ def add_path(self, path):
fragment=self.fragment,
)

def extend_path(self, path):
"""Extend the existing path value with the provided value.
.. versionadded:: 1.5.0
.. code-block:: python
>>> URIBuilder(path="/users").extend_path("/sigmavirus24")
URIBuilder(scheme=None, userinfo=None, host=None, port=None,
path='/users/sigmavirus24', query=None, fragment=None)
>>> URIBuilder(path="/users/").extend_path("/sigmavirus24")
URIBuilder(scheme=None, userinfo=None, host=None, port=None,
path='/users/sigmavirus24', query=None, fragment=None)
>>> URIBuilder(path="/users/").extend_path("sigmavirus24")
URIBuilder(scheme=None, userinfo=None, host=None, port=None,
path='/users/sigmavirus24', query=None, fragment=None)
>>> URIBuilder(path="/users").extend_path("sigmavirus24")
URIBuilder(scheme=None, userinfo=None, host=None, port=None,
path='/users/sigmavirus24', query=None, fragment=None)
"""
existing_path = self.path or ""
path = "{}/{}".format(existing_path.rstrip("/"), path.lstrip("/"))

return self.add_path(path)

def add_query_from(self, query_items):
"""Generate and add a query a dictionary or list of tuples.
Expand All @@ -260,6 +289,27 @@ def add_query_from(self, query_items):
fragment=self.fragment,
)

def extend_query_with(self, query_items):
"""Extend the existing query string with the new query items.
.. versionadded:: 1.5.0
.. code-block:: python
>>> URIBuilder(query='a=b+c').extend_query_with({'a': 'b c'})
URIBuilder(scheme=None, userinfo=None, host=None, port=None,
path=None, query='a=b+c&a=b+c', fragment=None)
>>> URIBuilder(query='a=b+c').extend_query_with([('a', 'b c')])
URIBuilder(scheme=None, userinfo=None, host=None, port=None,
path=None, query='a=b+c&a=b+c', fragment=None)
"""
original_query_items = compat.parse_qsl(self.query or "")
if not isinstance(query_items, list):
query_items = list(query_items.items())

return self.add_query_from(original_query_items + query_items)

def add_query(self, query):
"""Add a pre-formated query string to the URI.
Expand Down Expand Up @@ -324,3 +374,13 @@ def finalize(self):
self.query,
self.fragment,
)

def geturl(self):
"""Generate the URL from this builder.
.. versionadded:: 1.5.0
This is an alternative to calling :meth:`finalize` and keeping the
:class:`rfc3986.uri.URIReference` around.
"""
return self.finalize().unsplit()
6 changes: 6 additions & 0 deletions src/rfc3986/compat.py
Expand Up @@ -20,6 +20,11 @@
except ImportError: # Python 2.x
from urllib import quote as urlquote

try:
from urllib.parse import parse_qsl
except ImportError: # Python 2.x
from urlparse import parse_qsl

try:
from urllib.parse import urlencode
except ImportError: # Python 2.x
Expand All @@ -30,6 +35,7 @@
"to_str",
"urlquote",
"urlencode",
"parse_qsl",
)

PY3 = (3, 0) <= sys.version_info < (4, 0)
Expand Down
104 changes: 104 additions & 0 deletions tests/test_builder.py
Expand Up @@ -13,6 +13,11 @@
# See the License for the specific language governing permissions and
# limitations under the License.
"""Module containing the tests for the URIBuilder object."""
try:
from urllib.parse import parse_qs
except ImportError:
from urlparse import parse_qs

import pytest

from rfc3986 import builder, uri_reference
Expand Down Expand Up @@ -191,6 +196,88 @@ def test_add_fragment():
assert uribuilder.fragment == "section-2.5.1"


@pytest.mark.parametrize(
"uri, extend_with, expected_path",
[
("https://api.github.com", "/users", "/users"),
("https://api.github.com", "/users/", "/users/"),
("https://api.github.com", "users", "/users"),
("https://api.github.com", "users/", "/users/"),
("", "users/", "/users/"),
("", "users", "/users"),
("?foo=bar", "users", "/users"),
(
"https://api.github.com/users/",
"sigmavirus24",
"/users/sigmavirus24",
),
(
"https://api.github.com/users",
"sigmavirus24",
"/users/sigmavirus24",
),
(
"https://api.github.com/users",
"/sigmavirus24",
"/users/sigmavirus24",
),
],
)
def test_extend_path(uri, extend_with, expected_path):
"""Verify the behaviour of extend_path."""
uribuilder = (
builder.URIBuilder()
.from_uri(uri_reference(uri))
.extend_path(extend_with)
)
assert uribuilder.path == expected_path


@pytest.mark.parametrize(
"uri, extend_with, expected_query",
[
(
"https://github.com",
[("a", "b c"), ("d", "e&f")],
{"a": ["b c"], "d": ["e&f"]},
),
(
"https://github.com?a=0",
[("a", "b c"), ("d", "e&f")],
{"a": ["0", "b c"], "d": ["e&f"]},
),
(
"https://github.com?a=0&e=f",
[("a", "b c"), ("d", "e&f")],
{"a": ["0", "b c"], "e": ["f"], "d": ["e&f"]},
),
(
"https://github.com",
{"a": "b c", "d": "e&f"},
{"a": ["b c"], "d": ["e&f"]},
),
(
"https://github.com?a=0",
{"a": "b c", "d": "e&f"},
{"a": ["0", "b c"], "d": ["e&f"]},
),
(
"https://github.com?a=0&e=f",
{"a": "b c", "d": "e&f"},
{"a": ["0", "b c"], "e": ["f"], "d": ["e&f"]},
),
],
)
def test_extend_query_with(uri, extend_with, expected_query):
"""Verify the behaviour of extend_query_with."""
uribuilder = (
builder.URIBuilder()
.from_uri(uri_reference(uri))
.extend_query_with(extend_with)
)
assert parse_qs(uribuilder.query) == expected_query


def test_finalize():
"""Verify the whole thing."""
uri = (
Expand All @@ -207,3 +294,20 @@ def test_finalize():
"sigmavirus24/rfc3986"
)
assert expected == uri


def test_geturl():
"""Verify the short-cut to the URL."""
uri = (
builder.URIBuilder()
.add_scheme("https")
.add_credentials("sigmavirus24", "not-my-re@l-password")
.add_host("github.com")
.add_path("sigmavirus24/rfc3986")
.geturl()
)
expected = (
"https://sigmavirus24:not-my-re%40l-password@github.com/"
"sigmavirus24/rfc3986"
)
assert expected == uri

0 comments on commit a38427f

Please sign in to comment.