Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-100479: Add optional blueprint argument to pathlib.PurePath #100481

Closed
wants to merge 22 commits into from
Closed
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
a6fdd0e
Add `pathlib.PurePath.makepath()`; unify path object construction
barneygale Nov 20, 2022
b061747
Fix reST role name.
barneygale Dec 24, 2022
99eb8b1
Move call to `os.getcwd()` back into `Path.cwd()`
barneygale Dec 24, 2022
4759d01
Merge branch 'main' into gh-100479-add-makepath
barneygale Jan 5, 2023
ef6f4c3
Merge branch 'main' into gh-100479-add-makepath
barneygale Apr 3, 2023
595b8ae
Add news blurb.
barneygale Apr 3, 2023
dcfe70a
Merge branch 'main' into gh-100479-add-makepath
barneygale Apr 9, 2023
117fe4b
Add whatsnew entry
barneygale Apr 10, 2023
e75dedc
Merge branch 'main' into gh-100479-add-makepath
barneygale Apr 12, 2023
5a6bd3f
Merge branch 'main' into gh-100479-add-makepath
barneygale Apr 13, 2023
f2f1048
other --> pathsegments
barneygale Apr 24, 2023
3c172fb
Update Lib/pathlib.py
barneygale Apr 24, 2023
4637109
joinpath(*args) --> joinpath(*pathsegments)
barneygale Apr 24, 2023
ae48454
Restore _PathParents
barneygale Apr 25, 2023
e7a8fe3
Add note to `parents` about potential reference cycle.
barneygale Apr 25, 2023
7f12faa
Replace `makepath()` method with `template` initialiser argument.
barneygale Apr 25, 2023
687c764
Apply suggestions from code review
barneygale Apr 25, 2023
d7e326a
Fix docs for other classes.
barneygale Apr 25, 2023
a65d499
Pass template to `super()` to support diamond inheritance.
barneygale Apr 26, 2023
d4b15d7
Fixed missed `template` argument to super().
barneygale Apr 26, 2023
958b183
template --> blueprint
barneygale Apr 27, 2023
1e10188
Merge branch 'main' into gh-100479-add-makepath
barneygale May 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
34 changes: 30 additions & 4 deletions Doc/library/pathlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -359,8 +359,7 @@ Pure paths provide the following methods and properties:

.. attribute:: PurePath.parents

An immutable sequence providing access to the logical ancestors of
the path::
A tuple providing access to the logical ancestors of the path::

>>> p = PureWindowsPath('c:/foo/bar/setup.py')
>>> p.parents[0]
Expand All @@ -373,6 +372,9 @@ Pure paths provide the following methods and properties:
.. versionchanged:: 3.10
The parents sequence now supports :term:`slices <slice>` and negative index values.

.. versionchanged:: 3.12
Type changed from a tuple-like immutable sequence to a true tuple.

AlexWaygood marked this conversation as resolved.
Show resolved Hide resolved
.. attribute:: PurePath.parent

The logical parent of the path::
Expand Down Expand Up @@ -530,10 +532,10 @@ Pure paths provide the following methods and properties:
unintended effects.


.. method:: PurePath.joinpath(*other)
.. method:: PurePath.joinpath(*pathsegments)

Calling this method is equivalent to combining the path with each of
the *other* arguments in turn::
the given *pathsegments* in turn::

>>> PurePosixPath('/etc').joinpath('passwd')
PurePosixPath('/etc/passwd')
Expand All @@ -545,6 +547,30 @@ Pure paths provide the following methods and properties:
PureWindowsPath('c:/Program Files')


.. method:: PurePath.makepath(*pathsegments)

Create a new path object of the same type by combining the given
*pathsegments*. This method is called whenever a derivative path is created,
such as from :attr:`parent` and :meth:`relative_to`. Subclasses may
override this method to pass information to derivative paths, for example::

from pathlib import PurePosixPath

class MyPath(PurePosixPath):
def __init__(self, *args, session_id):
super().__init__(*args)
self.session_id = session_id

def makepath(self, *pathsegments):
return type(self)(*pathsegments, session_id=self.session_id)

etc = MyPath('/etc', session_id=42)
hosts = etc / 'hosts'
print(hosts.session_id) # 42

.. versionadded:: 3.12


.. method:: PurePath.match(pattern)

Match this path against the provided glob-style pattern. Return ``True``
Expand Down
5 changes: 5 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,11 @@ inspect
pathlib
-------

* Add support for subclassing :class:`pathlib.PurePath` and
:class:`~pathlib.Path`, plus their Posix- and Windows-specific variants.
Subclasses may override the new :meth:`~pathlib.PurePath.makepath` method to
pass information between path instances.

* Add :meth:`~pathlib.Path.walk` for walking the directory trees and generating
all file or directory names within them, similar to :func:`os.walk`.
(Contributed by Stanislav Zmiev in :gh:`90385`.)
Expand Down
91 changes: 35 additions & 56 deletions Lib/pathlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
import re
import sys
import warnings
from _collections_abc import Sequence
from errno import ENOENT, ENOTDIR, EBADF, ELOOP
from stat import S_ISDIR, S_ISLNK, S_ISREG, S_ISSOCK, S_ISBLK, S_ISCHR, S_ISFIFO
from urllib.parse import quote_from_bytes as urlquote_from_bytes
Expand Down Expand Up @@ -207,36 +206,6 @@ def _select_from(self, parent_path, is_dir, exists, scandir, normcase):
# Public API
#

class _PathParents(Sequence):
"""This object provides sequence-like access to the logical ancestors
of a path. Don't try to construct it yourself."""
__slots__ = ('_pathcls', '_drv', '_root', '_tail')

def __init__(self, path):
# We don't store the instance to avoid reference cycles
self._pathcls = type(path)
self._drv = path.drive
self._root = path.root
self._tail = path._tail

def __len__(self):
return len(self._tail)

def __getitem__(self, idx):
if isinstance(idx, slice):
return tuple(self[i] for i in range(*idx.indices(len(self))))

if idx >= len(self) or idx < -len(self):
raise IndexError(idx)
if idx < 0:
idx += len(self)
return self._pathcls._from_parsed_parts(self._drv, self._root,
self._tail[:-idx - 1])

def __repr__(self):
return "<{}.parents>".format(self._pathcls.__name__)
AlexWaygood marked this conversation as resolved.
Show resolved Hide resolved


class PurePath(object):
"""Base class for manipulating paths without I/O.

Expand Down Expand Up @@ -334,15 +303,14 @@ def _load_parts(self):
self._root = root
self._tail_cached = tail

@classmethod
def _from_parsed_parts(cls, drv, root, tail):
path = cls._format_parsed_parts(drv, root, tail)
self = cls(path)
self._str = path or '.'
self._drv = drv
self._root = root
self._tail_cached = tail
return self
def _from_parsed_parts(self, drv, root, tail):
path_str = self._format_parsed_parts(drv, root, tail)
path = self.makepath(path_str)
path._str = path_str or '.'
path._drv = drv
path._root = root
path._tail_cached = tail
return path

@classmethod
def _format_parsed_parts(cls, drv, root, tail):
Expand Down Expand Up @@ -576,8 +544,7 @@ def relative_to(self, other, /, *_deprecated, walk_up=False):
"scheduled for removal in Python {remove}")
warnings._deprecated("pathlib.PurePath.relative_to(*args)", msg,
remove=(3, 14))
path_cls = type(self)
other = path_cls(other, *_deprecated)
other = self.makepath(other, *_deprecated)
for step, path in enumerate([other] + list(other.parents)):
if self.is_relative_to(path):
break
Expand All @@ -586,7 +553,7 @@ def relative_to(self, other, /, *_deprecated, walk_up=False):
if step and not walk_up:
raise ValueError(f"{str(self)!r} is not in the subpath of {str(other)!r}")
parts = ['..'] * step + self._tail[len(path._tail):]
return path_cls(*parts)
return self.makepath(*parts)

def is_relative_to(self, other, /, *_deprecated):
"""Return True if the path is relative to another path or False.
Expand All @@ -597,7 +564,7 @@ def is_relative_to(self, other, /, *_deprecated):
"scheduled for removal in Python {remove}")
warnings._deprecated("pathlib.PurePath.is_relative_to(*args)",
msg, remove=(3, 14))
other = type(self)(other, *_deprecated)
other = self.makepath(other, *_deprecated)
return other == self or other in self.parents

@property
Expand All @@ -609,13 +576,20 @@ def parts(self):
else:
return tuple(self._tail)

def makepath(self, *args):
barneygale marked this conversation as resolved.
Show resolved Hide resolved
"""Construct a new path object from any number of path-like objects.
Subclasses may override this method to customize how new path objects
are created from methods like `iterdir()`.
"""
return type(self)(*args)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, wouldn't this be simpler as a classmethod?

Suggested change
def makepath(self, *args):
"""Construct a new path object from any number of path-like objects.
Subclasses may override this method to customize how new path objects
are created from methods like `iterdir()`.
"""
return type(self)(*args)
@classmethod
def makepath(cls, *args):
"""Construct a new path object from any number of path-like objects.
Subclasses may override this method to customize how new path objects
are created from methods like `iterdir()`.
"""
return cls(*args)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making it a classmethod means that you can't share state between paths no?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see -- sorry, just getting back to pathlib stuff after a while away!

I guess I find the "makepath" name quite unintuitive in that case. The name to me implies that it's "just" an alternative constructor, which I'd expect to be a classmethod. Maybe something like "sproutpath" (feel free to bikeshed the name), which is clearer that it is using the state of the current Path instance would be better?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with_segments(), perhaps? Complements with_name() etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with_segments sounds like it's kiiinda "the previous path's segements, but with some new ones added at the end".

What about "newpath"? I think that reinforces the fact that the returned object has completely different segments to the old one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although this method is mostly called under-the-hood, it may also be useful when dealing with subclasses of pathlib classes. For example:

from tarfile import TarFile, TarPath
readme_path = TarPath('README.txt', tarfile=TarFile('blah.tar.gz'))
license_path = readme_path.newpath('LICENSE.txt')

I mention this just in case it helps us with the naming question. In the above example, readme_path.newpath() is a sort of factory for path objects sharing the same underlying TarFile object. And I suppose factories make things. But I don't mind newpath() :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of the options floated so far, I like the "newpath" name the best. But it's your module now, afterall, so I don't feel like I should have the final call here :D

You could maybe start a thread on Discord or Discuss if you'd like the opinion of more core devs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

barneygale marked this conversation as resolved.
Show resolved Hide resolved

def joinpath(self, *args):
"""Combine this path with one or several arguments, and return a
new path representing either a subpath (if all arguments are relative
paths) or a totally different path (if one of the arguments is
anchored).
"""
return self.__class__(self._raw_path, *args)
return self.makepath(self._raw_path, *args)

def __truediv__(self, key):
try:
Expand All @@ -625,7 +599,7 @@ def __truediv__(self, key):

def __rtruediv__(self, key):
try:
return type(self)(key, self._raw_path)
return self.makepath(key, self._raw_path)
except TypeError:
return NotImplemented

Expand All @@ -641,8 +615,13 @@ def parent(self):

@property
def parents(self):
"""A sequence of this path's logical parents."""
return _PathParents(self)
"""A tuple of this path's logical parents."""
drv = self.drive
root = self.root
tail = self._tail
return tuple(
self._from_parsed_parts(drv, root, tail[:idx])
for idx in reversed(range(len(tail))))

def is_absolute(self):
"""True if the path is absolute (has both a root and, if applicable,
Expand Down Expand Up @@ -672,7 +651,7 @@ def match(self, path_pattern):
"""
Return True if this path matches the given pattern.
"""
pat = type(self)(path_pattern)
pat = self.makepath(path_pattern)
if not pat.parts:
raise ValueError("empty pattern")
pat_parts = pat._parts_normcase
Expand Down Expand Up @@ -747,7 +726,7 @@ def _make_child_relpath(self, name):
path_str = f'{path_str}{name}'
else:
path_str = name
path = type(self)(path_str)
path = self.makepath(path_str)
path._str = path_str
path._drv = self.drive
path._root = self.root
Expand Down Expand Up @@ -797,7 +776,7 @@ def samefile(self, other_path):
try:
other_st = other_path.stat()
except AttributeError:
other_st = self.__class__(other_path).stat()
other_st = self.makepath(other_path).stat()
return self._flavour.samestat(st, other_st)

def iterdir(self):
Expand Down Expand Up @@ -859,7 +838,7 @@ def absolute(self):
cwd = self._flavour.abspath(self.drive)
else:
cwd = os.getcwd()
return type(self)(cwd, self._raw_path)
return self.makepath(cwd, self._raw_path)

def resolve(self, strict=False):
"""
Expand All @@ -877,7 +856,7 @@ def check_eloop(e):
except OSError as e:
check_eloop(e)
raise
p = type(self)(s)
p = self.makepath(s)

# In non-strict mode, realpath() doesn't raise on symlink loops.
# Ensure we get an exception by calling stat()
Expand Down Expand Up @@ -967,7 +946,7 @@ def readlink(self):
"""
if not hasattr(os, "readlink"):
raise NotImplementedError("os.readlink() not available on this system")
return type(self)(os.readlink(self))
return self.makepath(os.readlink(self))

def touch(self, mode=0o666, exist_ok=True):
"""
Expand Down Expand Up @@ -1056,7 +1035,7 @@ def rename(self, target):
Returns the new Path instance pointing to the target path.
"""
os.rename(self, target)
return self.__class__(target)
return self.makepath(target)

def replace(self, target):
"""
Expand All @@ -1069,7 +1048,7 @@ def replace(self, target):
Returns the new Path instance pointing to the target path.
"""
os.replace(self, target)
return self.__class__(target)
return self.makepath(target)

def symlink_to(self, target, target_is_directory=False):
"""
Expand Down
52 changes: 37 additions & 15 deletions Lib/test/test_pathlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,12 @@
#

class _BasePurePathSubclass(object):
init_called = False

def __init__(self, *args):
def __init__(self, *args, session_id):
super().__init__(*args)
self.init_called = True
self.session_id = session_id

def makepath(self, *args):
return type(self)(*args, session_id=self.session_id)


class _BasePurePathTest(object):
Expand Down Expand Up @@ -121,20 +122,21 @@ def test_str_subclass_common(self):
self._check_str_subclass('a/b.txt')
self._check_str_subclass('/a/b.txt')

def test_init_called_common(self):
def test_makepath_common(self):
class P(_BasePurePathSubclass, self.cls):
pass
p = P('foo', 'bar')
self.assertTrue((p / 'foo').init_called)
self.assertTrue(('foo' / p).init_called)
self.assertTrue(p.joinpath('foo').init_called)
self.assertTrue(p.with_name('foo').init_called)
self.assertTrue(p.with_stem('foo').init_called)
self.assertTrue(p.with_suffix('.foo').init_called)
self.assertTrue(p.relative_to('foo').init_called)
self.assertTrue(p.parent.init_called)
p = P('foo', 'bar', session_id=42)
self.assertEqual(42, (p / 'foo').session_id)
self.assertEqual(42, ('foo' / p).session_id)
self.assertEqual(42, p.makepath('foo').session_id)
self.assertEqual(42, p.joinpath('foo').session_id)
self.assertEqual(42, p.with_name('foo').session_id)
self.assertEqual(42, p.with_stem('foo').session_id)
self.assertEqual(42, p.with_suffix('.foo').session_id)
self.assertEqual(42, p.relative_to('foo').session_id)
self.assertEqual(42, p.parent.session_id)
for parent in p.parents:
self.assertTrue(parent.init_called)
self.assertEqual(42, parent.session_id)

def _get_drive_root_parts(self, parts):
path = self.cls(*parts)
Expand Down Expand Up @@ -1623,6 +1625,26 @@ def test_home(self):
env['HOME'] = os.path.join(BASE, 'home')
self._test_home(self.cls.home())

def test_makepath(self):
class P(_BasePurePathSubclass, self.cls):
pass
p = P(BASE, session_id=42)
self.assertEqual(42, p.absolute().session_id)
self.assertEqual(42, p.resolve().session_id)
self.assertEqual(42, p.makepath('~').expanduser().session_id)
self.assertEqual(42, (p / 'fileA').rename(p / 'fileB').session_id)
self.assertEqual(42, (p / 'fileB').replace(p / 'fileA').session_id)
if os_helper.can_symlink():
self.assertEqual(42, (p / 'linkA').readlink().session_id)
for path in p.iterdir():
self.assertEqual(42, path.session_id)
for path in p.glob('*'):
self.assertEqual(42, path.session_id)
for path in p.rglob('*'):
self.assertEqual(42, path.session_id)
for dirpath, dirnames, filenames in p.walk():
self.assertEqual(42, dirpath.session_id)

def test_samefile(self):
fileA_path = os.path.join(BASE, 'fileA')
fileB_path = os.path.join(BASE, 'dirB', 'fileB')
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Add :meth:`pathlib.PurePath.makepath`, which creates a path object from
arguments. This method is called whenever a derivative path is created, such
as from :attr:`pathlib.PurePath.parent`. Subclasses may override this method
to pass information to derivative paths.
barneygale marked this conversation as resolved.
Show resolved Hide resolved