Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More category fixes #2

Closed
9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ or not, use:
<a href="{{ SITEURL }}/{{ cat.url }}">{{ cat }}</a>{{ ', ' if not loop.last }}
{% endfor %}

Additionally, this plugin adds `category.subcategories`: a `list` of categories
that have `category` as a parent.

{% for subcat in category.subcategories %}
<a href="{{ SITEURL }}/{{subcat.url}}">{{subcat.shortname|capitalize}}</a>
{% endfor %}

thecaligarmo marked this conversation as resolved.
Show resolved Hide resolved
## Nested categories
(This is a reimplementation of the `subcategory` plugin.)

Expand Down Expand Up @@ -73,4 +80,4 @@ categories always follow their parent:

aba
aba/dat
abaala
abaala
4 changes: 4 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Release type: minor

Added subcategories to categories to be able to cycle through which subcategories are present for a category.
Updated plugin to better work with modern pelican standards (by updating tasks.py and ensuring proper code formatting)
Comment on lines +1 to +4
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should probably add a changelog, but I don't understand this addition. Is it it meant to be converted to a changelog by some automation tool?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's my fault, Oliver. This bit is related to AutoPub, a tool that I built to automate GitHub + PyPI releases upon pull request merge. I had meant to add AutoPub integration to More Categories as well, but that to-do item seems to have slipped between the cracks. I will submit a separate pull request for that for you to review, which will include continuous integration (CI) for More Categories, so its tests will automatically be run on pull requests such as this one. 😄

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So should I leave this one as is for now?

44 changes: 29 additions & 15 deletions pelican/plugins/more_categories/more_categories.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Title: More Categories
Description: adds support for multiple categories per article and nested
categories
Requirements: Pelican 3.8 or higher
Requirements: Pelican 4.2.0 or higher
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 4.2.0? (3.8 became 4.0, but that should be enough.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That should say 4.0.0. Sorry. The reason I changed it is because in the README it said 4.0.0. From what I can tell, it's requiring 4.0 functionality, so I decided to change it. Was that wrong?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once this plugin has been published to PyPI, if the goal is to be able to run pip install pelican-more-categories and have everything Just Work™, I believe the next release (tentatively planned as version 4.5) would be required, as the namespace plugin functionality was merged to master after v4.2 was released.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thecaligarmo Changing that to 4.0.0 is fine, absolutely!

@justinmayer Yes, but, users with older versions of pelican could still use this plugin the old-fashioned way right? Also, any particular reason why you want to skip from 4.2 to 4.5, or are you planning intermediate releases?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oulenz: Good point. I agree that specifying 4.0.0 makes more sense. Regarding the next release, current master drops Python 2.7 support and will most likely also drop Python 3.5 support. Plus, there are a decent number of changes in master since 4.2. As I mentioned in another thread, there aren't any changes I consider to be truly backwards-incompatible, so 5.0 seemed too big and an unwarranted jump, while 4.3 didn't seem to capture the extent of the changes, so I split the difference and suggested 4.5. Sometimes strict semantic versioning doesn't fit all situations. 🤷‍♂️

"""

from collections import defaultdict
Expand All @@ -19,13 +19,13 @@ class Category(URLWrapper):
@property
def _name(self):
if self.parent:
return self.parent._name + '/' + self.shortname
return self.parent._name + "/" + self.shortname
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't change single quotes to double quotes.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done automatically using the tasks that are present within pelican. In the lint section they are using "black". Black makes it so that single quotes are turned to double quotes to make coding conventions the same across the board. Since black is included in every other plugin and in (latest) pelican, I assume that is what is wanted.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I thought it would be beneficial to have consistent code style across the board, I understand that code style is very much subjective and something that shouldn't be forcibly foisted upon plugin developers who prefer a different style. So plugin authors are free to use their own conventions, as @oulenz has done with More Categories. For example, as you can see, Black is (purposely) not among the tools run during the lint task in this repository. So while I fully understand and appreciate your intention here, it would be best to honor this repository's style and not change these quotation marks. 😊

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok cool. I'll go ahead and undo those then. Sorry about that @oulenz

return self.shortname

@_name.setter
def _name(self, val):
if '/' in val:
parentname, val = val.rsplit('/', 1)
if "/" in val:
parentname, val = val.rsplit("/", 1)
self.parent = self.__class__(parentname, self.settings)
else:
self.parent = None
Expand All @@ -38,14 +38,13 @@ def name(self, val):
@property
def slug(self):
if self._slug is None:
if 'CATEGORY_REGEX_SUBSTITUTIONS' in self.settings:
subs = self.settings['CATEGORY_REGEX_SUBSTITUTIONS']
if "CATEGORY_REGEX_SUBSTITUTIONS" in self.settings:
subs = self.settings["CATEGORY_REGEX_SUBSTITUTIONS"]
else:
subs = self.settings.get('SLUG_REGEX_SUBSTITUTIONS', [])
subs = self.settings.get("SLUG_REGEX_SUBSTITUTIONS", [])
self._slug = slugify(self.shortname, regex_subs=subs)
print(self._slug)
thecaligarmo marked this conversation as resolved.
Show resolved Hide resolved
if self.parent:
self._slug = self.parent.slug + '/' + self._slug
self._slug = self.parent.slug + "/" + self._slug
return self._slug

@property
Expand All @@ -56,14 +55,14 @@ def ancestors(self):

def as_dict(self):
d = super(Category, self).as_dict()
d['shortname'] = self.shortname
d["shortname"] = self.shortname
return d


def get_categories(generator, metadata):
categories = text_type(metadata.get('category')).split(',')
metadata['categories'] = [Category(name, generator.settings) for name in categories]
metadata['category'] = metadata['categories'][0]
categories = text_type(metadata.get("category")).split(",")
metadata["categories"] = [Category(name, generator.settings) for name in categories]
metadata["category"] = metadata["categories"][0]


def create_categories(generator):
Expand All @@ -75,9 +74,24 @@ def create_categories(generator):

generator.categories = sorted(
list(cat_dct.items()),
reverse=generator.settings.get('REVERSE_CATEGORY_ORDER') or False,
reverse=generator.settings.get("REVERSE_CATEGORY_ORDER") or False,
)
generator._update_context(['categories'])
generator._update_context(["categories"])

# Add subcategories
cats = {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you initialise this as cats = defaultdict(list), you don't need to test whether it already contains a given category on L86.

for category, articles in generator.categories:
for anc in category.ancestors:
if anc != category:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you iterate from categorate.ancestors[1:], you don't need this check.

if anc.slug in cats:
cats[anc.slug].append(category)
else:
cats[anc.slug] = [category]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can simply use the category itself as the key, since its hash is the hash of its slug.

for category, articles in generator.categories:
if category.slug in cats:
category.subcategories = cats[category.slug]
else:
category.subcategories = []
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you split this up into two attributes, children and descendents (which includes the children)?

Also, please sort them to make things more convenient in the theme templates.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure. Do you care which method of sort? Or just the normal sort() method on lists?



def register():
Expand Down
16 changes: 8 additions & 8 deletions pelican/plugins/more_categories/test_more_categories.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,21 +14,21 @@
class TestArticlesGenerator(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.temp_path = mkdtemp(prefix='pelicantests.')
cls.temp_path = mkdtemp(prefix="pelicantests.")
more_categories.register()
settings = get_settings()
settings['DEFAULT_CATEGORY'] = 'default'
settings['CACHE_CONTENT'] = False
settings['PLUGINS'] = more_categories
settings["DEFAULT_CATEGORY"] = "default"
settings["CACHE_CONTENT"] = False
settings["PLUGINS"] = more_categories
context = get_context(settings)

base_path = os.path.dirname(os.path.abspath(__file__))
test_data_path = os.path.join(base_path, 'test_data')
test_data_path = os.path.join(base_path, "test_data")
cls.generator = ArticlesGenerator(
context=context,
settings=settings,
path=test_data_path,
theme=settings['THEME'],
theme=settings["THEME"],
output_path=cls.temp_path,
)
cls.generator.generate_context()
Expand All @@ -42,14 +42,14 @@ def test_generate_categories(self):
including ancestor categories"""

cats_generated = [cat.name for cat, _ in self.generator.categories]
cats_expected = ['default', 'foo', 'foo/bar', 'foo/b#az']
cats_expected = ["default", "foo", "foo/bar", "foo/b#az"]
self.assertEqual(sorted(cats_generated), sorted(cats_expected))

def test_categories_slug(self):
"""Test whether category slug substitutions are used"""

slugs_generated = [cat.slug for cat, _ in self.generator.categories]
slugs_expected = ['default', 'foo', 'foo/bar', 'foo/baz']
slugs_expected = ["default", "foo", "foo/bar", "foo/baz"]
self.assertEqual(sorted(slugs_generated), sorted(slugs_expected))

def test_assign_articles_to_categories(self):
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ invoke = "^1.3"
isort = "^4.3"
livereload = "^2.6"
markdown = "^3.1.1"
pytest = "^5.0"
pytest = "5.3.5"
thecaligarmo marked this conversation as resolved.
Show resolved Hide resolved
pytest-cov = "^2.7"
pytest-pythonpath = "^0.7.3"
pytest-sugar = "^0.9.2"
Expand Down
83 changes: 71 additions & 12 deletions tasks.py
Original file line number Diff line number Diff line change
@@ -1,34 +1,93 @@
import os
from pathlib import Path
from shutil import which

from invoke import task

PKG_NAME = 'more_categories'
PKG_PATH = Path(f'pelican/plugins/{PKG_NAME}')
FLAKE8 = 'flake8'
ISORT = 'isort'
PYTEST = 'pytest'
PKG_NAME = "more_categories"
PKG_PATH = Path(f"pelican/plugins/{PKG_NAME}")

ACTIVE_VENV = os.environ.get("VIRTUAL_ENV", None)
VENV_HOME = Path(os.environ.get("WORKON_HOME", "~/.local/share/virtualenvs"))
VENV_PATH = Path(ACTIVE_VENV) if ACTIVE_VENV else (VENV_HOME / PKG_NAME)
VENV = str(VENV_PATH.expanduser())
Comment on lines +10 to +13
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer activating a suitable virtual environment manually. In particular because I (and maybe others) don't use virtualenv but conda.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just following the standard of all the other plugins and pelican itself. All the other ones are using virtual env and in order to keep everything consistent it would make sense to replicate those no?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, not if it disrupts the preferred tooling of plugin authors and others who prefer alternate workflows. 😉

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. Sorry .-. I'll go ahead and undo this as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So for this one. If I changed it so that it used the same thing like poetry:

POETRY = which("poetry") if which("poetry") else (VENV / Path("bin") / "poetry")

would that be ok @oulenz ? It would be the ebst of both worlds right? It uses paths when they exist, else it looks inside the environment bin. Or would you prefer I just use the same method that you had before?

Copy link
Collaborator

@oulenz oulenz Apr 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would work. I have to admit, I am getting utterly confused by how all these tools inter-operate. In this case, what I don't understand is why we'd want to use invoke to call poetry, rather than the other way around. If you do poetry run invoke ..., doesn't poetry then handle the environment, so we could have a cleaner tasks.py file? 🤔

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll go ahead and do that for now =) Maybe @justinmayer would have more idea why we use invoke for something like this? I'm still a newbie... .-.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My task automation priorities have been to (1) be flexible and accommodate different workflows and (2) make it easy for folks to set up a Pelican development environment. Regarding the first priority, there are (as we've discussed) many ways to handle virtual environments. I prefer to manage mine manually, rather than let Poetry manage them for me. At the same time, people should also be free to let Poetry handle virtual environments on their behalf. Along similar lines, some people (like me) prefer having Poetry and Pre-commit installed globally instead of having them only live in virtual environments. Some folks don't.

Regarding the second objective, the reason I use Invoke to call Poetry here is to keep things simple for new contributors. By putting as much as possible into Invoke tasks, that means that installing Invoke becomes the primary pre-requisite, and then Invoke can handle most of the rest. So in theory, pip install invoke && invoke setup becomes all that's required to get going. The concept of letting Poetry manage the environment and then using poetry run invoke … is valid enough, but that would depend on Poetry being globally installed, and I didn't want to force users to globally install anything just in order to contribute to the project. The way I configured tasks.py and the relevant instructions is that it all works with a minimal number of steps and regardless of whether Poetry and Pre-commit are globally installed or not.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation, it's helpful to understand what is logically necessary and what are design decisions. I agree that focusing everything on invoke and tasks.py is probably the easiest to understand for new contributors.


TOOLS = ["poetry", "pre-commit"]
POETRY = which("poetry") if which("poetry") else (VENV / Path("bin") / "poetry")
PRECOMMIT = (
which("pre-commit") if which("pre-commit") else (VENV / Path("bin") / "pre-commit")
)

LINT_TOOLS = ["flake8", "isort", "black"]


@task
def tests(c):
"""Run the test suite"""
c.run(f'{PYTEST}', pty=True)
c.run(f"{VENV}/bin/pytest", pty=True)


@task
def isort(c, check=False):
check_flag = ''
def black(c, check=False, diff=False):
"""Run Black auto-formatter, optionally with --check or --diff"""
check_flag, diff_flag = "", ""
if check:
check_flag = '-c'
c.run(f'{ISORT} {check_flag} --recursive {PKG_PATH}/* tasks.py')
check_flag = "--check"
if diff:
diff_flag = "--diff"
c.run(f"{VENV}/bin/black {check_flag} {diff_flag} {PKG_PATH} tasks.py")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This repo doesn't use black.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the other ones do... It would make sense to have a unified coding standard acorss the board no?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since plugin authors are volunteering their time and effort to maintain plugins, it's best to honor their wishes in this regard. For background, please read: getpelican/cookiecutter-pelican-plugin#2

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll go ahead and undo this.



@task
def isort(c, check=False, diff=False):
check_flag, diff_flag = "", ""
if check:
check_flag = "-c"
if diff:
diff_flag = "--diff"
c.run(
f"{VENV}/bin/isort {check_flag} {diff_flag} --recursive {PKG_PATH}/* tasks.py"
)


@task
def flake8(c):
c.run(f'{FLAKE8} {PKG_PATH} tasks.py')
c.run(f"{VENV}/bin/flake8 {PKG_PATH} tasks.py")


@task
def lint(c):
isort(c, check=True)
lint_tools(c)
flake8(c)
isort(c, check=True)
black(c, check=True)


@task
def lint_tools(c):
"""Install lint tools in the virtual environment if not already on PATH"""
for tool in LINT_TOOLS:
if not which(tool):
c.run(f"{VENV}/bin/pip install {tool}")


@task
def tools(c):
"""Install tools in the virtual environment if not already on PATH"""
for tool in TOOLS:
if not which(tool):
c.run(f"{VENV}/bin/pip install {tool}")


@task
def precommit(c):
"""Install pre-commit hooks to .git/hooks/pre-commit"""
c.run(f"{PRECOMMIT} install")


@task
def setup(c):
c.run(f"{VENV}/bin/pip install -U pip")
tools(c)
c.run(f"{POETRY} install")
precommit(c)