Feature/local extensions

[mwesterhof]

this is a proposed change to add a "_local_extensions" feature.
It will enable the use of an additional "_local_extensions" element in the cookiecutter.json file. This will have the exact same functionality as the "_extensions" element, except that it can be used to load jinja2 template extensions straight from the project template.

I'm not certain if this is the best approach to achieve this, but I'm happy to have some feedback, of course.

{
    "directory_name": "directory_name",
    "module_name": "module_name",
    "_local_extensions": ["local_extensions.SomeExtension"]
}

[croesnick]

I totally missed your PR, @mwesterhof, when quickly scanning the open PRs today for exactly this feature. While not being finally decided whether I like an implicit (silent) loading of local extensions or your more explicit approach via dedicated keyword, I definitely like your context-manager approach for prepending repo_dir to sys.path. 🙂

[mwesterhof]

@croesnick

I totally missed your PR, @mwesterhof, when quickly scanning the open PRs today for exactly this feature. While not being finally decided whether I like an implicit (silent) loading of local extensions or your more explicit approach via dedicated keyword, I definitely like your context-manager approach for prepending repo_dir to sys.path.

Thanks, it was a bit of a challenge to find the right way to implement this. I'd be very interested in this feature, though. When you decide on the way you want this to work, feel free to let me know. I'd be happy to make some changes

[insspb]

@mwesterhof @croesnick thank you for contribution.
As you both are active, we can speed up this feature implementation.
First it should be well documented from point how user should it use.

[mwesterhof]

@insspb @croesnick
i added some docs for this feature to the "advanced" section. I haven't specified a version number yet, of course... should I leave that the way it is, or can i just replace the X.x with the next upcoming version?

[con-f-use]

I'm sorry, it seems I've implemented a very similar thing in #1340 but at least mine is more documented than the other duplicate in #944 😅

For completeness I want to mention the issues, I know of, that have requested something like that #1211 and #547 so that people don't re-implement it again.

Btw. may I ask, why not just add the extension directory to the path. It will only stay there, as long as the interpreter process running cookiecutter lives, so it will not influence anything else. You don't even have to check of its existence as python ignores non-existent dirs on sys.path.

I'd gravitate toward the more simple solution that involves less code. For the same reason I'd also vote for using the _extensions keyword in the json for both local and package installed extensions.

If anything, make the folder name just extensions instead of local_extensions to go along with ./hooks/.

[grahamalama]

Love this PR @mwesterhof! I was thinking about how to tackle the modify config from a hook problem that others were having, and basically came to the same idea that you did here.

I had one idea that I thought might improve this implementation. I haven't fully thought it through, but I wanted to get your take:

As a cookiecutter user, I don't want to have to know / care about how to write a custom Jinja filter -- I just want to write a python function (or functions) that I can use in my cookiecutter.json.

What about instead of having users create a full-blown extension like:

class FoobarExtension(Extension):
    def __init__(self, environment):
        super(FoobarExtension, self).__init__(environment)
         
        def foobar():
             return "foobar"
        
        environment.filters['foobar'] = foobar

we had them create a module filters.py where they'd define a bunch of functions:

def foo():
   return "bar"

def baz():
   return "qux"

and we could do something like:

import inspect

import filters

functions = inspect.getmembers(filters, inspect.isfunction)
for function in functions:
    register_function_as_filter(function)

Thoughts on this?

[mwesterhof]

it's an interesting take, @grahamalama
I'm generally in favor of that kind of simplification, but i have a feeling it might be a bit too restrictive and too "magical".

I'm no expert on jinja, but i think we're ignoring a lot of the power of template filters by simplifying it this much. I'd propose a compromise:

@simple_filter
def foo():
    return 'stuff'

1. creating fully fledged jinja-based filters is still possible
2. creating a simple function based one is possible
3. registering the function based filters is explicit
4. it's possible to add other (helper) functions to the module without them erroneously being picked up as filters

it actually mimics some ideas in the django framework regarding template filters and template tags, and it's an approach that i'm a big fan of.

Any opinions? I'd be happy to implement this behavior in my branch

[con-f-use]

I still maintain, that just adding the extensions directory to the path and letting the user do whatever is the best, smallest, most simple and most generic solution.

I don't know what the extra _local_extensions keyword (the existing _extensions keyword works if the extensions dir is on the path) and elaborate import_patch... functionality brings to the table other than unnecessary complexity.

Either way, the simple_filter decorator could be imported from cookiecutter.utils or some place in the cookie-cutter namespace, though, as long as it's well documented. I like it.

[mwesterhof]

@con-f-use i would agree. importing the decorator as an additional helper from cookiecutter.utils is exactly what i had in mind. As far as separating the local extensions from the "regular" ones goes... i'm sure this is somewhat of a personal thing, but I'm personally a fan of having a clear distinction between "extensions bundled with the template" and "installed extensions". That's just my view on it, but i'd be happy to change that. I'm not entirely sure who makes that call, but please let me know

[mwesterhof]

@insspb @croesnick @con-f-use @grahamalama
i just resolved a few conflicts, and I'm wondering if anyone has time to look at this MR. I'm more than happy to implement some changes, if necessary

[gasbasd]

any news about this?

[grahamalama]

Would it be good to include tests for non-happy path? I can think of two cases, but maybe there are more:

• I include a filter in my template, but didn't include the extension in the _local_extensions list
• I include a filter in my template and extension in the _local_extensions list, but haven't defined that extension (or it's somehow not available to be imported)
  • It looks like there's a similar test for built-in Jinja extensions

[mwesterhof]

that's a good idea. i'll pick that up ASAP

[grahamalama]

If I only need one or two extensions and they're not that complicated, I can just define them directly in a module local_extensions.py, correct? Maybe instead of having the documentation suggest a more-complicated package setup, it could suggest simply that you have a module local_extensions available in the template root with extensions defined, with no reference to a folder or main.py. That would leave it up to the user to use a standalone module or a package structure depending on their needs.

[mwesterhof]

that seems fair. i'll make the change

[jonathan-s]

From what I gather reading the code you are adding the path of the local directory. So in effect you could create a folder that is named extensions that contains an __init__.py file. In that file you'll have the extensions and you'd refer to that in the cookiecutter.json file as 'extensions.MyExtension' and it should all work.

[jonathan-s]

I was looking for exactly this. I'm glad that this in the works. My preference, would be to not introduce new naming such as the _local_extensions and instead use the already existing _extensions. From reading the code on how you are patching the python path that shouldn't be a problem.

[jonathan-s]

@mwesterhof

it actually mimics some ideas in the django framework regarding template filters and template tags, and it's an approach that i'm a big fan of.

Any opinions? I'd be happy to implement this behavior in my branch

I like the @simplefilter approach as well.

[mwesterhof]

sorry guys, i've been quite sick recently and haven't gotten around to any work. I'll start working on this again though, so expect to see changes soon 💪

[jonathan-s]

Can we please get this in the 2.0 release?

[matrixise]

Hello, is there any update for this feature? Thank you

[con-f-use]

@audreyfeldroy Pretty please? 😁

[frankhuurman]

2 days ago I've opened up an issue requesting to use some sort of custom .py script or function while generating the cookiecutter project to set a value in the cookiecutter.json, I believe this feature is exactly what I'm looking for! (In my case I need to read out the git config file to grab the name of the user)

Is there any update on this?

This is the one feature needed to automate all the things with cookiecutter 😛

[con-f-use]

Yeah, please merge this asap, I'd prefer not to wait another 2.5 years. It would be incredibly useful!

[matrixise]

Hi, I hope the best for @insspb but unfortunately, there is no Github activities for his account since December 2021.

Could we review/merge this PR without him?

Thank you,

[mwesterhof]

@insspb indeed doesn't seem available long term. I'm sorry to once again have to bother @ssbarnea , but is there any way to proceed with this PR without his input?

[ssbarnea]

I was planning to merge it but I wanted to see the CI result first.

[mwesterhof]

@ssbarnea it seems that black linting was added somewhere along the line. I accounted for that, and it looks like everything passes now

[con-f-use]

I'm overjoyed 😁 Thanks ssabernea! 🥇 When can we expect that to be released?

Edit:
Included in 2.1.0 released on May 30th, 2022.

[frankhuurman]

Awesome, can't wait to use it cause this will really make things a lot more extensive!

Same question as @con-f-use, is there a planned release date(with updated Docs as well?).

[connerxyz]

I'm eager make use of this feature too! Thanks.

[alkatar21]

main.py must be __init__.py

[grahamalama]

@con-f-use looks like it made it into this release.