Skip to content

Why does isort sort packages ending with a number oddly? #1732

Closed
@bobwalker99

Description

@bobwalker99
Contributor

Given these, sorted alphanumerically:

from bob.apples import aardvark
from bob2.apples2 import aardvark

isort reorders them like this:

from bob2.apples2 import aardvark
from bob.apples import aardvark

Is there a config setting that controls this behaviour? I don't understand the behaviour or the intention here.

Python by default sorts them as I would expect:

>>> l= ['from bob2.apples2 import aardvark','from bob.apples import aardvark']
>>> l.sort()
>>> l
['from bob.apples import aardvark', 'from bob2.apples2 import aardvark']

So this must be intentional? I've experimented with force_alphabetical_sort_within_sections and lexicographical and length_sort but I can only get accidental success which adversely affects other behaviour.

(Appreciate this is a contrived example, but it mirrors an actual scenario we have in our production code).

Activity

anirudnits

anirudnits commented on May 21, 2021

@anirudnits
Collaborator

i think that the problem lies here:

return [_atoi(c) for c in re.split(r"(\d+)", text)]
. The key for the two modules bob2.apples2 and bob.apples would be ["bob", "2", ".apples", "2"] and ["bob.apples"] respectively, and the first list is smaller than the second so the bob2.apples2 is sorted before bob.apples

bobwalker99

bobwalker99 commented on May 21, 2021

@bobwalker99
ContributorAuthor

Thanks @anirudnits. I don't understand what that code is designed to handle?

timothycrosley

timothycrosley commented on May 22, 2021

@timothycrosley
Member

@bobwalker99, sorry this behavior didn't match your expectations, and I can see why you might expect the other! What isort is trying to handle by sorting numbers in this way, is natural number sorting:

import module9
import module10
import module200

which the naive Python sorting turns into

import module10
import module200
import module9

Like you alluded to, both cases are corner cases, however in practice these numbered imports have popped up more commonly in codebases that use isort, and to this date we haven't had any requests for the other kind of case. Does the difference matter to you, would an option here be useful?

Thanks!

~Timothy

bobwalker99

bobwalker99 commented on May 22, 2021

@bobwalker99
ContributorAuthor

Hi Timothy, many thanks for the quick response, I understand what’s going on now. A config option for this would be really good - I’m happy to submit a PR, but I might need a nudge in the right direction of how you think it needs to work.
Regards
Bob

added
enhancementNew feature or request
and removed
bugSomething isn't working
on May 23, 2021
timothycrosley

timothycrosley commented on May 23, 2021

@timothycrosley
Member

Hi @bobwalker99,

Would happily accept such a PR! I think right now, all sorting gets routed to isort.sorting.naturally:

def naturally(

It might be worth introducing a new central sorting function, that is config aware, that then checks if natural sorting is set or not, and if not goes to the Python standard way of sorting, and rerouting all the existing calls through that new function. This would make it easier as well to provide a way for users to write fully custom sorters, which has been an ask before as well.

Happy to help in any way I can, let me know what you think

~Timothy

bobwalker99

bobwalker99 commented on May 23, 2021

@bobwalker99
ContributorAuthor

Hi @timothycrosley, many thanks for the guidance.

I've just started thinking about this a bit more - I'm not sure that toggling this behaviour is actually what we'd want? I agree it's reasonable to want modules sorted this way:

import module9
import module10
import module200

But the addition of sub-packages and modules of the same name without numbers seems to actually break the output? e.g.:

import module.module
import module9.module9
import module9.module10
import module9.module200

Becomes this:

import module9.module9
import module9.module10
import module9.module200
import module.module

If I'd set a "natural language sorting" setting to True, I think I'd expect import module.module to come before import module9.module9; (arguably, I'd expect this regardless of such a setting) ?

Nonetheless, your pluggable sorter suggestion is a good one, so I'll have a go at implementing that. If my use case has an audience of 1, I can then always write something that will do what I'm expecting ;-) !

Appreciate your input on this, all the best,

Bob

bobwalker99

bobwalker99 commented on Jun 17, 2021

@bobwalker99
ContributorAuthor

Hi @timothycrosley would you mind taking a look at #1752 to check I'm on the right track before I go too far with it please?

It's very basic inasmuch as it just expects a string descriptor for a custom module/function and will just failover to naturally silently if it can't load it, but it works with my very light testing so far.

I'm worried about the amount of testing that might be required if some custom function was employed in conjunction with other config settings, and how I would even test that?
Regards,
Bob

timothycrosley

timothycrosley commented on Jun 20, 2021

@timothycrosley
Member

Closing as this has been fixed by @bobwalker99's pull request: #1756 which was just merged in develop 🎉. A new PyPI release should appear in the coming days that contains the new sort mode flag

added a commit that references this issue on Jun 21, 2021

Implemented #1732: Support for custom sort functions.

2 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @bobwalker99@timothycrosley@anirudnits

        Issue actions

          Why does isort sort packages ending with a number oddly? · Issue #1732 · PyCQA/isort