Organize imports collation

[rbuckton]

After some internal discussion about #51916 and #52090, I took a brief look at providing more control over the sorting behavior for "Organize imports" by adding some additional configuration options to UserPreferences:

• organizeImportsCollation - Indicates whether to perform "ordinal" (binary) sorting (i.e., by the numeric value associated with each code point), or "unicode" collation (using the Unicode Collation Algorithm/Intl.Collator). Default: "ordinal"
• organizeImportsLocale - Overrides the locale used for collation. Specify "auto" to use the UI locale. Only applies to organizeImportsCollation: "unicode". Default: "en"
• organizeImportsNumericCollation - Indicates whether to collate decimal numeric strings by their integer value (i.e., "a1" < "a2" < "a100") instead of just their code points (i.e., "a1" < "a100" < "a2"). Only applies to organizeImportsCollation: "unicode". Default: true
• organizeImportsAccentCollation - Indicates whether to compare characters with accents or other diacritic marks as unequal from the base character. Only applies to organizeImportsCollation: "unicode". Default: true.
• organizeImportsCaseFirst - Either "upper" to collate upper-case characters before lower-case, "lower" to collate lower-case characters before upper-case, or false to collate using the default for the locale. Only applies to organizeImportsCollation: "unicode". Default: false.

Each of these settings correlate to options passed to Intl.Collator. The organizeImportsNumericCollation and organizeImportsCaseFirst properties correspond to the numeric and caseFirst options to Intl.Collator. The combination of organizeImportsIgnoreCase and organizeImportsAccentCollation are used to determine the sensitivity of the collator:

organizeImportsIgnoreCase	organizeImportsAccentCollation	sensitivity
true	false	"base"
true	true	"accent"
false	false	"case"
false	true	"variant"

The default behavior is to perform an "ordinal" comparison in a way that is compatible with eslint's sort-imports rule, which is our current behavior. These collation rules only apply to import and export specifiers. The import statements themselves are sorted only by path, which continues to use "ordinal" sorting behavior.

These rules will also apply to auto-imports and import fixes, which will use the user's collation preferences to determine if a list of import or export specifiers is already sorted before selecting an insertion point.

These settings are not yet part of the VS Code settings schema, but can still be enabled using either the typescript.unstable or javascript.unstable configuration keys in settings.json:

{
  "typescript.unstable": {
    "organizeImportsCollation": "unicode",
    "organizeImportsCaseFirst": "upper"
  }
}

[typescript-bot]

Thanks for the PR! It looks like you've changed the TSServer protocol in some way. Please ensure that any changes here don't break consumers of the current TSServer API. For some extra review, we'll ping @sheetalkamat, @mjbvz, and @minestarks for you. Feel free to loop in other consumers/maintainers if necessary

[jakebailey]

The main thing I'll want to test is that it's possible to use sort-imports or simple-import-format (paired with some organize setting) and have organize be a no-op; that to me feels like the main blocker in that I know much of the team doesn't want to have ESLint enabled in the editor, so something has to get things right enough to not require a --fix (in which case, may as well not even do anything).

I do wonder how much of this we can detect within the file; if I have a file sorted via one of these methods, it'd be cool to not have to configure anything and let it be auto. That would work in our codebase as only new/ambiguous files would conflict with a lint rule, which I think would be pretty rare.

[rbuckton]

The main thing I'll want to test is that it's possible to use sort-imports or simple-import-format (paired with some organize setting) and have organize be a no-op; that to me feels like the main blocker in that I know much of the team doesn't want to have ESLint enabled in the editor, so something has to get things right enough to not require a --fix (in which case, may as well not even do anything).

Ideally, the configuration options should be enough to match compatibility though I may have to extend it slightly to fall back to ordinal comparisons depending on sensitivity so as to match rules like eslint-plugin-simple-import-sorting.

The default behavior ("ordinal") matches our current behavior in that it uses "eslint compatible" ordinal comparisons (where case-insensitivity/pseudo-case-folding is performed via .toLower() rather than .toUpper()).

I do wonder how much of this we can detect within the file; if I have a file sorted via one of these methods, it'd be cool to not have to configure anything and let it be auto. That would work in our codebase as only new/ambiguous files would conflict with a lint rule, which I think would be pretty rare.

For now, I've chosen to mostly leave sort detection alone (in that it only tests for case sensitive vs. case insensitive), except that it now uses the collation settings provided. I did experiment with collation settings detection, but I felt it was a bit unreliable since there are a lot of moving parts (such as determining the collation locale).

[jakebailey]

I finally had a chance to test this; it looks like (probably intentionally), using your example config produces an output that matches simple-import-sort (besides separating out blocks), which is awesome!

I've also tested this PR with #52090 applied to #51590 and it also works as expected. Very cool.

[jakebailey]

Hm, here's one case it differs. Organize says:

import {
    Baseline,
    Compiler,
    FileBasedTest,
    FileBasedTestConfiguration,
    getFileBasedTestConfigurationDescription,
    getFileBasedTestConfigurations,
    IO,
    RunnerBase,
    TestCaseParser,
    TestRunnerKind,
} from "./_namespaces/Harness";
import * as Utils from "./_namespaces/Utils";
import * as compiler from "./_namespaces/compiler";
import * as ts from "./_namespaces/ts";
import * as vpath from "./_namespaces/vpath";

But simple-import-sort says:

import * as compiler from "./_namespaces/compiler";
import {
    Baseline,
    Compiler,
    FileBasedTest,
    FileBasedTestConfiguration,
    getFileBasedTestConfigurationDescription,
    getFileBasedTestConfigurations,
    IO,
    RunnerBase,
    TestCaseParser,
    TestRunnerKind,
} from "./_namespaces/Harness";
import * as ts from "./_namespaces/ts";
import * as Utils from "./_namespaces/Utils";
import * as vpath from "./_namespaces/vpath";

which seems correct. It doesn't seem like this PR is applying the rules to the paths?

[rbuckton]

[...] It doesn't seem like this PR is applying the rules to the paths?

No, it isn't. I can, but wasn't sure if these rules made sense for paths as opposed to identifiers, so I left path comparisons as-is.

[rbuckton]

I've updated the PR to apply collation settings to paths as well.

[andrewbranch]

One option for exposing these settings in VS Code could be to make a single dropdown of “presets” that match popular formatters / lint rules 🤷

[rbuckton]

I started looking at detection of collation settings, in the same vein as detectSortCaseSensitivity, but I may wait on applying that.

[rbuckton]

Hm, here's one case it differs. Organize says: [...]

@jakebailey, can you check if the current version now aligns with simple-import-sort?

[jakebailey]

That example appears to be fixed, yeah, thanks! One thing I did notice is that in order to get numeric sorting to match (sort transformES5 before transformES2015), I also had to explicitly set:

"organizeImportsNumericCollation": true

But the doc comment seems to imply that it should be the default? (Maybe false is the default and the comment is outdated, or vice versa?)

[jakebailey]

Hm, just retested with:

    "typescript.unstable": {
        "organizeImportsCollation": "unicode",
        "organizeImportsCaseFirst": "upper",
        "organizeImportsNumericCollation": true,
    },

And I'm having trouble getting this to organize:

import {
    hasDecorators,
    HasDecorators,
} ...

HasDecorators should come first (to match simple-import-sort), but organize seems to accept it as sorted. Am I missing some option?

[jakebailey]

Just to finish this off, these are the settings now to make organize match simple-import-format:

    "typescript.unstable": {
        "organizeImportsCollation": "unicode",
        "organizeImportsCaseFirst": "upper",
        "organizeImportsIgnoreCase": false,
        "organizeImportsNumericCollation": true,
    },

[rbuckton]

[...] I'm having trouble getting this to organize [...]

Discussed offline and the issue was the default setting for organizeImportsIgnoreCase is "auto", which auto-detects case sensitivity from context. Since hasDecorators, HasDecorators is a valid case-insensitive sort order, it acted as if organizeImportsIgnoreCase was true. The organizeImportsCaseFirst setting is only respected when perform a case-sensitive sort.

[rubiesonthesky]

Just to finish this off, these are the settings now to make organize match simple-import-format:

    "typescript.unstable": {
        "organizeImportsCollation": "unicode",
        "organizeImportsCaseFirst": "upper",
        "organizeImportsIgnoreCase": false,
        "organizeImportsNumericCollation": true,
    },

There is still small inconsistent issue between Typescript's Organize imports and simple-import-sort plugin. Since this repository does not use (usually) dots in filenames, I don't think it will be problem for this repository, though.

Using VS Code Insiders with Typescript nightly and Typescript set to version 5.0.0-dev.20230202.

Organize imports

import {TreeHelperUtil} from '../utils/tree-helper.util';
import {TreeUtil} from '../utils/tree.util';

Eslint simple-import-sort

import {TreeUtil} from '../utils/tree.util';
import {TreeHelperUtil} from '../utils/tree-helper.util';

[jakebailey]

I believe that you're describing lydell/eslint-plugin-simple-import-sort#127.

Our general goal is to get the configurability (and behavior of various tools) to the point that we can offer up some presets that do "the right thing" for that plugin, dprint, etc, since this impacts auto-imports and it's nice to not have to immediately quick fix an eslint error.

[elboletaire]

I was about to open a bug as a regression of #23366, but after searching quite a lot I've found out these pull requests with these secret settings, and setting organizeImportsIgnoreCase to true seems to do what I want (note: using auto doesn't seem to be working well... since leaving it to "auto" was changing my imports from ignoring case to not ignoring it).

[mbest]

I'm seeing similar behavior. The detection is very inconsistent and seems to default to case sensitive. In version 4.x, organize imports always used case-insensitive sorting (see #24048).