Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Magic Trailing Commas in isort #1363

Merged
merged 37 commits into from
Dec 26, 2022
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
33e2744
Added magic comma support
colin99d Dec 24, 2022
ddc7be6
Removed print
colin99d Dec 24, 2022
5cc06bb
Updated test py file
colin99d Dec 24, 2022
a2cec4c
Got tests working
colin99d Dec 24, 2022
027747b
Cleaned up code
colin99d Dec 24, 2022
0bddc50
Clippy and fmt
colin99d Dec 24, 2022
c7ed221
Added requested test cases
colin99d Dec 24, 2022
bcaa370
Added some more fixes
colin99d Dec 24, 2022
eb68853
Minor formatting updates
colin99d Dec 24, 2022
c406311
Added some documentation
colin99d Dec 24, 2022
01adfee
Improved the logic
colin99d Dec 25, 2022
519de3a
reverted to previous change
colin99d Dec 25, 2022
aa9a801
Could be what we do next
colin99d Dec 25, 2022
227d548
Refactored to send location as another paramter
colin99d Dec 26, 2022
151e94c
Got it working
colin99d Dec 26, 2022
5c445f9
Merge branch 'main' into 1200
colin99d Dec 26, 2022
d8f2711
Removed prints, fmt, clippy
colin99d Dec 26, 2022
d2872de
Added clippy fixes
colin99d Dec 26, 2022
4b694ea
split comma checker into a separate function
colin99d Dec 26, 2022
d8d7010
Made separate function use a for statement
colin99d Dec 26, 2022
df06afa
Got it working
colin99d Dec 26, 2022
7ede557
Got fixes added
colin99d Dec 26, 2022
3411079
Added the option
colin99d Dec 26, 2022
d6d8682
Actually use the option
colin99d Dec 26, 2022
9332e36
Merge and resolve conflicts
colin99d Dec 26, 2022
46e12fe
Fixed outdated README
colin99d Dec 26, 2022
075f169
Added some more fixes
colin99d Dec 26, 2022
d4dcda8
Fixed line too long
colin99d Dec 26, 2022
de7520a
Use lexer-based detection
charliermarsh Dec 26, 2022
45a9739
Fixed clippy
colin99d Dec 26, 2022
8161e98
Make setting true by default
charliermarsh Dec 26, 2022
097d620
Invert tests
charliermarsh Dec 26, 2022
561d477
Update snapshots
charliermarsh Dec 26, 2022
ba855d5
Merge branch '1200' of github.com:colin99d/ruff into 1200
charliermarsh Dec 26, 2022
f66b8f4
Generate options, etc.
charliermarsh Dec 26, 2022
de1843b
Tweak signature
charliermarsh Dec 26, 2022
8998f45
Remove repeated comment
charliermarsh Dec 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2530,6 +2530,26 @@ known-third-party = ["src"]

---

#### [`split-on-trailing-comma`](#split-on-trailing-comma)

If a comma is placed after the last member in a multi-line import, then
the imports will never be folded into one line.

See isort's [`split-on-trailing-comma`](https://pycqa.github.io/isort/docs/configuration/options.html#split-on-trailing-comma) option.

**Default value**: `true`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also made this the default, to match isort's profile = "black".


**Type**: `bool`

**Example usage**:

```toml
[tool.ruff.isort]
split-on-trailing-comma = false
```

---

### `mccabe`

#### [`max-complexity`](#max-complexity)
Expand Down
38 changes: 38 additions & 0 deletions resources/test/fixtures/isort/magic_trailing_comma.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# This has a magic trailing comma, will be sorted, but not rolled into one line
from sys import (
stderr,
argv,
stdout,
exit,
)

# No magic comma, this will be rolled into one line.
from os import (
path,
environ,
execl,
execv
)
colin99d marked this conversation as resolved.
Show resolved Hide resolved

from glob import (
glob,
iglob,
escape, # Ends with a comment, should still treat as magic trailing comma.
)

# These will be combined, but without a trailing comma.
from foo import bar
from foo import baz

# These will be combined, _with_ a trailing comma.
from module1 import member1
from module1 import (
member2,
member3,
)

# These will be combined, _with_ a trailing comma.
from module2 import member1, member2
from module2 import (
member3,
)
7 changes: 7 additions & 0 deletions ruff.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -1102,6 +1102,13 @@
"items": {
"type": "string"
}
},
"split-on-trailing-comma": {
"description": "If a comma is placed after the last member in a multi-line import, then the imports will never be folded into one line.\n\nSee isort's [`split-on-trailing-comma`](https://pycqa.github.io/isort/docs/configuration/options.html#split-on-trailing-comma) option.",
"type": [
"boolean",
"null"
]
}
},
"additionalProperties": false
Expand Down
8 changes: 5 additions & 3 deletions src/isort/format.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ pub fn format_import_from(
line_length: usize,
force_wrap_aliases: bool,
is_first: bool,
trailing_comma: bool,
) -> String {
if aliases.len() == 1
&& aliases
Expand All @@ -53,9 +54,10 @@ pub fn format_import_from(

// We can only inline if: (1) none of the aliases have atop comments, and (3)
// only the last alias (if any) has inline comments.
if aliases
.iter()
.all(|(_, CommentSet { atop, .. })| atop.is_empty())
if !trailing_comma
&& aliases
.iter()
.all(|(_, CommentSet { atop, .. })| atop.is_empty())
&& aliases
.iter()
.rev()
Expand Down
30 changes: 30 additions & 0 deletions src/isort/helpers.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,37 @@
use rustpython_ast::Stmt;
use rustpython_parser::lexer;
use rustpython_parser::lexer::Tok;

use crate::ast::types::Range;
use crate::isort::types::TrailingComma;
use crate::source_code_locator::SourceCodeLocator;

/// Return `true` if a `StmtKind::ImportFrom` statement ends with a magic
/// trailing comma.
pub fn trailing_comma(stmt: &Stmt, locator: &SourceCodeLocator) -> TrailingComma {
let contents = locator.slice_source_code_range(&Range::from_located(stmt));
let mut count: usize = 0;
let mut trailing_comma = TrailingComma::Absent;
for (_, tok, _) in lexer::make_tokenizer(&contents).flatten() {
if matches!(tok, Tok::Lpar) {
count += 1;
}
if matches!(tok, Tok::Rpar) {
count -= 1;
}
if count == 1 {
if matches!(tok, Tok::Newline | Tok::Indent | Tok::Dedent | Tok::Comment) {
continue;
} else if matches!(tok, Tok::Comma) {
trailing_comma = TrailingComma::Present;
} else {
trailing_comma = TrailingComma::Absent;
}
}
}
trailing_comma
}
colin99d marked this conversation as resolved.
Show resolved Hide resolved

/// Return `true` if a `Stmt` is preceded by a "comment break"
pub fn has_comment_break(stmt: &Stmt, locator: &SourceCodeLocator) -> bool {
// Starting from the `Stmt` (`def f(): pass`), we want to detect patterns like
Expand Down
80 changes: 66 additions & 14 deletions src/isort/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,14 @@ use rustpython_ast::{Stmt, StmtKind};

use crate::isort::categorize::{categorize, ImportType};
use crate::isort::comments::Comment;
use crate::isort::helpers::trailing_comma;
use crate::isort::sorting::{cmp_import_froms, cmp_members, cmp_modules};
use crate::isort::track::{Block, Trailer};
use crate::isort::types::{
AliasData, CommentSet, ImportBlock, ImportFromData, Importable, OrderedImportBlock,
TrailingComma,
};
use crate::SourceCodeLocator;

mod categorize;
mod comments;
Expand All @@ -32,6 +35,7 @@ pub struct AnnotatedAliasData<'a> {
pub atop: Vec<Comment<'a>>,
pub inline: Vec<Comment<'a>>,
}

#[derive(Debug)]
pub enum AnnotatedImport<'a> {
Import {
Expand All @@ -45,12 +49,15 @@ pub enum AnnotatedImport<'a> {
level: Option<&'a usize>,
atop: Vec<Comment<'a>>,
inline: Vec<Comment<'a>>,
trailing_comma: TrailingComma,
},
}

fn annotate_imports<'a>(
imports: &'a [&'a Stmt],
comments: Vec<Comment<'a>>,
locator: &SourceCodeLocator,
split_on_trailing_comma: bool,
) -> Vec<AnnotatedImport<'a>> {
let mut annotated = vec![];
let mut comments_iter = comments.into_iter().peekable();
Expand Down Expand Up @@ -137,6 +144,11 @@ fn annotate_imports<'a>(
module: module.as_ref(),
names: aliases,
level: level.as_ref(),
trailing_comma: if split_on_trailing_comma {
trailing_comma(import, locator)
} else {
TrailingComma::default()
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also tweaked the order of operations: instead of collecting locations, and looking at the locations at the end, we detect a trailing comma for each block, and then propagate it through the code as we merge imports.

atop,
inline,
});
Expand Down Expand Up @@ -190,7 +202,9 @@ fn normalize_imports(imports: Vec<AnnotatedImport>, combine_as_imports: bool) ->
level,
atop,
inline,
trailing_comma,
} => {
// Associate the comments with the first alias (best effort).
let single_import = names.len() == 1;

// If we're dealing with a multi-import block (i.e., a non-star, non-aliased
Expand Down Expand Up @@ -289,6 +303,15 @@ fn normalize_imports(imports: Vec<AnnotatedImport>, combine_as_imports: bool) ->
entry.inline.push(comment.value);
}
}

// Propagate trailing commas.
if matches!(trailing_comma, TrailingComma::Present) {
if let Some(entry) =
block.import_from.get_mut(&ImportFromData { module, level })
{
entry.2 = trailing_comma;
}
}
}
}
}
Expand Down Expand Up @@ -412,6 +435,7 @@ fn sort_imports(block: ImportBlock) -> OrderedImportBlock {
inline: comments.inline,
},
)]),
TrailingComma::Absent,
),
)
}),
Expand Down Expand Up @@ -439,40 +463,44 @@ fn sort_imports(block: ImportBlock) -> OrderedImportBlock {
inline: comments.inline,
},
)]),
TrailingComma::Absent,
),
)
}),
)
.map(|(import_from, (comments, aliases))| {
.map(|(import_from, (comments, aliases, locations))| {
// Within each `StmtKind::ImportFrom`, sort the members.
(
import_from,
comments,
locations,
aliases
.into_iter()
.sorted_by(|(alias1, _), (alias2, _)| cmp_members(alias1, alias2))
.collect::<Vec<(AliasData, CommentSet)>>(),
)
})
.sorted_by(|(import_from1, _, aliases1), (import_from2, _, aliases2)| {
cmp_import_froms(import_from1, import_from2).then_with(|| {
match (aliases1.first(), aliases2.first()) {
(None, None) => Ordering::Equal,
(None, Some(_)) => Ordering::Less,
(Some(_), None) => Ordering::Greater,
(Some((alias1, _)), Some((alias2, _))) => cmp_members(alias1, alias2),
}
})
}),
.sorted_by(
|(import_from1, _, _, aliases1), (import_from2, _, _, aliases2)| {
cmp_import_froms(import_from1, import_from2).then_with(|| {
match (aliases1.first(), aliases2.first()) {
(None, None) => Ordering::Equal,
(None, Some(_)) => Ordering::Less,
(Some(_), None) => Ordering::Greater,
(Some((alias1, _)), Some((alias2, _))) => cmp_members(alias1, alias2),
}
})
},
),
);

ordered
}

#[allow(clippy::too_many_arguments)]
pub fn format_imports(
block: &Block,
comments: Vec<Comment>,
locator: &SourceCodeLocator,
line_length: usize,
src: &[PathBuf],
package: Option<&Path>,
Expand All @@ -481,9 +509,10 @@ pub fn format_imports(
extra_standard_library: &BTreeSet<String>,
combine_as_imports: bool,
force_wrap_aliases: bool,
split_on_trailing_comma: bool,
) -> String {
let trailer = &block.trailer;
let block = annotate_imports(&block.imports, comments);
let block = annotate_imports(&block.imports, comments, locator, split_on_trailing_comma);

// Normalize imports (i.e., deduplicate, aggregate `from` imports).
let block = normalize_imports(block, combine_as_imports);
Expand Down Expand Up @@ -521,14 +550,15 @@ pub fn format_imports(
}

// Format `StmtKind::ImportFrom` statements.
for (import_from, comments, aliases) in &import_block.import_from {
for (import_from, comments, trailing_comma, aliases) in &import_block.import_from {
output.append(&format::format_import_from(
import_from,
comments,
aliases,
line_length,
force_wrap_aliases,
is_first_statement,
split_on_trailing_comma && matches!(trailing_comma, TrailingComma::Present),
));
is_first_statement = false;
}
Expand Down Expand Up @@ -588,6 +618,7 @@ mod tests {
#[test_case(Path::new("split.py"))]
#[test_case(Path::new("trailing_suffix.py"))]
#[test_case(Path::new("type_comments.py"))]
#[test_case(Path::new("magic_trailing_comma.py"))]
fn default(path: &Path) -> Result<()> {
let snapshot = format!("{}", path.to_string_lossy());
let mut checks = test_path(
Expand Down Expand Up @@ -646,4 +677,25 @@ mod tests {
insta::assert_yaml_snapshot!(snapshot, checks);
Ok(())
}

#[test_case(Path::new("magic_trailing_comma.py"))]
fn no_split_on_trailing_comma(path: &Path) -> Result<()> {
let snapshot = format!("split_on_trailing_comma_{}", path.to_string_lossy());
let mut checks = test_path(
Path::new("./resources/test/fixtures/isort")
.join(path)
.as_path(),
&Settings {
isort: isort::settings::Settings {
split_on_trailing_comma: false,
..isort::settings::Settings::default()
},
src: vec![Path::new("resources/test/fixtures/isort").to_path_buf()],
..Settings::for_rule(CheckCode::I001)
},
)?;
checks.sort_by_key(|check| check.location);
insta::assert_yaml_snapshot!(snapshot, checks);
Ok(())
}
}
2 changes: 2 additions & 0 deletions src/isort/plugins.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ pub fn check_imports(
let expected = format_imports(
block,
comments,
locator,
settings.line_length - indentation.len(),
&settings.src,
package,
Expand All @@ -80,6 +81,7 @@ pub fn check_imports(
&settings.isort.extra_standard_library,
settings.isort.combine_as_imports,
settings.isort.force_wrap_aliases,
settings.isort.split_on_trailing_comma,
);

// Expand the span the entire range, including leading and trailing space.
Expand Down