Skip to content

Commit

Permalink
Implement builtin import removal (#1645)
Browse files Browse the repository at this point in the history
  • Loading branch information
charliermarsh committed Jan 5, 2023
1 parent 2ff816f commit 7339d7e
Show file tree
Hide file tree
Showing 12 changed files with 313 additions and 56 deletions.
25 changes: 13 additions & 12 deletions README.md
Expand Up @@ -698,6 +698,7 @@ For more, see [pyupgrade](https://pypi.org/project/pyupgrade/3.2.0/) on PyPI.
| UP026 | RewriteMockImport | `mock` is deprecated, use `unittest.mock` | 🛠 |
| UP027 | RewriteListComprehension | Replace unpacked list comprehension with a generator expression | 🛠 |
| UP028 | RewriteYieldFrom | Replace `yield` over `for` loop with `yield from` | 🛠 |
| UP029 | UnnecessaryBuiltinImport | Unnecessary builtin import: `...` | 🛠 |
### pep8-naming (N)
Expand Down Expand Up @@ -963,7 +964,7 @@ For more, see [flake8-simplify](https://pypi.org/project/flake8-simplify/0.19.3/
| Code | Name | Message | Fix |
| ---- | ---- | ------- | --- |
| SIM105 | UseContextlibSuppress | Use 'contextlib.suppress(..)' instead of try-except-pass | |
| SIM105 | UseContextlibSuppress | Use `contextlib.suppress(...)` instead of try-except-pass | |
| SIM118 | KeyInDict | Use `key in dict` instead of `key in dict.keys()` | 🛠 |
| SIM220 | AAndNotA | Use `False` instead of `... and not ...` | 🛠 |
| SIM221 | AOrNotA | Use `True` instead of `... or not ...` | 🛠 |
Expand Down Expand Up @@ -1337,11 +1338,11 @@ Under those conditions, Ruff implements every rule in Flake8.
Ruff also re-implements some of the most popular Flake8 plugins and related code quality tools
natively, including:
- [`autoflake`](https://pypi.org/project/autoflake/) (1/7)
- [`autoflake`](https://pypi.org/project/autoflake/) ([#1647](https://github.com/charliermarsh/ruff/issues/1647))
- [`eradicate`](https://pypi.org/project/eradicate/)
- [`flake8-2020`](https://pypi.org/project/flake8-2020/)
- [`flake8-annotations`](https://pypi.org/project/flake8-annotations/)
- [`flake8-bandit`](https://pypi.org/project/flake8-bandit/) (7/40)
- [`flake8-bandit`](https://pypi.org/project/flake8-bandit/) ([#1646](https://github.com/charliermarsh/ruff/issues/1646))
- [`flake8-blind-except`](https://pypi.org/project/flake8-blind-except/)
- [`flake8-boolean-trap`](https://pypi.org/project/flake8-boolean-trap/)
- [`flake8-bugbear`](https://pypi.org/project/flake8-bugbear/)
Expand All @@ -1354,19 +1355,19 @@ natively, including:
- [`flake8-errmsg`](https://pypi.org/project/flake8-errmsg/)
- [`flake8-implicit-str-concat`](https://pypi.org/project/flake8-implicit-str-concat/)
- [`flake8-import-conventions`](https://github.com/joaopalmeiro/flake8-import-conventions)
- [`flake8-pie`](https://pypi.org/project/flake8-pie/) (3/7)
- [`flake8-pie`](https://pypi.org/project/flake8-pie/) ([#1543](https://github.com/charliermarsh/ruff/issues/1543))
- [`flake8-print`](https://pypi.org/project/flake8-print/)
- [`flake8-quotes`](https://pypi.org/project/flake8-quotes/)
- [`flake8-return`](https://pypi.org/project/flake8-return/)
- [`flake8-simplify`](https://pypi.org/project/flake8-simplify/) (7/30)
- [`flake8-simplify`](https://pypi.org/project/flake8-simplify/) ([#998](https://github.com/charliermarsh/ruff/issues/998))
- [`flake8-super`](https://pypi.org/project/flake8-super/)
- [`flake8-tidy-imports`](https://pypi.org/project/flake8-tidy-imports/)
- [`isort`](https://pypi.org/project/isort/)
- [`mccabe`](https://pypi.org/project/mccabe/)
- [`pep8-naming`](https://pypi.org/project/pep8-naming/)
- [`pydocstyle`](https://pypi.org/project/pydocstyle/)
- [`pygrep-hooks`](https://github.com/pre-commit/pygrep-hooks) (3/6)
- [`pyupgrade`](https://pypi.org/project/pyupgrade/) (21/33)
- [`pygrep-hooks`](https://github.com/pre-commit/pygrep-hooks) ([#980](https://github.com/charliermarsh/ruff/issues/980))
- [`pyupgrade`](https://pypi.org/project/pyupgrade/) ([#827](https://github.com/charliermarsh/ruff/issues/827))
- [`yesqa`](https://github.com/asottile/yesqa)
Note that, in some cases, Ruff uses different error code prefixes than would be found in the
Expand Down Expand Up @@ -1406,7 +1407,7 @@ Today, Ruff can be used to replace Flake8 when used with any of the following pl
- [`flake8-2020`](https://pypi.org/project/flake8-2020/)
- [`flake8-annotations`](https://pypi.org/project/flake8-annotations/)
- [`flake8-bandit`](https://pypi.org/project/flake8-bandit/) (7/40)
- [`flake8-bandit`](https://pypi.org/project/flake8-bandit/) ([#1646](https://github.com/charliermarsh/ruff/issues/1646))
- [`flake8-blind-except`](https://pypi.org/project/flake8-blind-except/)
- [`flake8-boolean-trap`](https://pypi.org/project/flake8-boolean-trap/)
- [`flake8-bugbear`](https://pypi.org/project/flake8-bugbear/)
Expand All @@ -1419,11 +1420,11 @@ Today, Ruff can be used to replace Flake8 when used with any of the following pl
- [`flake8-errmsg`](https://pypi.org/project/flake8-errmsg/)
- [`flake8-implicit-str-concat`](https://pypi.org/project/flake8-implicit-str-concat/)
- [`flake8-import-conventions`](https://github.com/joaopalmeiro/flake8-import-conventions)
- [`flake8-pie`](https://pypi.org/project/flake8-pie/) (3/7)
- [`flake8-pie`](https://pypi.org/project/flake8-pie/) ([#1543](https://github.com/charliermarsh/ruff/issues/1543))
- [`flake8-print`](https://pypi.org/project/flake8-print/)
- [`flake8-quotes`](https://pypi.org/project/flake8-quotes/)
- [`flake8-return`](https://pypi.org/project/flake8-return/)
- [`flake8-simplify`](https://pypi.org/project/flake8-simplify/) (7/30)
- [`flake8-simplify`](https://pypi.org/project/flake8-simplify/) ([#998](https://github.com/charliermarsh/ruff/issues/998))
- [`flake8-super`](https://pypi.org/project/flake8-super/)
- [`flake8-tidy-imports`](https://pypi.org/project/flake8-tidy-imports/)
- [`mccabe`](https://pypi.org/project/mccabe/)
Expand All @@ -1432,8 +1433,8 @@ Today, Ruff can be used to replace Flake8 when used with any of the following pl
Ruff can also replace [`isort`](https://pypi.org/project/isort/),
[`yesqa`](https://github.com/asottile/yesqa), [`eradicate`](https://pypi.org/project/eradicate/),
[`pygrep-hooks`](https://github.com/pre-commit/pygrep-hooks) (3/6), and a subset of the rules
implemented in [`pyupgrade`](https://pypi.org/project/pyupgrade/) (21/33).
[`pygrep-hooks`](https://github.com/pre-commit/pygrep-hooks) ([#980](https://github.com/charliermarsh/ruff/issues/980)), and a subset of the rules
implemented in [`pyupgrade`](https://pypi.org/project/pyupgrade/) ([#827](https://github.com/charliermarsh/ruff/issues/827)).
If you're looking to use Ruff, but rely on an unsupported Flake8 plugin, feel free to file an Issue.
Expand Down
9 changes: 9 additions & 0 deletions resources/test/fixtures/pyupgrade/UP029.py
@@ -0,0 +1,9 @@
from builtins import *
from builtins import ascii, bytes, compile
from builtins import str as _str
from six.moves import filter, zip, zip_longest
from io import open
import io
import six
import six.moves
import builtins
1 change: 1 addition & 0 deletions ruff.schema.json
Expand Up @@ -996,6 +996,7 @@
"UP026",
"UP027",
"UP028",
"UP029",
"W",
"W2",
"W29",
Expand Down
26 changes: 24 additions & 2 deletions src/autofix/helpers.rs
Expand Up @@ -211,11 +211,33 @@ pub fn remove_unused_imports<'a>(
Some(SmallStatement::ImportFrom(import_body)) => {
if let ImportNames::Aliases(names) = &mut import_body.names {
(names, import_body.module.as_ref())
} else if let ImportNames::Star(..) = &import_body.names {
// Special-case: if the import is a `from ... import *`, then we delete the
// entire statement.
let mut found_star = false;
for unused_import in unused_imports {
let full_name = match import_body.module.as_ref() {
Some(module_name) => format!("{}.*", compose_module_path(module_name),),
None => "*".to_string(),
};
if unused_import == full_name {
found_star = true;
} else {
bail!(
"Expected \"*\" for unused import (got: \"{}\")",
unused_import
);
}
}
if !found_star {
bail!("Expected \'*\' for unused import");
}
return delete_stmt(stmt, parent, deleted, locator);
} else {
bail!("Expected ImportNames::Aliases")
bail!("Expected: ImportNames::Aliases | ImportNames::Star");
}
}
_ => bail!("Expected SmallStatement::ImportFrom or SmallStatement::Import"),
_ => bail!("Expected: SmallStatement::ImportFrom | SmallStatement::Import"),
};

// Preserve the trailing comma (or not) from the last entry.
Expand Down
9 changes: 7 additions & 2 deletions src/checkers/ast.rs
Expand Up @@ -890,14 +890,19 @@ where
}
}

if let Some("__future__") = module.as_deref() {
if self.settings.enabled.contains(&CheckCode::UP010) {
if self.settings.enabled.contains(&CheckCode::UP010) {
if let Some("__future__") = module.as_deref() {
pyupgrade::plugins::unnecessary_future_import(self, stmt, names);
}
}
if self.settings.enabled.contains(&CheckCode::UP026) {
pyupgrade::plugins::rewrite_mock_import(self, stmt);
}
if self.settings.enabled.contains(&CheckCode::UP029) {
if let Some(module) = module.as_deref() {
pyupgrade::plugins::unnecessary_builtin_import(self, stmt, module, names);
}
}

if self.settings.enabled.contains(&CheckCode::TID251) {
if let Some(module) = module {
Expand Down
1 change: 1 addition & 0 deletions src/pyupgrade/mod.rs
Expand Up @@ -50,6 +50,7 @@ mod tests {
#[test_case(CheckCode::UP027, Path::new("UP027.py"); "UP027")]
#[test_case(CheckCode::UP028, Path::new("UP028_0.py"); "UP028_0")]
#[test_case(CheckCode::UP028, Path::new("UP028_1.py"); "UP028_1")]
#[test_case(CheckCode::UP029, Path::new("UP029.py"); "UP029")]
fn checks(check_code: CheckCode, path: &Path) -> Result<()> {
let snapshot = format!("{}_{}", check_code.as_ref(), path.to_string_lossy());
let checks = test_path(
Expand Down
2 changes: 2 additions & 0 deletions src/pyupgrade/plugins/mod.rs
Expand Up @@ -16,6 +16,7 @@ pub use rewrite_yield_from::rewrite_yield_from;
pub use super_call_with_parameters::super_call_with_parameters;
pub use type_of_primitive::type_of_primitive;
pub use typing_text_str_alias::typing_text_str_alias;
pub use unnecessary_builtin_import::unnecessary_builtin_import;
pub use unnecessary_encode_utf8::unnecessary_encode_utf8;
pub use unnecessary_future_import::unnecessary_future_import;
pub use unnecessary_lru_cache_params::unnecessary_lru_cache_params;
Expand Down Expand Up @@ -43,6 +44,7 @@ mod rewrite_yield_from;
mod super_call_with_parameters;
mod type_of_primitive;
mod typing_text_str_alias;
mod unnecessary_builtin_import;
mod unnecessary_encode_utf8;
mod unnecessary_future_import;
mod unnecessary_lru_cache_params;
Expand Down
107 changes: 107 additions & 0 deletions src/pyupgrade/plugins/unnecessary_builtin_import.rs
@@ -0,0 +1,107 @@
use itertools::Itertools;
use log::error;
use rustpython_ast::{Alias, AliasData, Located};
use rustpython_parser::ast::Stmt;

use crate::ast::types::Range;
use crate::autofix;
use crate::checkers::ast::Checker;
use crate::registry::{Check, CheckKind};

const BUILTINS: &[&str] = &[
"*",
"ascii",
"bytes",
"chr",
"dict",
"filter",
"hex",
"input",
"int",
"isinstance",
"list",
"map",
"max",
"min",
"next",
"object",
"oct",
"open",
"pow",
"range",
"round",
"str",
"super",
"zip",
];
const IO: &[&str] = &["open"];
const SIX_MOVES_BUILTINS: &[&str] = BUILTINS;
const SIX: &[&str] = &["callable", "next"];
const SIX_MOVES: &[&str] = &["filter", "input", "map", "range", "zip"];

/// UP029
pub fn unnecessary_builtin_import(
checker: &mut Checker,
stmt: &Stmt,
module: &str,
names: &[Located<AliasData>],
) {
let deprecated_names = match module {
"builtins" => BUILTINS,
"io" => IO,
"six" => SIX,
"six.moves" => SIX_MOVES,
"six.moves.builtins" => SIX_MOVES_BUILTINS,
_ => return,
};

let mut unused_imports: Vec<&Alias> = vec![];
for alias in names {
if alias.node.asname.is_some() {
continue;
}
if deprecated_names.contains(&alias.node.name.as_str()) {
unused_imports.push(alias);
}
}

if unused_imports.is_empty() {
return;
}
let mut check = Check::new(
CheckKind::UnnecessaryBuiltinImport(
unused_imports
.iter()
.map(|alias| alias.node.name.to_string())
.sorted()
.collect(),
),
Range::from_located(stmt),
);

if checker.patch(check.kind.code()) {
let deleted: Vec<&Stmt> = checker.deletions.iter().map(|node| node.0).collect();
let defined_by = checker.current_stmt();
let defined_in = checker.current_stmt_parent();
let unused_imports: Vec<String> = unused_imports
.iter()
.map(|alias| format!("{module}.{}", alias.node.name))
.collect();
match autofix::helpers::remove_unused_imports(
unused_imports.iter().map(String::as_str),
defined_by.0,
defined_in.map(|node| node.0),
&deleted,
checker.locator,
) {
Ok(fix) => {
if fix.content.is_empty() || fix.content == "pass" {
checker.deletions.insert(defined_by.clone());
}
check.amend(fix);
}
Err(e) => error!("Failed to remove builtin import: {e}"),
}
}
checker.add_check(check);
}
3 changes: 3 additions & 0 deletions src/pyupgrade/plugins/unnecessary_future_import.rs
Expand Up @@ -38,6 +38,9 @@ pub fn unnecessary_future_import(checker: &mut Checker, stmt: &Stmt, names: &[Lo

let mut unused_imports: Vec<&Alias> = vec![];
for alias in names {
if alias.node.asname.is_some() {
continue;
}
if (target_version >= PythonVersion::Py33
&& PY33_PLUS_REMOVE_FUTURES.contains(&alias.node.name.as_str()))
|| (target_version >= PythonVersion::Py37
Expand Down
@@ -0,0 +1,79 @@
---
source: src/pyupgrade/mod.rs
expression: checks
---
- kind:
UnnecessaryBuiltinImport:
- "*"
location:
row: 1
column: 0
end_location:
row: 1
column: 22
fix:
content: ""
location:
row: 1
column: 0
end_location:
row: 2
column: 0
parent: ~
- kind:
UnnecessaryBuiltinImport:
- ascii
- bytes
location:
row: 2
column: 0
end_location:
row: 2
column: 42
fix:
content: from builtins import compile
location:
row: 2
column: 0
end_location:
row: 2
column: 42
parent: ~
- kind:
UnnecessaryBuiltinImport:
- filter
- zip
location:
row: 4
column: 0
end_location:
row: 4
column: 46
fix:
content: from six.moves import zip_longest
location:
row: 4
column: 0
end_location:
row: 4
column: 46
parent: ~
- kind:
UnnecessaryBuiltinImport:
- open
location:
row: 5
column: 0
end_location:
row: 5
column: 19
fix:
content: ""
location:
row: 5
column: 0
end_location:
row: 6
column: 0
parent: ~

0 comments on commit 7339d7e

Please sign in to comment.