Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement builtin import removal #1645

Merged
merged 1 commit into from Jan 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
25 changes: 13 additions & 12 deletions README.md
Expand Up @@ -698,6 +698,7 @@ For more, see [pyupgrade](https://pypi.org/project/pyupgrade/3.2.0/) on PyPI.
| UP026 | RewriteMockImport | `mock` is deprecated, use `unittest.mock` | 🛠 |
| UP027 | RewriteListComprehension | Replace unpacked list comprehension with a generator expression | 🛠 |
| UP028 | RewriteYieldFrom | Replace `yield` over `for` loop with `yield from` | 🛠 |
| UP029 | UnnecessaryBuiltinImport | Unnecessary builtin import: `...` | 🛠 |

### pep8-naming (N)

Expand Down Expand Up @@ -963,7 +964,7 @@ For more, see [flake8-simplify](https://pypi.org/project/flake8-simplify/0.19.3/

| Code | Name | Message | Fix |
| ---- | ---- | ------- | --- |
| SIM105 | UseContextlibSuppress | Use 'contextlib.suppress(..)' instead of try-except-pass | |
| SIM105 | UseContextlibSuppress | Use `contextlib.suppress(...)` instead of try-except-pass | |
| SIM118 | KeyInDict | Use `key in dict` instead of `key in dict.keys()` | 🛠 |
| SIM220 | AAndNotA | Use `False` instead of `... and not ...` | 🛠 |
| SIM221 | AOrNotA | Use `True` instead of `... or not ...` | 🛠 |
Expand Down Expand Up @@ -1337,11 +1338,11 @@ Under those conditions, Ruff implements every rule in Flake8.
Ruff also re-implements some of the most popular Flake8 plugins and related code quality tools
natively, including:

- [`autoflake`](https://pypi.org/project/autoflake/) (1/7)
- [`autoflake`](https://pypi.org/project/autoflake/) ([#1647](https://github.com/charliermarsh/ruff/issues/1647))
- [`eradicate`](https://pypi.org/project/eradicate/)
- [`flake8-2020`](https://pypi.org/project/flake8-2020/)
- [`flake8-annotations`](https://pypi.org/project/flake8-annotations/)
- [`flake8-bandit`](https://pypi.org/project/flake8-bandit/) (7/40)
- [`flake8-bandit`](https://pypi.org/project/flake8-bandit/) ([#1646](https://github.com/charliermarsh/ruff/issues/1646))
- [`flake8-blind-except`](https://pypi.org/project/flake8-blind-except/)
- [`flake8-boolean-trap`](https://pypi.org/project/flake8-boolean-trap/)
- [`flake8-bugbear`](https://pypi.org/project/flake8-bugbear/)
Expand All @@ -1354,19 +1355,19 @@ natively, including:
- [`flake8-errmsg`](https://pypi.org/project/flake8-errmsg/)
- [`flake8-implicit-str-concat`](https://pypi.org/project/flake8-implicit-str-concat/)
- [`flake8-import-conventions`](https://github.com/joaopalmeiro/flake8-import-conventions)
- [`flake8-pie`](https://pypi.org/project/flake8-pie/) (3/7)
- [`flake8-pie`](https://pypi.org/project/flake8-pie/) ([#1543](https://github.com/charliermarsh/ruff/issues/1543))
- [`flake8-print`](https://pypi.org/project/flake8-print/)
- [`flake8-quotes`](https://pypi.org/project/flake8-quotes/)
- [`flake8-return`](https://pypi.org/project/flake8-return/)
- [`flake8-simplify`](https://pypi.org/project/flake8-simplify/) (7/30)
- [`flake8-simplify`](https://pypi.org/project/flake8-simplify/) ([#998](https://github.com/charliermarsh/ruff/issues/998))
- [`flake8-super`](https://pypi.org/project/flake8-super/)
- [`flake8-tidy-imports`](https://pypi.org/project/flake8-tidy-imports/)
- [`isort`](https://pypi.org/project/isort/)
- [`mccabe`](https://pypi.org/project/mccabe/)
- [`pep8-naming`](https://pypi.org/project/pep8-naming/)
- [`pydocstyle`](https://pypi.org/project/pydocstyle/)
- [`pygrep-hooks`](https://github.com/pre-commit/pygrep-hooks) (3/6)
- [`pyupgrade`](https://pypi.org/project/pyupgrade/) (21/33)
- [`pygrep-hooks`](https://github.com/pre-commit/pygrep-hooks) ([#980](https://github.com/charliermarsh/ruff/issues/980))
- [`pyupgrade`](https://pypi.org/project/pyupgrade/) ([#827](https://github.com/charliermarsh/ruff/issues/827))
- [`yesqa`](https://github.com/asottile/yesqa)

Note that, in some cases, Ruff uses different error code prefixes than would be found in the
Expand Down Expand Up @@ -1406,7 +1407,7 @@ Today, Ruff can be used to replace Flake8 when used with any of the following pl

- [`flake8-2020`](https://pypi.org/project/flake8-2020/)
- [`flake8-annotations`](https://pypi.org/project/flake8-annotations/)
- [`flake8-bandit`](https://pypi.org/project/flake8-bandit/) (7/40)
- [`flake8-bandit`](https://pypi.org/project/flake8-bandit/) ([#1646](https://github.com/charliermarsh/ruff/issues/1646))
- [`flake8-blind-except`](https://pypi.org/project/flake8-blind-except/)
- [`flake8-boolean-trap`](https://pypi.org/project/flake8-boolean-trap/)
- [`flake8-bugbear`](https://pypi.org/project/flake8-bugbear/)
Expand All @@ -1419,11 +1420,11 @@ Today, Ruff can be used to replace Flake8 when used with any of the following pl
- [`flake8-errmsg`](https://pypi.org/project/flake8-errmsg/)
- [`flake8-implicit-str-concat`](https://pypi.org/project/flake8-implicit-str-concat/)
- [`flake8-import-conventions`](https://github.com/joaopalmeiro/flake8-import-conventions)
- [`flake8-pie`](https://pypi.org/project/flake8-pie/) (3/7)
- [`flake8-pie`](https://pypi.org/project/flake8-pie/) ([#1543](https://github.com/charliermarsh/ruff/issues/1543))
- [`flake8-print`](https://pypi.org/project/flake8-print/)
- [`flake8-quotes`](https://pypi.org/project/flake8-quotes/)
- [`flake8-return`](https://pypi.org/project/flake8-return/)
- [`flake8-simplify`](https://pypi.org/project/flake8-simplify/) (7/30)
- [`flake8-simplify`](https://pypi.org/project/flake8-simplify/) ([#998](https://github.com/charliermarsh/ruff/issues/998))
- [`flake8-super`](https://pypi.org/project/flake8-super/)
- [`flake8-tidy-imports`](https://pypi.org/project/flake8-tidy-imports/)
- [`mccabe`](https://pypi.org/project/mccabe/)
Expand All @@ -1432,8 +1433,8 @@ Today, Ruff can be used to replace Flake8 when used with any of the following pl

Ruff can also replace [`isort`](https://pypi.org/project/isort/),
[`yesqa`](https://github.com/asottile/yesqa), [`eradicate`](https://pypi.org/project/eradicate/),
[`pygrep-hooks`](https://github.com/pre-commit/pygrep-hooks) (3/6), and a subset of the rules
implemented in [`pyupgrade`](https://pypi.org/project/pyupgrade/) (21/33).
[`pygrep-hooks`](https://github.com/pre-commit/pygrep-hooks) ([#980](https://github.com/charliermarsh/ruff/issues/980)), and a subset of the rules
implemented in [`pyupgrade`](https://pypi.org/project/pyupgrade/) ([#827](https://github.com/charliermarsh/ruff/issues/827)).

If you're looking to use Ruff, but rely on an unsupported Flake8 plugin, feel free to file an Issue.

Expand Down
9 changes: 9 additions & 0 deletions resources/test/fixtures/pyupgrade/UP029.py
@@ -0,0 +1,9 @@
from builtins import *
from builtins import ascii, bytes, compile
from builtins import str as _str
from six.moves import filter, zip, zip_longest
from io import open
import io
import six
import six.moves
import builtins
1 change: 1 addition & 0 deletions ruff.schema.json
Expand Up @@ -996,6 +996,7 @@
"UP026",
"UP027",
"UP028",
"UP029",
"W",
"W2",
"W29",
Expand Down
26 changes: 24 additions & 2 deletions src/autofix/helpers.rs
Expand Up @@ -211,11 +211,33 @@ pub fn remove_unused_imports<'a>(
Some(SmallStatement::ImportFrom(import_body)) => {
if let ImportNames::Aliases(names) = &mut import_body.names {
(names, import_body.module.as_ref())
} else if let ImportNames::Star(..) = &import_body.names {
// Special-case: if the import is a `from ... import *`, then we delete the
// entire statement.
let mut found_star = false;
for unused_import in unused_imports {
let full_name = match import_body.module.as_ref() {
Some(module_name) => format!("{}.*", compose_module_path(module_name),),
None => "*".to_string(),
};
if unused_import == full_name {
found_star = true;
} else {
bail!(
"Expected \"*\" for unused import (got: \"{}\")",
unused_import
);
}
}
if !found_star {
bail!("Expected \'*\' for unused import");
}
return delete_stmt(stmt, parent, deleted, locator);
} else {
bail!("Expected ImportNames::Aliases")
bail!("Expected: ImportNames::Aliases | ImportNames::Star");
}
}
_ => bail!("Expected SmallStatement::ImportFrom or SmallStatement::Import"),
_ => bail!("Expected: SmallStatement::ImportFrom | SmallStatement::Import"),
};

// Preserve the trailing comma (or not) from the last entry.
Expand Down
9 changes: 7 additions & 2 deletions src/checkers/ast.rs
Expand Up @@ -890,14 +890,19 @@ where
}
}

if let Some("__future__") = module.as_deref() {
if self.settings.enabled.contains(&CheckCode::UP010) {
if self.settings.enabled.contains(&CheckCode::UP010) {
if let Some("__future__") = module.as_deref() {
pyupgrade::plugins::unnecessary_future_import(self, stmt, names);
}
}
if self.settings.enabled.contains(&CheckCode::UP026) {
pyupgrade::plugins::rewrite_mock_import(self, stmt);
}
if self.settings.enabled.contains(&CheckCode::UP029) {
if let Some(module) = module.as_deref() {
pyupgrade::plugins::unnecessary_builtin_import(self, stmt, module, names);
}
}

if self.settings.enabled.contains(&CheckCode::TID251) {
if let Some(module) = module {
Expand Down
1 change: 1 addition & 0 deletions src/pyupgrade/mod.rs
Expand Up @@ -50,6 +50,7 @@ mod tests {
#[test_case(CheckCode::UP027, Path::new("UP027.py"); "UP027")]
#[test_case(CheckCode::UP028, Path::new("UP028_0.py"); "UP028_0")]
#[test_case(CheckCode::UP028, Path::new("UP028_1.py"); "UP028_1")]
#[test_case(CheckCode::UP029, Path::new("UP029.py"); "UP029")]
fn checks(check_code: CheckCode, path: &Path) -> Result<()> {
let snapshot = format!("{}_{}", check_code.as_ref(), path.to_string_lossy());
let checks = test_path(
Expand Down
2 changes: 2 additions & 0 deletions src/pyupgrade/plugins/mod.rs
Expand Up @@ -16,6 +16,7 @@ pub use rewrite_yield_from::rewrite_yield_from;
pub use super_call_with_parameters::super_call_with_parameters;
pub use type_of_primitive::type_of_primitive;
pub use typing_text_str_alias::typing_text_str_alias;
pub use unnecessary_builtin_import::unnecessary_builtin_import;
pub use unnecessary_encode_utf8::unnecessary_encode_utf8;
pub use unnecessary_future_import::unnecessary_future_import;
pub use unnecessary_lru_cache_params::unnecessary_lru_cache_params;
Expand Down Expand Up @@ -43,6 +44,7 @@ mod rewrite_yield_from;
mod super_call_with_parameters;
mod type_of_primitive;
mod typing_text_str_alias;
mod unnecessary_builtin_import;
mod unnecessary_encode_utf8;
mod unnecessary_future_import;
mod unnecessary_lru_cache_params;
Expand Down
107 changes: 107 additions & 0 deletions src/pyupgrade/plugins/unnecessary_builtin_import.rs
@@ -0,0 +1,107 @@
use itertools::Itertools;
use log::error;
use rustpython_ast::{Alias, AliasData, Located};
use rustpython_parser::ast::Stmt;

use crate::ast::types::Range;
use crate::autofix;
use crate::checkers::ast::Checker;
use crate::registry::{Check, CheckKind};

const BUILTINS: &[&str] = &[
"*",
"ascii",
"bytes",
"chr",
"dict",
"filter",
"hex",
"input",
"int",
"isinstance",
"list",
"map",
"max",
"min",
"next",
"object",
"oct",
"open",
"pow",
"range",
"round",
"str",
"super",
"zip",
];
const IO: &[&str] = &["open"];
const SIX_MOVES_BUILTINS: &[&str] = BUILTINS;
const SIX: &[&str] = &["callable", "next"];
const SIX_MOVES: &[&str] = &["filter", "input", "map", "range", "zip"];

/// UP029
pub fn unnecessary_builtin_import(
checker: &mut Checker,
stmt: &Stmt,
module: &str,
names: &[Located<AliasData>],
) {
let deprecated_names = match module {
"builtins" => BUILTINS,
"io" => IO,
"six" => SIX,
"six.moves" => SIX_MOVES,
"six.moves.builtins" => SIX_MOVES_BUILTINS,
_ => return,
};

let mut unused_imports: Vec<&Alias> = vec![];
for alias in names {
if alias.node.asname.is_some() {
continue;
}
if deprecated_names.contains(&alias.node.name.as_str()) {
unused_imports.push(alias);
}
}

if unused_imports.is_empty() {
return;
}
let mut check = Check::new(
CheckKind::UnnecessaryBuiltinImport(
unused_imports
.iter()
.map(|alias| alias.node.name.to_string())
.sorted()
.collect(),
),
Range::from_located(stmt),
);

if checker.patch(check.kind.code()) {
let deleted: Vec<&Stmt> = checker.deletions.iter().map(|node| node.0).collect();
let defined_by = checker.current_stmt();
let defined_in = checker.current_stmt_parent();
let unused_imports: Vec<String> = unused_imports
.iter()
.map(|alias| format!("{module}.{}", alias.node.name))
.collect();
match autofix::helpers::remove_unused_imports(
unused_imports.iter().map(String::as_str),
defined_by.0,
defined_in.map(|node| node.0),
&deleted,
checker.locator,
) {
Ok(fix) => {
if fix.content.is_empty() || fix.content == "pass" {
checker.deletions.insert(defined_by.clone());
}
check.amend(fix);
}
Err(e) => error!("Failed to remove builtin import: {e}"),
}
}
checker.add_check(check);
}
3 changes: 3 additions & 0 deletions src/pyupgrade/plugins/unnecessary_future_import.rs
Expand Up @@ -38,6 +38,9 @@ pub fn unnecessary_future_import(checker: &mut Checker, stmt: &Stmt, names: &[Lo

let mut unused_imports: Vec<&Alias> = vec![];
for alias in names {
if alias.node.asname.is_some() {
continue;
}
if (target_version >= PythonVersion::Py33
&& PY33_PLUS_REMOVE_FUTURES.contains(&alias.node.name.as_str()))
|| (target_version >= PythonVersion::Py37
Expand Down
@@ -0,0 +1,79 @@
---
source: src/pyupgrade/mod.rs
expression: checks
---
- kind:
UnnecessaryBuiltinImport:
- "*"
location:
row: 1
column: 0
end_location:
row: 1
column: 22
fix:
content: ""
location:
row: 1
column: 0
end_location:
row: 2
column: 0
parent: ~
- kind:
UnnecessaryBuiltinImport:
- ascii
- bytes
location:
row: 2
column: 0
end_location:
row: 2
column: 42
fix:
content: from builtins import compile
location:
row: 2
column: 0
end_location:
row: 2
column: 42
parent: ~
- kind:
UnnecessaryBuiltinImport:
- filter
- zip
location:
row: 4
column: 0
end_location:
row: 4
column: 46
fix:
content: from six.moves import zip_longest
location:
row: 4
column: 0
end_location:
row: 4
column: 46
parent: ~
- kind:
UnnecessaryBuiltinImport:
- open
location:
row: 5
column: 0
end_location:
row: 5
column: 19
fix:
content: ""
location:
row: 5
column: 0
end_location:
row: 6
column: 0
parent: ~