Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't require --platform when running on a development machine #1727

Merged
merged 2 commits into from Feb 28, 2024
Merged

Don't require --platform when running on a development machine #1727

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4db49f5
Don't require `--platform` when running on a development machine
joerick Jan 21, 2024
75627e8
Limit line length to 100 cols
joerick Feb 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 0 additions & 3 deletions bin/dev_run_test
View file
Open in desktop

This file was deleted.

33 changes: 11 additions & 22 deletions cibuildwheel/__main__.py
View file
Open in desktop
Expand Up @@ -49,10 +49,9 @@ def main() -> None:
default=None,
help="""
Platform to build for. Use this option to override the
auto-detected platform or to run cibuildwheel on your development
machine. Specifying "macos" or "windows" only works on that
operating system, but "linux" works on all three, as long as
Docker/Podman is installed. Default: auto.
auto-detected platform. Specifying "macos" or "windows" only works
on that operating system, but "linux" works on all three, as long
as Docker/Podman is installed. Default: auto.
""",
)

Expand Down Expand Up @@ -184,20 +183,7 @@ def _compute_platform_only(only: str) -> PlatformName:
sys.exit(2)


def _compute_platform_ci() -> PlatformName:
if detect_ci_provider() is None:
print(
textwrap.dedent(
"""
cibuildwheel: Unable to detect platform. cibuildwheel should run on your CI server;
Travis CI, AppVeyor, Azure Pipelines, GitHub Actions, CircleCI, Gitlab, and Cirrus CI
are supported. You can run on your development machine or other CI providers
using the --platform argument. Check --help output for more information.
"""
),
file=sys.stderr,
)
sys.exit(2)
def _compute_platform_auto() -> PlatformName:
if sys.platform.startswith("linux"):
return "linux"
elif sys.platform == "darwin":
Expand All @@ -206,8 +192,9 @@ def _compute_platform_ci() -> PlatformName:
return "windows"
else:
print(
'cibuildwheel: Unable to detect platform from "sys.platform" in a CI environment. You can run '
"cibuildwheel using the --platform argument. Check --help output for more information.",
'cibuildwheel: Unable to detect platform from "sys.platform". cibuildwheel doesn\'t '
"support building wheels for this platform. You might be able to build for a different "
"platform using the --platform argument. Check --help output for more information.",
file=sys.stderr,
)
sys.exit(2)
Expand Down Expand Up @@ -238,7 +225,7 @@ def _compute_platform(args: CommandLineArguments) -> PlatformName:
elif platform_option_value != "auto":
return typing.cast(PlatformName, platform_option_value)

return _compute_platform_ci()
return _compute_platform_auto()


class PlatformModule(Protocol):
Expand Down Expand Up @@ -292,7 +279,9 @@ def build_in_directory(args: CommandLineArguments) -> None:
# Add CIBUILDWHEEL environment variable
os.environ["CIBUILDWHEEL"] = "1"

# Python is buffering by default when running on the CI platforms, giving problems interleaving subprocess call output with unflushed calls to 'print'
# Python is buffering by default when running on the CI platforms, giving
# problems interleaving subprocess call output with unflushed calls to
# 'print'
sys.stdout = Unbuffered(sys.stdout) # type: ignore[assignment]

# create the cache dir before it gets printed & builds performed
Expand Down
6 changes: 4 additions & 2 deletions docs/options.md
View file
Open in desktop
Expand Up @@ -184,22 +184,24 @@ Options: `auto` `linux` `macos` `windows`

Default: `auto`

`auto` will auto-detect platform using environment variables, such as `TRAVIS_OS_NAME`/`APPVEYOR`/`CIRCLECI`.
`auto` will build wheels for the current platform.

- For `linux`, you need [Docker or Podman](#container-engine) running, on Linux, macOS, or Windows.
- For `macos` and `windows`, you need to be running on the respective system, with a working compiler toolchain installed - Xcode Command Line tools for macOS, and MSVC for Windows.

This option can also be set using the [command-line option](#command-line) `--platform`. This option is not available in the `pyproject.toml` config.

!!! tip
You can use this option to locally debug your cibuildwheel config, instead of pushing to CI to test every change. For example:
You can use this option to locally debug your cibuildwheel config on Linux, instead of pushing to CI to test every change. For example:

```bash
export CIBW_BUILD='cp37-*'
export CIBW_TEST_COMMAND='pytest {package}/tests'
cibuildwheel --platform linux .
```

Linux builds are the easiest to test locally, because all the build tools are supplied in the container, and they run exactly the same locally as in CI.

This is even more convenient if you store your cibuildwheel config in [`pyproject.toml`](#configuration-file).

You can also run a single identifier with `--only <identifier>`. This will
Expand Down
51 changes: 10 additions & 41 deletions docs/setup.md
View file
Open in desktop
Expand Up @@ -11,45 +11,14 @@ locally to quickly iterate and track down issues without even touching CI.

Install cibuildwheel and run a build like this:

!!! tab "Linux"
```sh
# using pipx (https://github.com/pypa/pipx)
pipx run cibuildwheel

Using [pipx](https://github.com/pypa/pipx):
```sh
pipx run cibuildwheel --platform linux
```

Or,
```sh
pip install cibuildwheel
cibuildwheel --platform linux
```

!!! tab "macOS"

Using [pipx](https://github.com/pypa/pipx):
```sh
pipx run cibuildwheel --platform macos
```

Or,
```sh
pip install cibuildwheel
cibuildwheel --platform macos
```


!!! tab "Windows"

Using [pipx](https://github.com/pypa/pipx):
```bat
pipx run cibuildwheel --platform windows
```

Or,
```bat
pip install cibuildwheel
cibuildwheel --platform windows
```
# or,
pip install cibuildwheel
cibuildwheel
```

You should see the builds taking place. You can experiment with options using environment variables or pyproject.toml.

Expand All @@ -63,15 +32,15 @@ You should see the builds taking place. You can experiment with options using en
# run a command to set up the build system
export CIBW_BEFORE_ALL='apt install libpng-dev'

cibuildwheel --platform linux
cibuildwheel
```

> CMD (Windows)

```bat
set CIBW_BEFORE_ALL='apt install libpng-dev'

cibuildwheel --platform linux
cibuildwheel
```

!!! tab "pyproject.toml"
Expand All @@ -88,7 +57,7 @@ You should see the builds taking place. You can experiment with options using en
Then invoke cibuildwheel, like:

```console
cibuildwheel --platform linux
cibuildwheel
```

### Linux builds
Expand Down
35 changes: 18 additions & 17 deletions unit_test/main_tests/main_platform_test.py
View file
Open in desktop
Expand Up @@ -10,25 +10,26 @@
from ..conftest import MOCK_PACKAGE_DIR


def test_unknown_platform_non_ci(monkeypatch, capsys):
monkeypatch.delenv("CI", raising=False)
monkeypatch.delenv("BITRISE_BUILD_NUMBER", raising=False)
monkeypatch.delenv("AZURE_HTTP_USER_AGENT", raising=False)
monkeypatch.delenv("TRAVIS", raising=False)
monkeypatch.delenv("APPVEYOR", raising=False)
monkeypatch.delenv("GITHUB_ACTIONS", raising=False)
monkeypatch.delenv("GITLAB_CI", raising=False)
monkeypatch.delenv("CIRCLECI", raising=False)
monkeypatch.delenv("CIRRUS_CI", raising=False)
monkeypatch.delenv("CIBW_PLATFORM", raising=False)
@pytest.mark.parametrize("option_value", [None, "auto"])
def test_platform_unset_or_auto(monkeypatch, intercepted_build_args, option_value):
if option_value is None:
monkeypatch.delenv("CIBW_PLATFORM", raising=False)
else:
monkeypatch.setenv("CIBW_PLATFORM", option_value)

with pytest.raises(SystemExit) as exit:
main()
assert exit.value.code == 2
_, err = capsys.readouterr()
main()

assert "cibuildwheel: Unable to detect platform." in err
assert "cibuildwheel should run on your CI server" in err
options = intercepted_build_args.args[0]

# check that the platform was auto detected to build for the current system
if sys.platform.startswith("linux"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this feels like too much logic in a test. Any chance to simplify it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know what you mean, it's essentially just repeating the logic in the main program. But I would like to be sure that both auto and unspecified are doing the right thing. A little extra verbosity in the test seems like a small price to pay IMO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, why not try solving this on the level of parametrization? It'd be even clearer if it shows up in the test report.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even not parametrization but simeple dict could solve this:

platform_dkt = {"linux": "linux", "darwin": "macos", "win32": "windows"}

assert sys.platform in platform_dkt, f"Unknown platform: {sys.platform}"
assert platform_dkt[sys.platform] == options.platform

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I appreciate the focus on craft and code quality, there becomes a point when it's not worth our time to debate such minor points. Especially in tests, where the control flow complexity is so low, because each test is an isolated execution. All that matters in a test is the actions that are taken and the asserts that are reached - if these are clear to the reader, it's fine. If they grow to become unmaintainable, it's easy to rewrite them.

assert options.platform == "linux"
elif sys.platform == "darwin":
assert options.platform == "macos"
elif sys.platform == "win32":
assert options.platform == "windows"
else:
pytest.fail(f"Unknown platform: {sys.platform}")


def test_unknown_platform_on_ci(monkeypatch, capsys):
Expand Down