pip-compile should accept special files (pipes, fifo, pty, sockets, ...) as output files

[manuco]

When running pip-compile -o /dev/stdout, the command blocks indefinitely. It shouldn't and should write the requirement file in the given file, even if it's not a bare file.

Please note that pip-compile | <other program> doesn't work either (the other program doesn't receive anything).

Please also note that stderr is cluttered by some eventual warnings not parseable by pip and is so not usable as a viable content source.

Environment Versions

1. Linux VirtualBox 5.4.0-165-generic # 182-Ubuntu SMP Mon Oct 2 19:43:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

2. Python version: Python 3.8.10

3. pip version: pip 23.3.1

4. pip-tools version: pip-compile, version 7.3.0

Steps to replicate

1. git clone <your favourite "using setup.py" project> && cd in
2. pip-compile setup.py -o /dev/stdout
3. nothing happens

Expected result

output on stdout (not stderr)
warnings (if any) on stderr

Actual result

<nothing and process blocked>

Steps to replicate (advanced)

TTY 1 :

1. mkfifo /tmp/mylovelyfifo
2. cat /tmp/mylovelyfifo

TTY 2 :

1. git clone <your favourite "using setup.py" project> && cd in
2. pip-compile setup.py -o /tmp/mylovelyfifo

Expected result

output on TTY 1
warnings (if any) on TTY 2

Actual result

<nothing and process blocked>

[manuco]

The main problem is that pip compile try to read the output file in order to find previous requirement and update it. pip compile should ensure that previous requirements is only read from a real file (with stat.S_ISREG(mode)), but should not trust that reading from this file won't block.

For example, setting /dev/stdout as an output file will make pip compile try to read this file. And it won't generate any error because /dev/stdout as a link to your terminal (/dev/pts/*n*) is also readable (since /dev/pts/*n* is also stdin and stderr). But it won't give any data until you type on your keyboard (which is not what's expected).

[manuco]

Also the documentation should emphasize that - is a legit output filename redirecting it to stdout.

[chrysle]

Thanks for raising this issue, sounds like a reasonable request! I'll look into it.

[webknjaz]

@manuco are you willing to contribute tests, at least?

[manuco]

@webknjaz I'll have a look at it if it can help. Since those tests will be integrations tests (with the IOs of the OS), will they fit in the current tests ?

[manuco]

@webknjaz Is this test ok for you ?

[webknjaz]

Looks good, mostly.