Compiling C sources with absolute path bug

[pearu]

BPO	668662
Nosy	@mwhudson, @akuchling
Files	force_relpath_distutils_ccompiler.diff: Patch fixing 668662

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/akuchling'
closed_at = <Date 2003-02-26.19:02:12.000>
created_at = <Date 2003-01-15.18:59:58.000>
labels = ['library']
title = 'Compiling C sources with absolute path bug'
updated_at = <Date 2003-02-26.19:02:12.000>
user = 'https://bugs.python.org/pearu'

bugs.python.org fields:

activity = <Date 2003-02-26.19:02:12.000>
actor = 'akuchling'
assignee = 'akuchling'
closed = True
closed_date = None
closer = None
components = ['Distutils']
creation = <Date 2003-01-15.18:59:58.000>
creator = 'pearu'
dependencies = []
files = ['745']
hgrepos = []
issue_num = 668662
keywords = []
message_count = 6.0
messages = ['14081', '14082', '14083', '14084', '14085', '14086']
nosy_count = 3.0
nosy_names = ['mwh', 'akuchling', 'pearu']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue668662'
versions = ['Python 2.3']

[pearu]

Py-2.3 distutils is broken in the following situation
when building extension modules:
If a file in C sources list is given with absolute
path, say
/path/to/foo.c then Py-2.3 distutils tries to create
the corresponding
object file to /path/to/foo.o (earlier versions would
create it to, say, build/temp.linux-i686-2.2/foo.o).
This causes problems when

1. an user does not have permissions to write to
   /path/to/, e.g. in multiuser systems
2. an user builds extension modules, say, using
   different compilers or flags. Then /path/to/foo.o will
   be in a way for subsequent builds.
   IMHO, distutils should not create any files outside the
   given
   build directory, unless explicitely specified using
   --build-temp.

Fix:
The problematic code seems to be in the method
ccompiler.object_filenames in file distutils/ccompiler.py.
As a fix, I suggest using the following object name for
a C source if given with absolute path:
./build/temp.linux-i686-2.2/path/to/foo.o.

This can be achived by inserting the following line
just after
the line base, ext = os.path.splitext(src_name):

 base = base[os.path.isabs(base):]

Pearu

[mwhudson]

Logged In: YES
user_id=6656

I'm a little boggled by how distutils would get an absolute
path, but I'm going to assign this to amk anyway :)

[pearu]

Logged In: YES
user_id=88489

FYI, distutils does not make this absolute path, but one can
specify
absolute paths in setup.py files, for instance.
Also, tools that create extension modules (e.g. f2py), may
ship with C sources that are installed as data files and
compiled
for each generated extension module separately (possibly
with different
compilers and flags so that pre-compilation of such sources
would not be an option).

[mwhudson]

Logged In: YES
user_id=6656

Actually, I guessed after posting that something like that
could happen.

BTW, I strongly suspect this is related to the fix for this bug:

[ 599248 ] ext module generation problem

Can you come up with a patch for your suggested fix? I'm
sure Andrew would appreciate it :-)

[pearu]

Logged In: YES
user_id=88489

Attached patch maps the base of a source file with absolute
path (and
drive if present) to relative path before composing the
corresponding object file. This avoids creation of object
files out of the build directory.

Pearu

[akuchling]

Logged In: YES
user_id=11375

Checked in as patch 1.56 of ccompiler.py. I'll look into backporting to 2.2. Thanks!

[jaraco]

The patch applied for this issue has caused problems (namely in pypa/distutils#169). When invoking the compiler in an isolated fashion (not in the context of a distutils build), it causes the artifacts to be generated in the current directory, but if the sources are passed with a relative path, the artifacts are generated in the same directory. This divergence of behavior is surprising and confusing. I'm exploring deprecating this behavior to see if it can be reverted.