build_ext + cythonize cannot handle unconventional package structure

[bsolomon1124]

Per yaml/pyyaml#407, setup.py build_ext seems to have trouble finding .pxd files when a certain un-conventional repository structure is used.

Fully reproducible steps (Python 3.8):

python -m venv venv
source ./venv/bin/activate
python -m pip install -U pip setuptools Cython
git clone -q --branch wheels git@github.com:bsolomon1124/pyyaml.git
cd pyyaml

# Broken version - why?
git checkout --detach 75ccf4c
USE_CYTHON=1 python setup.py build_ext --inplace

# Magically working version
git checkout --detach 3ad7bb0
USE_CYTHON=1 python setup.py build_ext --inplace

---

As of 75ccf4c63dd7b4463230f060cee652388ab88899, the repo looks like:

$ tree ext
ext
├── _yaml.h
├── _yaml.pxd
└── _yaml.pyx

With the extension invocation looking like:

extension = Extension(
    'yaml._yaml',
    sources=['ext/_yaml.%s' % 'pyx' if with_cython else 'c'],
    libraries=['yaml'],
)

Note that the pure-Python implementation is Python-major-version specific, in lib (Python2) versus lib3 (Python3).

Building with this structure fails:

$ USE_CYTHON=1 python setup.py build_ext --inplace
Compiling ext/_yaml.pyx because it changed.
[1/1] Cythonizing ext/_yaml.pyx
/Users/brad/Scripts/python/projects/3rdpty/pyyaml/venv/lib/python3.8/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/brad/Scripts/python/projects/3rdpty/pyyaml/ext/_yaml.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)

Error compiling Cython file:
------------------------------------------------------------
...
#        self.end_mark = end_mark
#        self.style = style

cdef class CParser:

    cdef yaml_parser_t parser
        ^
------------------------------------------------------------

ext/_yaml.pyx:249:9: 'yaml_parser_t' is not a type identifier

...

Traceback (most recent call last):
  File "setup.py", line 78, in <module>
    ext_modules=(cythonize(extension) if with_cython else [extension]),
  File "/Users/brad/Scripts/python/projects/3rdpty/pyyaml/venv/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1102, in cythonize
    cythonize_one(*args)
  File "/Users/brad/Scripts/python/projects/3rdpty/pyyaml/venv/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1225, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: ext/_yaml.pyx

There are several hundred of those "xxx is not a type identifier" errors indicating ext/_yaml.pxd is not found.

If, however, I make the simple changes of:

• Moving ext to yaml
• Adding yaml/__init__.pxd

Everything magically works. Those changes are shown in 3ad7bb06bf08432c97a49c0f2f2c964861508439.