Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tables 3.8.0 #979

Merged
merged 31 commits into from Dec 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c8f1596
Preliminaries for release 3.8.0
FrancescAlted Dec 12, 2022
43fa687
Comment out data_files property
FrancescAlted Dec 12, 2022
b863d47
Enable package_data again
FrancescAlted Dec 12, 2022
84f2683
Remove the -rc1 suffix in version
FrancescAlted Dec 12, 2022
7ac5a2c
Remove binaries for win32
FrancescAlted Dec 13, 2022
0e39d40
[DOC] Add a bench on Blosc/Blosc2 performance
FrancescAlted Dec 9, 2022
0f42375
Improved section on Blosc2 and inkernel queries
FrancescAlted Dec 13, 2022
e1a5627
Add the blog on Blosc2 integration with PyTables
FrancescAlted Dec 13, 2022
1cd1ecd
Minor edit
FrancescAlted Dec 13, 2022
e44b3b2
Copy libs manually only when building the package by hand
FrancescAlted Dec 13, 2022
a9e9900
Update setup.py
FrancescAlted Dec 14, 2022
9e7e4b5
Do copy blosc2 shared when not building wheels
FrancescAlted Dec 14, 2022
5bf12de
Enable arm64 wheels on MacOSX
FrancescAlted Dec 14, 2022
51b50d4
Enable testing arm64 on Mac OSX
FrancescAlted Dec 14, 2022
8c25784
Update wheels.yml
FrancescAlted Dec 14, 2022
00a33b0
Disable a test that fails (sometimes) on Mac arm64
FrancescAlted Dec 14, 2022
71b8e55
support NUMEXPR_MAX_THREADS environment variable
ivanov Dec 14, 2022
bdd5000
Simplify build matrix
FrancescAlted Dec 15, 2022
b1f43a3
Update to latest C-Blosc sources
FrancescAlted Dec 15, 2022
1de83e7
Handle arm64 Mac wheels via CIBW_ARCHS_MACOS
FrancescAlted Dec 16, 2022
6f61759
Run osx arm64 and x86 builds in different containers to prevent binar…
matham Dec 20, 2022
4ae2495
Require python-blosc2 0.6.6, as this brings actual arm64 wheels
FrancescAlted Dec 20, 2022
9dd23b2
Use python -m pip
FrancescAlted Dec 21, 2022
36420aa
Remove a spurious char
FrancescAlted Dec 21, 2022
94b035e
Get macosx arm64 out of wheels, and test linux aarch64
FrancescAlted Dec 22, 2022
19fb804
Always copy the shared library to tables/ package
FrancescAlted Dec 22, 2022
6f8ffd6
Do a copy of the blosc2 libs
FrancescAlted Dec 22, 2022
8ea4b98
Silence a warning for recent NumPy
FrancescAlted Dec 22, 2022
a0fccf2
Silence more warnings for recent NumPy (1.24)
FrancescAlted Dec 22, 2022
5d02ad6
Merge branch 'master' into tables-3.8.0
FrancescAlted Dec 22, 2022
0f14177
Continue silencing warnings for recent NumPy (1.24)
FrancescAlted Dec 22, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/workflows/ci.yml
Expand Up @@ -30,8 +30,6 @@ jobs:
- name: Install dependencies
run: |
conda install setuptools pip wheel build packaging numpy cython bzip2 hdf5 lzo 'typing_extensions<4.2'
# We can use either the c-blosc2 package in conda/conda-forge or the official wheel (blosc2 in PyPI).
# conda install c-blosc2
pip install -r requirements.txt
# conda install sphinx sphinx_rtd_theme numpydoc ipython
- name: Source distribution
Expand Down
20 changes: 12 additions & 8 deletions .github/workflows/wheels.yml
Expand Up @@ -64,8 +64,9 @@ jobs:
include:
- os: macos-latest
arch: x86_64
- os: macos-latest
arch: arm64
# Still having issues here: https://github.com/PyTables/PyTables/pull/979
# - os: macos-latest
# arch: arm64

steps:
- uses: actions/checkout@v3
Expand Down Expand Up @@ -183,11 +184,14 @@ jobs:
os: [ 'ubuntu-latest', 'windows-latest', 'macos-latest' ]
python-version: ['3.8', '3.9', '3.10', '3.11']
arch: ['x64']
# include:
# This cannot be tested because NumPy does not distribute wheels for arm64
# - arch: arm64
# cibw_python: "cp*"
# os: macos-latest
include:
- arch: aarch64
cibw_python: "cp*"
os: ubuntu-latest
# Still having issues with wheels for Mac arm64
# - arch: arm64
# cibw_python: "cp*"
# os: macos-latest

steps:
- uses: actions/download-artifact@v3
Expand All @@ -202,7 +206,7 @@ jobs:

- name: Install tables on ${{ matrix.python-version }}
run: |
python -m pip install cython>=0.29.21 numpy>=1.19.0 numexpr>=2.6.2 packaging py-cpuinfo>=9.0.0 blosc2
python -m pip install cython>=0.29.21 numpy>=1.19.0 numexpr>=2.6.2 packaging py-cpuinfo>=9.0.0 blosc2~=2.0.0
python -m pip install --no-index --find-links wheelhouse/artifact/ tables

- name: Run tests on ${{ matrix.python-version }}
Expand Down
7 changes: 4 additions & 3 deletions doc/source/usersguide/installation.rst
Expand Up @@ -57,8 +57,9 @@ First, make sure that you have
* c-blosc_ >= 1.11.1 (sources are bundled with PyTables sources but the user can
use an external version of sources using the :envvar:`BLOSC_DIR` environment
variable or the `--blosc` flag of the :file:`setup.py`)
* blosc2_ >= 0.6.6 This is the Python wheel containing *both* the C-Blosc2 libs
and headers (>= 2.3.0), as well as the Python wrapper for Blosc2 (not used).
* python-blosc2_ ~= 2.0.0 This is the Python wheel containing *both* the C-Blosc2 libs
and headers (>= 2.6.0), as well as the Python wrapper for Blosc2 (not currently
used, but it might be in the future).

installed (for testing purposes, we are using HDF5_ 1.10.7, NumPy_ 1.23.2
and Numexpr_ 2.8.1 currently). If you don't, fetch and install them before
Expand All @@ -70,7 +71,7 @@ proceeding.
.. _Numexpr: http://code.google.com/p/numexpr
.. _Cython: http://www.cython.org
.. _c-blosc: https://github.com/Blosc/c-blosc
.. _blosc2_: https://github.com/Blosc/python-blosc2
.. _python-blosc2_: https://github.com/Blosc/python-blosc2

Compile and install these packages (but see :ref:`prerequisitesBinInst` for
instructions on how to install pre-compiled binaries if you are not willing
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Expand Up @@ -6,6 +6,6 @@ requires = [
"packaging",
"py-cpuinfo",
"Cython >=0.29.21",
"blosc2 >=0.6.6"
"blosc2 ~=2.0.0"
]
build-backend = "setuptools.build_meta"
2 changes: 1 addition & 1 deletion requirements.txt
Expand Up @@ -6,6 +6,6 @@ numpy>=1.19.0
numexpr>=2.6.2
# blosc2 wheel is actually only needed when compiling on conda envs.
# Otherwise, lib comes bundled in PyTables wheels (but it doesn't hurt either).
blosc2>=0.6.6
blosc2~=2.0.0
packaging
py-cpuinfo
4 changes: 1 addition & 3 deletions setup.py
Expand Up @@ -113,9 +113,7 @@ def get_blosc2_directories():
try:
import blosc2
except ModuleNotFoundError:
raise EnvironmentError(
"Cannot find neither the c-blosc2 package nor the "
"python-blosc2 wheel")
raise EnvironmentError("Cannot import the blosc2 requirement")
version = blosc2.__version__
basepath = Path(os.path.dirname(blosc2.__file__))
recinfo = basepath.parent / f'blosc2-{version}.dist-info' / 'RECORD'
Expand Down
2 changes: 1 addition & 1 deletion tables/__init__.py
Expand Up @@ -31,7 +31,7 @@
# Necessary imports to get versions stored on the cython extension
from .utilsextension import get_hdf5_version as _get_hdf5_version

__version__ = "3.7.1.dev0"
__version__ = "3.8.0"
"""The PyTables version number."""

hdf5_version = _get_hdf5_version()
Expand Down
14 changes: 7 additions & 7 deletions tables/tests/test_numpy.py
Expand Up @@ -548,24 +548,24 @@ def test04_setIndexNumPy(self):
# Modify row 1
# From PyTables 2.0 on, assignments to records can be done
# only as tuples (see http://projects.scipy.org/scipy/numpy/ticket/315)
# table[coords[0]] = ["aasa","x"]+[232]*12
# table[coords[0]] = ["aasa","x"]+[123]*12

n = len(Record.columns) - 2

table[coords[0]] = tuple(["aasa", "x"]+[232]*n) # XXX
table[coords[0]] = tuple(["aasa", "x"]+[123]*n) # XXX
# record = list(table[coords[0]])
record = table.read(coords[0], coords[0] + 1)
if common.verbose:
print("Original row:\n"
"['aasa', 'x', True, -24, 232, 232, 232, 232, 232L, "
"232, 232.0, 232.0, (232 + 0j), (232+0j), 232.0, "
"(232+0j)]\n")
"['aasa', 'x', True, 123, 123, 123, 123, 123, 123L, "
"123, 123.0, 123.0, (123 + 0j), (123+0j), 123.0, "
"(123+0j)]\n")
print("Read row:\n", record)
self.assertEqual(record['var1'], b'aasa')
self.assertEqual(record['var2'], b'x')
self.assertEqual(record['var3'], True)
self.assertEqual(record['var4'], -24)
self.assertEqual(record['var7'], 232)
self.assertEqual(record['var4'], 123)
self.assertEqual(record['var7'], 123)


# The declaration of the nested table:
Expand Down
2 changes: 1 addition & 1 deletion tables/tests/test_timetype.py
Expand Up @@ -348,7 +348,7 @@ def test00_CompareTable(self):

row = tbl.row
for i in range(nrows):
row['i8col'] = i
row['i8col'] = np.array(i).astype('i1')
row['t32col'] = i
j = i * 2
row['t64col'] = (j + 0.012, j+1+0.012)
Expand Down
4 changes: 2 additions & 2 deletions tables/tests/test_types.py
Expand Up @@ -51,7 +51,7 @@ def test00_range(self):
i = self.maxshort
rec['var1'] = '%04d' % (i)
rec['var2'] = i
rec['var3'] = i
rec['var3'] = np.array(i).astype('i2')
rec['var4'] = float(i)
rec['var5'] = float(i)
rec['var6'] = float(i)
Expand Down Expand Up @@ -83,7 +83,7 @@ def test01_type(self):
i = self.maxshort
rec['var1'] = '%04d' % (i)
rec['var2'] = i
rec['var3'] = i % self.maxshort
rec['var3'] = np.array(i % self.maxshort).astype('i2')
rec['var5'] = float(i)

with self.assertRaises(TypeError):
Expand Down