7.2.0

7.2.0 (2023-12-18)

Features

• api: add ArrayValue.flatten method and operation (e6e995c)
• api: add ibis.range function for generating sequences (f5a0a5a)
• api: add timestamp range (c567fe0)
• base: add to_pandas method to BaseBackend (3d1cf66)
• clickhouse: implement array flatten support (d15c6e6)
• common: node.replace() now supports mappings for quick lookup-like substitutions (bbc93c7)
• common: add node.find_topmost() method to locate matching nodes without descending further to their children (15acf7d)
• common: allow matching on dictionaries in possibly nested patterns (1d314f7)
• common: expose node.__children__ property to access the flattened list of children of a node (2e91476)
• duckdb: add initial support for geospatial functions (65f496c)
• duckdb: add read_geo function (b19a8ce)
• duckdb: enforce aswkb for projections, coerce to geopandas (33327dc)
• duckdb: implement array flatten support (0a0eecc)
• exasol: add exasol backend (295903d)
• export: allow passing keyword arguments to PyArrow ParquetWriter and CSVWriter (40558fd)
• flink: implement nested schema support (057fabc)
• flink: implement windowed computations (256767f)
• geospatial: add support for GeoTransform on duckdb (ec533c1)
• geospatial: update read_geo to support url (3baf509)
• pandas/dask: implement flatten (c2e8d9d)
• polars: add streaming kwarg to to_pandas (703507f)
• polars: implement array flatten support (19b2aa0)
• pyspark: enable multiple values in .substitute (291a290)
• pyspark: implement array flatten support (5d1fadf)
• snowflake: implement array flatten support (d3c754f)
• snowflake: read_csv with https (72752eb)
• snowflake: support udf arguments for reading from staged files (529a3a2)
• snowflake: use upstream array_sort (9624341)
• sqlalchemy: support expressions in window bounds (5dbb3b1)
• trino: implement array flatten support (0d1faaa)

Bug Fixes

• api: avoid casting to bool for table.info() nullable column (3b3bd7b)
• bigquery: escape the schema (project ID) for BQ builtin UDFs (8096552)
• bigquery: fully qualified memtable names in compile (a81e432)
• clickhouse: use backwards compatible methods of getting query metadata (975556f)
• datafusion: bring back UDF registration (43084fa)
• datafusion: ensure that non-matching re_search calls return bool values when patterns do not match (088b027)
• datafusion: support computed group by when the aggregation is count distinct (18bdb7e)
• decompile: handle isin (6857751)
• deferred: don't pass expression in fstringified error message (724859d)
• deps: update dependency datafusion to v33 (57047a2)
• deps: update dependency sqlglot to v20 (13bc6e2)
• duckdb: ensure that already quoted identifiers are not erased (45ee391)
• duckdb: ensure that parameter names are unlikely to overlap with column names (d93dbe2)
• duckdb: gate geoalchemy import in duckdb geospatial (8f012c4)
• duckdb: render dates, times, timestamps and none literals correctly (5d8866a)
• duckdb: use functions for temporal literals (b1407f8)
• duckdb: use the UDF's signature instead of arguments' output type for generating a duckdb signature (233dce1)
• flink: add more test (33e1a31)
• flink: add os to the cache key (1b92b33)
• flink: add test cases for recreate table (1413de9)
• flink: customize the list of base idenitifers (0b5d343)
• flink: fix recreating table/view issue on flink backend (0c9791f)
• flink: implement TypeMapper and SchemaMapper for Flink backend (f983bfa)
• flink: use lazy import to prevent premature loading of pyflink during gen_matrix (d042402)
• geospatial: pretty print data in interactive mode (afb04ed)
• ir: ensure that join projection columns are all always nullable (f5f35c6)
• ir: handle renaming for scalar operations (6f77f17)
• ir: handle the case of non-overlapping data and add a test (1c9ae1b)
• ir: implicitly convert None literals with dt.Null type to the requested type during value coercion (d51ec4e)
• ir: merge window frames for bound analytic window functions with a subsequent over call (e12ce8d)
• ir: raise if Concrete.copy() receives unexpected arguments (442199a)
• memtable: ensure column names match provided data (faf99df)
• memtables: disallow duplicate column names when constructing memtables (4937b48)
• mssql: compute the length of strings correctly (64d2957)
• mssql: render dates, times and timestamps correctly (aca30e1)
• mysql: render dates and timestamps correctly (19e878c)
• oracle: ensure that .sql metadata results are in column-definition order (26a3c1f)
• oracle: render dates and timestamps correctly (66fbad6)
• pandas-format: convert map keys (bb92e9f)
• pandas: ensure that empty arrays unnest to nothing (fa9831f)
• pandas: fix integer wraparound when extracting epoch seconds (e98fa3c)
• pandas: handle non-nullable type mapping (c6a6c56)
• parse_sql: parse IN clauses (8b1f7b5)
• polars: handle new categorical types (5d6d6ae)
• polars: handle the case of an empty InValues list (b26aa55)
• polars: project first when creating computed grouping keys (7f9fdd4)
• postgres: render dates, times, timestamps and none literals correctly (a3c1c07)
• pyarrow: avoid catching ValueError and hiding legitimate failures (b7f650c)
• pyspark,polars: add packaging extra (bdde3a4)
• pyspark: custom format converter to handle pyspark timestamps (758ec25)
• snowflake: convert arrays, maps and structs using the base class implementation (f361891)
• snowflake: convert path to str when checking for a prefix (c5f884c)
• snowflake: ensure that empty arrays unnest to nothing (28c2498)
• snowflake: fix array printing by using a pyarrow extension type (7d8fe5a)
• snowflake: fix creating table in a different database (9b65b48)
• snowflake: fix quoting across all apis (7bf8e84)
• substitute: allow mappings with None keys (4b28ff1)

Documentation

• add exasol to the backend coverage app (3575858)
• arrays: document behavior of unnest in the presence of empty array rows (5526c40)
• backends: include docs for inherited members (c04bf67)
• blog-post: add blog post comparing ibis to pandas and dask (a7fd32b)
• blog-post: add blogpost ibis duckdb geospatial (def8031)
• blog-post: pydata performance part 2; polars and datafusion (36e1db5)
• blog: add dbt-ibis post (d73c156)
• blog: add pypi compiled file extension blog (751cfcf)
• build: allow building individual docs without rendering api docs first (529ee6c)
• build: turn off interactive mode before every example (502b88c)
• fix minor typo in sql.qmd (17aa929)
• fix typo in ir.Table docstring (e3b9611)
• fix typos (9a4d1f8)
• make minor edits to duckdb-geospatial post (2365e10)
• name: improve docstring of ibis.param API (2f9ec90)
• name: improve docstring of Value.name API (dd66af2)
• perf: use an unordered list instead of an ordered one (297be44)
• pypi-metadata-post: add Fortran pattern and fix regex (12058f2)
• remove confusing backend page (c1d19c7)
• replace deprecated relabels with renames (6bc9e15)
• sql: emphasize the need to close a raw_sql cursor only when using SELECT statements (74379a8)
• tests: add API docs for the testing base classes (173e9a9)
• tests: document class variables in BackendTest (e814c6b)

Refactors

• analysis: always merge frames during windowization (66fd69c)
• bigquery: move BigQueryType to use sqlglot for type parsing and generation (6e3219f)
• clickhouse: clean up session timezone handling (66220c7)
• clickhouse: use isoformat instead of manual specification (a3fac3e)
• common: consolidate the finder and replacer inputs for the various graph methods (a1881eb)
• common: remove traverse() function's filter argument since it can be expressed using the visitor (e4e2993)
• common: unify the node.find() and node.match() methods to transparently support types and patterns (3c14091)
• datafusion: simplify execute and to_pyarrow implementations (c572eab)
• duckdb: use pyarrow for all memtable registration (d6a2f09)
• formats: move the TableProxy object to formats from the operations (05964b1)
• pandas-format: move to classmethods to pickup super class behavior where possible (7bb0470)
• snowflake: use upstream map-from-arrays function instead of a custom UDF (318459c)
• tests: remove test rounding mixins (3b730d9)
• tests: remove UnorderedComparator class (ab0a8f6)

Performance

• common: improve the performance of replacing nodes by using a specialized node.__recreate__() method (f3da926)