Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ffi: define some cpython/unicodeobject bindings #1777

Merged
merged 4 commits into from
Aug 14, 2021
Merged

ffi: define some cpython/unicodeobject bindings #1777

merged 4 commits into from
Aug 14, 2021

Commits on Aug 14, 2021

  1. ffi: define some cpython/unicodeobject bindings 

    pyo3 doesn't currently define various Unicode bindings that allow the
retrieval of raw data from Python strings. Said bindings are a
prerequisite to possibly exposing this data in the Rust API (PyO3#1776).
Even if those high-level APIs never materialize, the FFI bindings are
necessary to enable consumers of the raw C API to utilize them.

This commit partially defines the FFI bindings as defined in
CPython's Include/cpython/unicodeobject.h file.

I used the latest CPython 3.9 Git commit for defining the order
of the symbols and the implementation of various inline preprocessor
macros. I tried to be as faithful as possible to the original
implementation, preserving intermediate `#define`s as inline functions.

Missing symbols have been annotated with `skipped` and symbols currently
defined in `src/ffi/unicodeobject.rs` have been annotated with `move`.

The `state` field of `PyASCIIObject` is a bitfield, which Rust doesn't
support. So we've provided accessor functions for retrieving these
fields' values. No accessor functions are present because you shouldn't
be touching these values from Rust code.

Tests of the bitfield APIs and macro implementations have been added.
    @indygreg
    indygreg committed Aug 14, 2021
    Configuration menu
    Copy the full SHA
    b8738b0 View commit details
    Browse the repository at this point in the history

  2. ffi: move limited API unicode symbols to cpython/unicodeobject.rs 

    All symbols which are canonically defined in cpython/unicodeobject.h
and had been defined in our unicodeobject.rs have been moved to our
corresponding cpython/unicodeobject.rs file.

This module is only present in non-limited build configurations, so
we were able to drop the cfg annotations as part of moving the
definitions.
    @indygreg
    indygreg committed Aug 14, 2021
    Configuration menu
    Copy the full SHA
    9690e9e View commit details
    Browse the repository at this point in the history

  3. ffi: PyPy and Python 3.10 attributes for unicodeobject

    @davidhewitt @indygreg
    davidhewitt authored and indygreg committed Aug 14, 2021
    Configuration menu
    Copy the full SHA
    974a782 View commit details
    Browse the repository at this point in the history

  4. string: enable use of PyUnicode_AsUTF8AndSize on all Python versions 

    This API is available in all supported Python versions and isn't
deprecated.
    @indygreg
    indygreg committed Aug 14, 2021
    Configuration menu
    Copy the full SHA
    807c8ee View commit details
    Browse the repository at this point in the history