Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

MNT: Update expat 2.2.9 #10038

Merged
merged 3 commits into from Mar 19, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGES.rst
Expand Up @@ -305,6 +305,8 @@ Other Changes and Additions
manage the version number, and adding a ``pyproject.toml`` to opt in to
isolated builds as described in PEP 517/518. [#9726]

- Bundled ``expat`` is updated to version 2.2.9. [#10038]

4.0.1 (unreleased)
==================

Expand Down
3 changes: 1 addition & 2 deletions astropy/utils/xml/setup_package.py
Expand Up @@ -23,8 +23,7 @@ def get_extensions(build_type='release'):
EXPAT_DIR = 'cextern/expat/lib'
cfg['sources'].extend([
join(EXPAT_DIR, fn) for fn in
["xmlparse.c", "xmlrole.c", "xmltok.c", "xmltok_impl.c",
"loadlibrary.c"]])
["xmlparse.c", "xmlrole.c", "xmltok.c", "xmltok_impl.c"]])
cfg['include_dirs'].extend([XML_DIR, EXPAT_DIR])
if sys.platform.startswith('linux'):
# This is to ensure we only export the Python entry point
Expand Down
44 changes: 27 additions & 17 deletions astropy/utils/xml/src/expat_config.h
@@ -1,18 +1,18 @@
/* expat_config.h. Generated from expat_config.h.in by configure. */
/* expat_config.h.in. Generated from configure.ac by autoheader. */

/* 1234 = LIL_ENDIAN, 4321 = BIGENDIAN */
/* #define BYTEORDER 1234 */
/* Define if building universal (internal helper macro) */
/* #undef AC_APPLE_UNIVERSAL_BUILD */

/* 1234 = LILENDIAN, 4321 = BIGENDIAN */
#define BYTEORDER 1234

/* Define to 1 if you have the `arc4random' function. */
/* #undef HAVE_ARC4RANDOM */

/* Define to 1 if you have the `arc4random_buf' function. */
/* #undef HAVE_ARC4RANDOM_BUF */

/* Define to 1 if you have the `bcopy' function. */
#define HAVE_BCOPY 1

/* Define to 1 if you have the <dlfcn.h> header file. */
#define HAVE_DLFCN_H 1

Expand All @@ -23,17 +23,14 @@
#define HAVE_GETPAGESIZE 1

/* Define to 1 if you have the `getrandom' function. */
/* #define HAVE_GETRANDOM 1 */
/* #undef HAVE_GETRANDOM */

/* Define to 1 if you have the <inttypes.h> header file. */
#define HAVE_INTTYPES_H 1

/* Define to 1 if you have the `bsd' library (-lbsd). */
/* #undef HAVE_LIBBSD */

/* Define to 1 if you have the `memmove' function. */
#define HAVE_MEMMOVE 1

/* Define to 1 if you have the <memory.h> header file. */
#define HAVE_MEMORY_H 1

Expand All @@ -53,7 +50,7 @@
#define HAVE_STRING_H 1

/* Define to 1 if you have `syscall' and `SYS_getrandom'. */
#undef HAVE_SYSCALL_GETRANDOM
/* #undef HAVE_SYSCALL_GETRANDOM */

/* Define to 1 if you have the <sys/param.h> header file. */
#define HAVE_SYS_PARAM_H 1
Expand All @@ -65,7 +62,7 @@
#define HAVE_SYS_TYPES_H 1

/* Define to 1 if you have the <unistd.h> header file. */
/* #define HAVE_UNISTD_H 1 */
#define HAVE_UNISTD_H 1

/* Define to the sub-directory where libtool stores uninstalled libraries. */
#define LT_OBJDIR ".libs/"
Expand All @@ -80,7 +77,7 @@
#define PACKAGE_NAME "expat"

/* Define to the full name and version of this package. */
#define PACKAGE_STRING "expat 2.2.6"
#define PACKAGE_STRING "expat 2.2.9"

/* Define to the one symbol short name of this package. */
#define PACKAGE_TARNAME "expat"
Expand All @@ -89,16 +86,29 @@
#define PACKAGE_URL ""

/* Define to the version of this package. */
#define PACKAGE_VERSION "2.2.6"
#define PACKAGE_VERSION "2.2.9"

/* Define to 1 if you have the ANSI C header files. */
#define STDC_HEADERS 1

/* Version number of package */
#define VERSION "2.2.6"

/* whether byteorder is bigendian */
/* #undef WORDS_BIGENDIAN */
#define VERSION "2.2.9"

/* Define WORDS_BIGENDIAN to 1 if your processor stores words with the most
significant byte first (like Motorola and SPARC, unlike Intel). */
#if defined AC_APPLE_UNIVERSAL_BUILD
# if defined __BIG_ENDIAN__
# define WORDS_BIGENDIAN 1
# endif
#else
# ifndef WORDS_BIGENDIAN
/* # undef WORDS_BIGENDIAN */
# endif
#endif

/* Define to allow retrieving the byte offsets for attribute names and values.
*/
/* #undef XML_ATTR_INFO */

/* Define to specify how much context to retain around the current parse
point. */
Expand Down
41 changes: 41 additions & 0 deletions cextern/expat/.gitignore
@@ -0,0 +1,41 @@
/autom4te.cache/
m4/
CMakeFiles/
Testing/
aclocal.m4
CMakeCache.txt
cmake_install.cmake
CTestTestfile.cmake
install_manifest.txt
Makefile
.deps
Makefile.in
.libs
*.la
configure
config.cache
config.log
config.status
expat_config.h.in
expat_config.h
libtool
expat.ncb
expat.opt
.project
expat.pc
*.gcda
*.gcno
*.gcov
*.nccout
*.expand
/callgraph.svg
/libexpat.so.*
/run.sh
build__R*
coverage__R*
source__R*
/expat-*.tar.bz2
/expat-*.tar.bz2.asc
/stamp-h1
/libexpat*.dll
/changelog
156 changes: 156 additions & 0 deletions cextern/expat/Changes
Expand Up @@ -2,6 +2,162 @@ NOTE: We are looking for help with a few things:
https://github.com/libexpat/libexpat/labels/help%20wanted
If you can help, please get in touch. Thanks!

Release 2.2.9 Wed Septemper 25 2019
Other changes:
examples: Drop executable bits from elements.c
#349 Windows: Change the name of the Windows DLLs from expat*.dll
to libexpat*.dll once more (regression from 2.2.8, first
fixed in 1.95.3, issue #61 on SourceForge today,
was issue #432456 back then); needs a fix due
case-insensitive file systems on Windows and the fact that
Perl's XML::Parser::Expat compiles into Expat.dll.
#347 Windows: Only define _CRT_RAND_S if not defined
Version info bumped from 7:10:6 to 7:11:6

Special thanks to:
Ben Wagner

Release 2.2.8 Fri Septemper 13 2019
Security fixes:
#317 #318 CVE-2019-15903 -- Fix heap overflow triggered by
XML_GetCurrentLineNumber (or XML_GetCurrentColumnNumber),
and deny internal entities closing the doctype;
fixed in commit c20b758c332d9a13afbbb276d30db1d183a85d43

Bug fixes:
#240 Fix cases where XML_StopParser did not have any effect
when called from inside of an end element handler
#341 xmlwf: Fix exit code for operation without "-d DIRECTORY";
previously, only "-d DIRECTORY" would give you a proper
exit code:
# xmlwf -d . <<<'<not well-formed>' 2>/dev/null ; echo $?
2
# xmlwf <<<'<not well-formed>' 2>/dev/null ; echo $?
0
Now both cases return exit code 2.

Other changes:
#299 #302 Windows: Replace LoadLibrary hack to access
unofficial API function SystemFunction036 (RtlGenRandom)
by using official API function rand_s (needs WinXP+)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also interesting since I had to include loadlibrary.c for windows:
ef5b44e

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Does that mean I can remove what you added to setup_package.py?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think so.

#325 Windows: Drop support for Visual Studio <=7.1/2003
and document supported compilers in README.md
#286 Windows: Remove COM code from xmlwf; in case it turns
out needed later, there will be a dedicated repository
below https://github.com/libexpat/ for that code
#322 Windows: Remove explicit MSVC solution and project files.
You can generate Visual Studio solution files through
CMake, e.g.: cmake -G"Visual Studio 15 2017" .
#338 xmlwf: Make "xmlwf -h" help output more friendly
#339 examples: Improve elements.c
#244 #264 Autotools: Add argument --enable-xml-attr-info
#239 #301 Autotools: Add arguments
--with-getrandom
--without-getrandom
--with-sys-getrandom
--without-sys-getrandom
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using these new options with ./configure would simplify the update process.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which one is the correct option? 馃槅

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think both --without-getrandom and --without-sys-getrandom, you can check the diff on expat_config.h.

#312 #343 Autotools: Fix linking issues with "./configure LD=clang"
Autotools: Fix "make run-xmltest" for out-of-source builds
#329 #336 CMake: Pull all options from Expat <=2.2.7 into namespace
prefix EXPAT_ with the exception of DOCBOOK_TO_MAN:
- BUILD_doc -> EXPAT_BUILD_DOCS (plural)
- BUILD_examples -> EXPAT_BUILD_EXAMPLES
- BUILD_shared -> EXPAT_SHARED_LIBS
- BUILD_tests -> EXPAT_BUILD_TESTS
- BUILD_tools -> EXPAT_BUILD_TOOLS
- DOCBOOK_TO_MAN -> DOCBOOK_TO_MAN (unchanged)
- INSTALL -> EXPAT_ENABLE_INSTALL
- MSVC_USE_STATIC_CRT -> EXPAT_MSVC_STATIC_CRT
- USE_libbsd -> EXPAT_WITH_LIBBSD
- WARNINGS_AS_ERRORS -> EXPAT_WARNINGS_AS_ERRORS
- XML_CONTEXT_BYTES -> EXPAT_CONTEXT_BYTES
- XML_DEV_URANDOM -> EXPAT_DEV_URANDOM
- XML_DTD -> EXPAT_DTD
- XML_NS -> EXPAT_NS
- XML_UNICODE -> EXPAT_CHAR_TYPE=ushort (!)
- XML_UNICODE_WCHAR_T -> EXPAT_CHAR_TYPE=wchar_t (!)
#244 #264 CMake: Add argument -DEXPAT_ATTR_INFO=(ON|OFF),
default OFF
#326 CMake: Add argument -DEXPAT_LARGE_SIZE=(ON|OFF),
default OFF
#328 CMake: Add argument -DEXPAT_MIN_SIZE=(ON|OFF),
default OFF
#239 #277 CMake: Add arguments
-DEXPAT_WITH_GETRANDOM=(ON|OFF|AUTO), default AUTO
-DEXPAT_WITH_SYS_GETRANDOM=(ON|OFF|AUTO), default AUTO
#326 CMake: Install expat_config.h to include directory
#326 CMake: Generate and install configuration files for
future find_package(expat [..] CONFIG [..])
CMake: Now produces a summary of applied configuration
CMake: Require C++ compiler only when tests are enabled
#330 CMake: Fix compilation for 16bit character types,
i.e. ex -DXML_UNICODE=ON (and ex -DXML_UNICODE_WCHAR_T=ON)
#265 CMake: Fix linking with MinGW
#330 CMake: Add full support for MinGW; to enable, use
-DCMAKE_TOOLCHAIN_FILE=[expat]/cmake/mingw-toolchain.cmake
#330 CMake: Port "make run-xmltest" from GNU Autotools to CMake
#316 CMake: Windows: Make binary postfix match MSVC
Old: expat[d].lib
New: expat[w][d][MD|MT].lib
CMake: Migrate files from Windows to Unix line endings
#308 CMake: Integrate OSS-Fuzz fuzzers, option
-DEXPAT_BUILD_FUZZERS=(ON|OFF), default OFF
#14 Drop an OpenVMS support leftover
#235 #268 ..
#270 #310 ..
#313 #331 #333 Address compiler warnings
#282 #283 ..
#284 #285 Address cppcheck warnings
#294 #295 Address Clang Static Analyzer warnings
#24 #293 Mass-apply clang-format 9 (and ensure conformance during CI)
Version info bumped from 7:9:6 to 7:10:6

Special thanks to:
David Loffredo
Joonun Jang
Khajapasha Mohammed
Kishore Kunche
Marco Maggi
Mitch Phillips
Rolf Ade
xantares
Zhongyuan Zhou

Release 2.2.7 Wed June 19 2019
Security fixes:
#186 #262 CVE-2018-20843 -- Fix extraction of namespace prefixes from
XML names; XML names with multiple colons could end up in
the wrong namespace, and take a high amount of RAM and CPU
resources while processing, opening the door to
use for denial-of-service attacks

Other changes:
#195 #197 Autotools/CMake: Utilize -fvisibility=hidden to stop
exporting non-API symbols
#227 Autotools: Add --without-examples and --without-tests
#228 Autotools: Modernize configure.ac
#245 #246 Autotools: Fix check for -fvisibility=hidden for Clang
#247 #248 Autotools: Fix compilation for lack of docbook2x-man
#236 #258 Autotools: Produce .tar.{gz,lz,xz} release archives
#212 CMake: Make libdir of pkgconfig expat.pc support multilib
#158 #263 CMake: Build man page in PROJECT_BINARY_DIR not _SOURCE_DIR
#219 Remove fallback to bcopy, assume that memmove(3) exists
#257 Use portable "/usr/bin/env bash" shebang (e.g. for OpenBSD)
#243 Windows: Fix syntax of .def module definition files
Version info bumped from 7:8:6 to 7:9:6

Special thanks to:
Benjamin Peterson
Caol谩n McNamara
Hanno B枚ck
KangLin
Kishore Kunche
Marco Maggi
Rhodri James
Sebastian Dr枚ge
userwithuid
Yury Gribov

Release 2.2.6 Sun August 12 2018
Bug fixes:
#170 #206 Avoid doing arithmetic with NULL pointers in XML_GetBuffer
Expand Down