Minutes_2020_09_08

Public Numba Meeting: 2020-09-08

Attendees: Graham, Hameer, Juan, Stuart, Siu, Luk, Val, Todd

0. Feature Discussion

1. Changes to the release cycle. Longer development period? Longer release-candidate period?
   • Graham: other motivations for longer release candidate period being downstream projects/org that has more complicated CI system need longer time to test
   • Stuart: Reason for longer development
     • scale of tasks and increasing complexity of tasks
     • fewer low-hanging fruits
     • no time to address community contribution because of time loss to delay of development due to bugfixing last release
   • Hameer: stable > rushing things sooner
   • Luk: half way point release for bug fixes?
   • Val: addressing ^ on CI status.
     • CI being stabilized and improved for faster turnaround for bugfix release
   • Stuart: may need to change our Git model. Split tracks for bugfixes and features.
   • Siu: not so sure about half way releases, every quarter with a half way release would be about every 6 weeks again which is still quite quick.
   • Stuart: try out Quarterly schedule. Other compiler projects are half-yearly but we are not there yet (in terms of stability)
2. How to control and select unstable features?
   • There are increasing number of community contributions and core developer implemented features that are potentially unstable/experimental. How does Numba incorporate experimental features in a way that a user can opt-in or do something explicit to enable these features? Want to keep the core compiler stable. Has been spoken about in a number of dev meetings, there's clearly a need.
   • Siu: Thoughts around how Rust does this. They have two series of releases, nightly which contains experimental features enabled that are accessible via pragmas, and then stable release which don't even have the experimental/unusable features present in the binaries. Numba could do with something like this so that the core compiler remains stable but at the same time the experimental parts need to be present and running.
   • Stuart: has been thinking about this. Thinking about how to group things together and opt-in into things.
     • Classes of changes:
       • compiler passes. Easy to opt-in; turn on and off
       • specific changes to internal features. These may interfere with other features. Harder to isolate.
       • type system changes. Most dangerous and most different to accomodate.
     • Some suggestions are very niche. Existence of suggestion implies known use-cases but may not be interesting to most users.
     • Difficulties ...
   • Luk: secondary repository where Numba-extras perhaps lives. This could be a testing/migration ground for additive features that could eventually go into the main repo. What about doing things around the edges of e.g. type inference, is this easier?
     • Subtyping is one of these things.
     • Could case by case work? Additional decorators?
   • Graham: Could composable decorators help?
   • Siu: A lot of solutions require a lot more improvement to the core compiler in terms of compilation. Particularly in terms of "context", flags/customisation etc. Does this make it easier to do?
   • Stuart: Does this indicate Numba is not ready for all these things? That we can't come up with a system hints at this.
   • Siu: Luk's idea about the extras repo does provide that opt-in capability which is very explicit.
   • Luk: Another rule could be unstable features have to go in early so there's more testing time?
   • Siu: That's like Rust model in that it can be turned off to stable release. Other concern is that experimental features become standard and users end up relying on APIs etc.
   • Hameer: Release multiple versions of packages. One version which is stable another with all the experimental stuff. Another option is envvars as global config, it failed in NumPy for a load of reasons, recommends avoiding.
   • Graham: Continuing with Luk's idea of separate repo would also enforce only things that can go in as being things that can technically be done. i.e. if there's no extension API, can't be done.
   • Stuart: external repository enforces the boundary and highlights the holes in our extension system. Avoids the difficulty to having conflicting options.
   • Siu: So this implies the Numba Devs are signing up to manage the numba-extras repo.
   • Stu: Thinks this is fine, if it's required that it's a no-C-code pure python package then its just shipping a basic python package which is quite easy. Does it make it easier for community contributors etc?
   • Val: similar to 0MQ core in C and subprojects that add bindings. Could well make it easier for the community to help out and take on maintenance etc.

Quick Q&A

• Hameer asked about trouble building on OSX. Compiler complain about not supported OpenMP. This is due to Numba needs a newer compiler that is not provided by XCode. There are flags to make XCode compiler work.

Couldn't get to. Perhaps next time

3. Cache njit function at import or install

1. New Issues

• #6218 - Cannot unify array(int64, 2d, F) and array(float64, 2d, C) for 'X.2', np.linalg.inv() even with np.ascontiguousarray() .astype(np.float64) everywhere
• #6217 - fastmath=True gives invalid np.isfinite() value
  • should we warn or detect isfinite() in fastmath=True?
  • 2 ppl affected for many years. Leave it for now
• #6216 - Vulnerable Regular Expression in nrtopt
  • very unlikely security exploit
• #6214 - Suggested Numpy features to implement in Numba
• #6211 - numpy.roots type error - casting from complex128 to float64
  • known limitation of numba
  • nothing we can do
• #6207 - jit compilation no longer works on google colab after v0.51.1
• #6205 - Release 0.51.2 checklist
• #6199 - @guvectorize(cache=True) calls obsolete version of DUFunc
  • known behavior but confusing to users
• #6197 - [ROCm][DOC]: Need an ROCm Python API reference
• #6196 - ROCm: 'gfx906' is not a recognized processor for this target (ignoring processor)

Closed Issues

• #6219 - Numba support for Dictionary of dictionary
• #6210 - I am getting typing error.
• #6198 - String formatting support for dict keys

2. New PRs

• #6221 - Fix C99 (and other) maths calls.
• #6215 - Attempt immutable ir
• #6213 - Fix #6177. Remove AOT dependency on the Numba package
• #6212 - Fix for objmode recompilation issue
• #6209 - Fix deferred type for structref
• #6208 - [WIP] Do not try to terminate TBB if fork was called not from main thread
• #6206 - Build all extension modules with -g -Wall -Werror on Linux x86, provide -O0 flag option
  • discussed on turning debug off by default
• #6204 - Fix unicode kind selection for getitem.
• #6203 - Make find_callname only lookup functions that are likely part of NumPy.
• #6202 - Speed up str(int)

Closed PRs

• #6220 - Try this.
• #6201 - Update CHANGE_LOG for 0.51.2
• #6200 - Workaround #5973

3. Next Release: Version 0.52.0, RC=?, Final ?

• Requests for 0.52
  • Fast(er) typed.List/typed.Dict? Doesn't have to be in 0.52, next 3-5 months is fine.

4. Upcoming tasks