Matthew Krafczyk's WG ideas

Ideas for Spack Environments WG by Matthew Krafczyk

Portable And Reproducible Packaging

A use case I would love to develop would be the creation of a kind of portable spack package. This would essentially just be a tarball which would contain a directory with spack inside it, a directory containing tarballs of the necessary source code, and any configuration files necessary properly configured to point spack to those tarballs. It would then also contain some scripts for building all the necessary software and a script for making available all packages in the shell environment. This would all be configured to work correctly on any posix compliant machine (or as close to that as possible). It would enable a unified packaging and installing system which would be useful for researchers and software developers alike.

To facilitate this, I envision a command, spack export <spec list> or something like that which would copy itself, download and copy all necessary tarballs, and create and copy all necessary scripts and config files into a directory which is then tarred. This could then be untarred on another machine and all the mechanisms discussed above would be available to the user without any work at all.

Additional issues

1. I've got the feeling that the current spec language is too limited. It's fine for a kind of abbreviated spec, but once build packages are correctly handled, there will be no way to specify the build environments of each necessary package in a spec. I propose an expansion of spec grammar to include things like () or special dependencies indicating run time or build only dependencies. Something like this A^B^(C^*D^E). Here A depends on B, and C which was built with a build dependency D and run dependency E. (-MK)

2. Separation of builtin repo from spack itself? Please excuse this question if it's already been discussed. I've found that many commits in the github repo only upgrade packages and do not change the functionality of spack itself. It is annoying to have to rebase so frequently, so I'd like to have the builtin repository somehow separated from the spack logic. This should make logic improvements more clear and allow users to get package version updates without the possibility of changing spack logic. (-MK)