Telcon: 2017 04 06

Participants

1. Todd Gamblin (LLNL)
2. Ben Boeckel (Kitware)
3. Kiel Friedt (LLNL)
4. Brian Van Essen (LLNL)
5. Patrick Gartung (Fermi)
6. Peter Scheibel (LLNL)
7. Steve (NERSC)
8. Mario Melara (NERSC)

News

1. LLNL Spack Tutorial this morning (3 hr rehash of SC16 -- but on LLNL machines, not AWS)

   • Good turnout! 46 attendees (26 in person, 20 on WebEx)

2. Plan to reconvene environments working group soon -- as we now have people who can work on this.

   • Would like binary packaging as well
   • envs + binary packaging would make tutorial much more smooth.
   • potentially useful for binary packaging:
     • Patrick Gartung looked into python tool to relocate mac software on Linux machines
       • machotools - Mach-O binary format tools in python
     • has a python script that can do this
     • patchelf + install_name_tool working well for CERN so far

3. Python 3 compatibility merged; Spack now works with Python 3!

4. Merging more PRs, starting to give more people authority to review core PRs

   • @scheibelp reviewed and merged https://github.com/LLNL/spack/pull/2789

5. New concretizer still being worked on.

   • Will probably do a 0.11 release soon instead of going straight to 1.0
   • much cleanup in Python3 support helping with concretizer implementation.

6. Other interesting new features:

   • better URL parsing: https://github.com/LLNL/spack/pull/2972
   • MakefilePackage auto-create: https://github.com/LLNL/spack/pull/3710
   • Better tests for spack versions and web spidering: https://github.com/LLNL/spack/pull/3654
   • Extendable Perl: https://github.com/LLNL/spack/pull/3614
   • OpenFOAM: https://github.com/LLNL/spack/pull/3528
     • more coverage (https://github.com/LLNL/spack/pull/3669)
     • packages using openfoam:
       • swak4foam https://github.com/LLNL/spack/pull/3650
       • community adios/openfoam, isoAdvector https://github.com/LLNL/spack/pull/3726

7. As always, check out the Pulse for recent PRs and issues: https://github.com/LLNL/spack/pulse

Discussion

• Brian Van Essen (LLNL):

  • Started to use Spack to package ML tools like LBANN (https://github.com/LLNL/lbann)
  • Some feedback on installing Spack packages and building stacks:
    • Really need to have 1st-class environments like conda for reproducing a stack
    • Spack currently requires a lot of command to build up a particular stack
      • See here for install process:
        • https://github.com/ECP-CANDLE/Benchmarks/blob/master/Pilot2/P2B1/INSTALL.md
        • https://github.com/ECP-CANDLE/Benchmarks/blob/master/README.setup.linux
    • Need a file format for defining an installed ecosystem
      • Todd: this is the goal of the env working group
        • ability to create, use, and share environments with software stacks
    • Would like ability to continue using what is installed
  • Todd: plan is to handle this stuff with environments and

• Mario and Steve from NERSC:

  • currently two types of spec formats:
    • {HASH}: just the hash
    • $/: the hash with / before it (like a user would type it)
  • architecture spec format was not consistent
    • attempting a refactor of that.
    • issues with putting a space before or after the arch specifier
  • Todd: let's try to come up with some consistent arch formatting in the PR

• Steve:

  • thinking about what might make it easier to write package files
  • Todd: better error output when builds go wrong

• Peter Scheibel:

  • Question came up while implementing #2548 (https://github.com/LLNL/spack/pull/2548)
    • What if we allow packages to specify how dependents should depend on it?
    • sensible defaults for whether something is a build, link, or run dependency might be helpful
  • Peter: may need a way to specify that a particular build dependency needs to be "merged" with a transitive run dependency of other build deps. Packager could specify this.
    • Todd & Ben:
      • This is a SAT solve problem (something we're looking at in the new concretizer)
      • packager shouldn't need to know about transitive run dependencies of build dependencies (let's call then build/run* deps).
        • Spack should work out that build/run* deps need to be concretized along with direct build deps.
        • transitive build dependencies don't need to be merged into a build env, but build/run* deps do.
    • Ben points out also that, in the merge, we should bias for the dep version of the root package.
      • don't force root to change based on build dependencies -- rebuild a build dep if necessary and stick with preferred thing for the root if we can do so.
    • lots of constraints!