Telcon: 2015 06 04

Jump to bottom Edit New page

Todd Gamblin edited this page Jun 6, 2015 · 12 revisions

Participants

Alex Ballmer (ANL)
Mike Collette (LLNL)
Todd Gamblin (LLNL)
Jim Galarowicz (Krell)
Matt Legendre (LLNL)
David Shrader (LANL)

News

Mike Collette (LLNL) has Python 2.7 building with Spack on BG/Q!
- Mike has nightly builds set up for his packages
- Running in cron job

New Features

SPACK-69: Spack now generates build logs. Merged to develop.
- spack install now hides build output by default
- spack install -v will show detailed build output, including commands (formerly had to use spack debug for that)
- Commands and build output are stored in the build directory:
```
$stage_dir/
    archive-1.0.tar.gz
    archive-1.0/
        spack-build.out
        ... archive files ...
```
- On install this is copied to the .spack directory in the prefix:
```
$spack/opt/spack/.../$prefix/
    .spack/
        spec.yaml
        build.out
    ... installed files ...
```
- Log is generated whether you use install -v or not.
Log implementation was actually kind of interesting
- Wrote a context handler in Python that forks, executes a with block in another process, and redirects the output
  - See log.py#L48 for details
  - Interesting with hack uses Python's f_trace field in the Stack frame to set a trace function that skips the with block on the parent process.
- Matt Legendre and others like build output by default
  - David says most users are going to want quiet (Todd agrees)
  - make a config option to allow turning on verbose output by default.

Bug Fixes

Two fixes for bugs reported by Florent Provost (INRIA):
- Fixed #46: make(parallel=False) was broken (didn't disable parallel). Added tests to prevent regressions.
- Fixed #48: Mac OS X metadata confused Spack's exploding tarball countermeasures.

Contributions

Matt Legendre's PR for concretization preferences is being integrated.
- Also wrote documentation which will be contributed soon
PR #50: New version of Mitos from Alfredo Gimenez (LLNL/UC Davis)
- See also his MemAxes tool, which visualizes Mitos data.
Todd will integrate the FastMath PR from Saravan Pantham this week.
- Saravan is going to be working at VMWare
- We'll get his changes in before he heads out.
Todd will also integrate the outstanding BLAS/ATLAS/LAPACK PRs.
- Sorry these took so long!

Random thoughts

Need to write better developer docs for Spack.
Spack actually includes a fairly general framework for building command line tools.
- Same framework for commands used in both Spack and Cram.
- Could factor that out into a separate project for bootstrapping Python command line tool projects
- General pieces of llnl.tty:
  - llnl.tty: Bare bones console message printing, different levels of console logging.
  - llnl.tty.colify: Smart variable-width columns (like GNU ls)
  - llnl.tty.color: better syntax for ANSI color (kind of like printf, but with @{})
  - llnl.tty.log: Smart logging
- Spack script has infrastructure to run commands out of a prefix without need for installing:
  - Finds itself
  - Finds Python include path in $prefix/lib/spack
  - Invokes subcommands from $prefix/lib/spack/spack/cmd
    - File per command
    - Each subcommand is automatically added to spack command as a sub-parser.
- Not on the immediate priority list to factor this better, but could be useful.
  - Seem like common patterns -- pretty sure others have written similar things.
  - Maybe not as general and in one place?
  - Anyone interested?

General questions/discussion

Greg Lee has an RZ-only package that shouldn't go in the mainline
- Either Matt or Todd will integrate the external package repo support that Matt wrote
- Then RZ packages can go in a separate repository
Jim Galarowicz wants to build MRNet with the thread-safe option
- How to get this back in the mainline?
- 2 options:
  1. Just modify the configure line in the package and submit a pull request to the mainline
  2. Use variants, e.g.:
```
class Mrnet(Package):
    ...
    variant('thread-safe', default=False, "Build MRNet with thread safety")
    ...

    def install(self, spec, prefix):
        if '+thread-safe' in spec:
            configure(... thread safe args ...)
        else:
            configure(... without thread safe args ...)
```
  Now if your package needs the thread-safe feature, you can depend on it explicitly:
```
class OpenSpeedShop(Package):
     ...
     depends_on('mrnet+thread-safe')
     ...
```
- See Telcon: 2015-05-14 for details on variants and optional dependencies.
  - not in the docs yet, but there's at least an overview in the notes.

Clone this wiki locally