Transform signature

[davej]

Following on from talk in #810 about properties that describe transforms to be performed.

What do you think of having a property on the asset that tracks transforms by building a transform signature?

I used this in Pingy to avoid doing unneeded compilation, minification, autoprefixing and templating. The basic formula is that if the input file(s) is/are the same, the transform signature is the same and the existing output file is the same then I avoid the work and just leave the existing version of the output file in place. I was able to achieve ~1 second build times on a project that was taking nearly a minute without this feature.

My transform signatures were strings that looked like this: "node-sass@4.5.2::autoprefixer@7.1.6=last 2 versions::map::minify@4.18.1", but there is a probably a nicer way to do it if given some thought. The entire data structure for an output file in Pingy looked like below, but I am only suggesting that assetgraph would keep a transform signature.

{
  "type": "node-sass@4.5.2::autoprefixer@7.1.6=last 2 versions::map::minify@4.18.1",
  "inputSha": [
    {
      "file": "styles/main.scss",
      "sha": "2f75a6fc6a520b1e27cfe4542b6634b7d444855f"
    },
    {
      "file": "styles/_hero.scss",
      "sha": "1b1e462f8a8f23ab22d6ac2899b19828abd36518"
    },
    {
      "file": "bower_components/neat/core/neat/mixins/_grid-column.scss",
      "sha": "a27f46d3e2336edf125621edbeaf8e060b3f06e7"
    },
    {
      "file": "styles/_videos.scss",
      "sha": "391a23a973da79480ca75610e5929e50846ca700"
    },
    // ...
  ],
  "outputSha": "bd4eeae3da70ceb85c8af1383cb840354163e2d6",
  "input": "styles/main.scss",
  "output": [
    "styles/main.css.map",
    "styles/main.css"
  ]
}

[papandreou]

Thanks for writing up the suggestion!

I've thought about a very similar idea for some time, but haven't had the time to explore it yet. Essentially every asset in the graph would carry some representation of its "lineage" that would both be cheap to compute (ie. shouldn't require computing a hash after each step), and capture everything that was done to it, including the involved module versions etc.

I was mostly thinking about incremental builds, where leaving a serialized version of that structure would make it possible to use a folder containing a previous build as a "cache" when rebuilding -- but it would probably work fine for "watch mode" like tools as well.

I have a hunch that it would be possible to keep the core AssetGraph class ignorant about this and implement the tracing engine by wrapping a Proxy around an AssetGraph instance for the purpose of observing the assets and relations touched by a given transform.

[davej]

Yes, that's exactly how pingy export currently works. I want to replicate this feature in the assetgraph version of pingy export. I think I can probably do it completely outside of assetgraph but if there is a way that makes sense to add some of the functionality to assetgraph directly then I would be interested in exploring that.

[papandreou]

@davej, I think getting it to work in the general case is an "XL task", so it might take some time to get there. I think it would be a great pilot project to do it externally for pingy export. Then we'd also have a better idea about the requirements. I'd be happy to help, of course.