Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable CI platforms to build Linux, Windows and OS X releases #63

Merged
merged 104 commits into from
Apr 13, 2018
Merged
Show file tree
Hide file tree
Changes from 101 commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
5f4b5c6
Seeing what happens if I commit these three rough CI configs
JamesCropcho Mar 1, 2018
4f81b44
Trigger build
JamesCropcho Mar 1, 2018
d4a3457
Created a `yarn.lock`
JamesCropcho Mar 1, 2018
06fd74b
added `yarn-error.log` to .gitignore
JamesCropcho Mar 3, 2018
2997004
added `cross-env`; added `pack` script
JamesCropcho Mar 3, 2018
aedf597
attempting to move release files on Circle to a new directory
JamesCropcho Mar 3, 2018
b895334
Attempting pack on CircleCI
JamesCropcho Mar 3, 2018
0e5d8dd
Making .travis.yml more like Falcon's
JamesCropcho Mar 3, 2018
5dec6c4
Using Yarn instead of NPM in `package.json`
JamesCropcho Mar 3, 2018
5d6da67
making Electron-Builder a 'dev' dependency
JamesCropcho Mar 3, 2018
3827fb2
Appeasing Yarn with a new Node.js v6x.
JamesCropcho Mar 3, 2018
eb57c14
storing (hopefully) /dist artifacts
JamesCropcho Mar 3, 2018
2d0e1f9
Really this time, using node v6.13 and not v6.10
JamesCropcho Mar 3, 2018
e2fe196
Minor-upgrade Node.js images; only `pack` on v8.x, and once
JamesCropcho Mar 3, 2018
83afcc3
fixing a typo?
JamesCropcho Mar 3, 2018
7d2267d
Fixed typo?
JamesCropcho Mar 3, 2018
29c0c4a
Moving `steps` more smartly
JamesCropcho Mar 3, 2018
0ac0e72
using proper Workflow (CircleCI)
JamesCropcho Mar 3, 2018
23abdeb
I can't really work effectively with nondeterminstic timeout failures…
JamesCropcho Mar 3, 2018
5c856f2
Need to `checkout`?
JamesCropcho Mar 3, 2018
87e571e
I'm restoring the Worflow for CircleCI since I'm not working on it fo…
JamesCropcho Mar 3, 2018
a011055
Wrong place for CircleCI configuration
JamesCropcho Mar 3, 2018
2c27132
Trigger build
JamesCropcho Mar 3, 2018
bee2f94
Fixes for Travic CI configuration
JamesCropcho Mar 3, 2018
2b83e69
Commenting out Falcon-esque AppVeyor stuff
JamesCropcho Mar 3, 2018
1a16385
Hoping AppVeyor keeps the dist files
JamesCropcho Mar 3, 2018
0563675
better caching?
JamesCropcho Mar 3, 2018
e41a435
Yarn, we need not force you, yes, I hope?
JamesCropcho Mar 3, 2018
1dd78b5
Even more caching maybe?
JamesCropcho Mar 3, 2018
8c05123
Trigger build
JamesCropcho Mar 3, 2018
e132f65
Attempting restoration of the 7zip thing at @n-riesco's suggestion
JamesCropcho Mar 3, 2018
8fad4d7
Made my `package.json` more like Falcon's
JamesCropcho Mar 5, 2018
2bfbf5d
Fixed CI-specific config to match the new `package.json`
JamesCropcho Mar 5, 2018
51e7ebd
Trigger build
JamesCropcho Mar 5, 2018
be2e085
Trigger build
JamesCropcho Mar 5, 2018
3405a96
Trigger build
JamesCropcho Mar 5, 2018
e4287dd
Attempting "pro-ASAR" and "pro-compile"
JamesCropcho Mar 6, 2018
f99625d
Seeing what happens if I commit these three rough CI configs
JamesCropcho Mar 1, 2018
f4222f6
Created a `yarn.lock`
JamesCropcho Mar 1, 2018
131c35e
added `yarn-error.log` to .gitignore
JamesCropcho Mar 3, 2018
814b121
added `cross-env`; added `pack` script
JamesCropcho Mar 3, 2018
12b1663
attempting to move release files on Circle to a new directory
JamesCropcho Mar 3, 2018
60f9023
Attempting pack on CircleCI
JamesCropcho Mar 3, 2018
c38f288
Making .travis.yml more like Falcon's
JamesCropcho Mar 3, 2018
153c08a
Using Yarn instead of NPM in `package.json`
JamesCropcho Mar 3, 2018
a06bebf
making Electron-Builder a 'dev' dependency
JamesCropcho Mar 3, 2018
353b032
Appeasing Yarn with a new Node.js v6x.
JamesCropcho Mar 3, 2018
98ec707
storing (hopefully) /dist artifacts
JamesCropcho Mar 3, 2018
a3ff530
Really this time, using node v6.13 and not v6.10
JamesCropcho Mar 3, 2018
6bff159
Minor-upgrade Node.js images; only `pack` on v8.x, and once
JamesCropcho Mar 3, 2018
476d690
fixing a typo?
JamesCropcho Mar 3, 2018
2a3183c
Fixed typo?
JamesCropcho Mar 3, 2018
21a1071
Moving `steps` more smartly
JamesCropcho Mar 3, 2018
475c082
using proper Workflow (CircleCI)
JamesCropcho Mar 3, 2018
5e74260
I can't really work effectively with nondeterminstic timeout failures…
JamesCropcho Mar 3, 2018
38b65e0
Need to `checkout`?
JamesCropcho Mar 3, 2018
9cbb9cd
I'm restoring the Worflow for CircleCI since I'm not working on it fo…
JamesCropcho Mar 3, 2018
dcd04aa
Wrong place for CircleCI configuration
JamesCropcho Mar 3, 2018
5bc57e3
Fixes for Travic CI configuration
JamesCropcho Mar 3, 2018
56837e3
Commenting out Falcon-esque AppVeyor stuff
JamesCropcho Mar 3, 2018
beeeec1
Hoping AppVeyor keeps the dist files
JamesCropcho Mar 3, 2018
f161a07
better caching?
JamesCropcho Mar 3, 2018
4ec838a
Yarn, we need not force you, yes, I hope?
JamesCropcho Mar 3, 2018
a24d7fa
Even more caching maybe?
JamesCropcho Mar 3, 2018
0161fda
Attempting restoration of the 7zip thing at @n-riesco's suggestion
JamesCropcho Mar 3, 2018
e416deb
Made my `package.json` more like Falcon's
JamesCropcho Mar 5, 2018
63056cc
Fixed CI-specific config to match the new `package.json`
JamesCropcho Mar 5, 2018
d3ec2e0
Attempting "pro-ASAR" and "pro-compile"
JamesCropcho Mar 6, 2018
a6cef24
Merge branch 'test-builds-across-travis-circle-appveyor' of github.co…
JamesCropcho Mar 26, 2018
cf49276
Making .travis.yml look (even more?) like Falcon's w/r/t artifacts
JamesCropcho Mar 26, 2018
068399f
Hopefully fixing Travis CI build via paths issue resolving
JamesCropcho Mar 30, 2018
b7781fe
A typo, perhaps?
JamesCropcho Mar 30, 2018
cc257b7
No `release*.yml`
JamesCropcho Mar 30, 2018
5846500
"We don't really want to include release/linux-unpacked (131MB)."
JamesCropcho Apr 6, 2018
d0ddfca
Trigger build
JamesCropcho Apr 6, 2018
7bca355
"In Falcon, we also keep a cache for .git."
JamesCropcho Apr 6, 2018
a8c0007
"If I'm not mistaken using {{ .Environment.CIRCLE_JOB }} defeats the …
JamesCropcho Apr 6, 2018
416727d
Trigger build
JamesCropcho Apr 6, 2018
5565f69
Trigger build
JamesCropcho Apr 6, 2018
c8366be
"I'd add the same steps to save and restore caches for .git and node_…
JamesCropcho Apr 6, 2018
16074a8
Trigger build
JamesCropcho Apr 6, 2018
0ab9188
"I don't know why these paths are listed in the cache. Have you found…
JamesCropcho Apr 6, 2018
8afaf50
"we need to disable asar"
JamesCropcho Apr 6, 2018
fea522b
"getExportedArgs should use slice(1) instead of slice(2) when the exe…
JamesCropcho Apr 6, 2018
50c728b
Trigger build
JamesCropcho Apr 6, 2018
7b7a265
Trigger build
JamesCropcho Apr 7, 2018
e61bddf
Trigger build
JamesCropcho Apr 7, 2018
a8c5a89
Revert ""getExportedArgs should use slice(1) instead of slice(2) when…
JamesCropcho Apr 7, 2018
b308e9f
"@JamesCropcho solving the issue with getExportedArgs requires a bit …
JamesCropcho Apr 7, 2018
1b85311
"The upside being a package lock file? In that case, that's not an up…
JamesCropcho Apr 7, 2018
0868830
Committing package-lock.json
JamesCropcho Apr 7, 2018
cb7b0a2
`npm ERR! peer dep missing: ajv@^6.0.0, required by ajv-keywords@3.1.0`
JamesCropcho Apr 7, 2018
324f7bf
"If we include cross-env in dependecies, then cross-env will be inclu…
JamesCropcho Apr 7, 2018
7e66d21
`npm ERR! missing: ajv@^5.1.0, required by har-validator@5.0.3`
JamesCropcho Apr 7, 2018
a5b0e51
Use a Plotly e-mail address for 'maintainer'
JamesCropcho Apr 7, 2018
1d0b175
"We still need to include AppImage in build.linux.target."
JamesCropcho Apr 7, 2018
4d7fd09
Optimized CircleCi workflow
JamesCropcho Apr 7, 2018
4625de9
"In Falcon we use 2 caches: one for .git, and another for node_modules"
JamesCropcho Apr 7, 2018
2225b5a
Trigger build
JamesCropcho Apr 7, 2018
0880a4e
`ERR! missing script: rebuild:modules:electron`
JamesCropcho Apr 7, 2018
582bb93
Trigger build
JamesCropcho Apr 7, 2018
890ac78
"I prefer locking down the image versions."
JamesCropcho Apr 11, 2018
4a2c81c
"I forgot about Étienne's comment: we should remove `ajv` from the de…
JamesCropcho Apr 11, 2018
1b78585
"How about merging docker-push and docker-build into one job? Then, t…
JamesCropcho Apr 11, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
92 changes: 77 additions & 15 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,17 @@ workflows:
jobs:
- test-node-v6
- test-node-v8
- docker-build
- docker-push:
- docker-build:
requires:
- test-node-v6
- test-node-v8
- docker-push:
requires:
- docker-build
- electron-pack-and-release:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job didn't run:

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@n-riesco It appears to me that "electron-pack-and-release" is running: https://circleci.com/gh/plotly/image-exporter/tree/test-builds-across-travis-circle-appveyor

(See #1124 for example.)

Please pardon, am I missing something simple?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JamesCropcho For some reason, github doesn't show the build (as seen on the above screenshot). Since circleCI runs the job, then this isn't an issue.

requires:
- test-node-v6
- test-node-v8

version: 2
jobs:
Expand All @@ -31,11 +36,22 @@ jobs:
command: |
echo "npm: $(npm --version)"
echo "node: $(node --version)"
- restore_cache:
key: v1-source-{{ .Branch }}-{{ .Revision }}
keys:
- v1-source-{{ .Branch }}-
- v1-source-
- checkout
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

- save_cache:
key: v1-source-{{ .Branch }}-{{ .Revision }}
paths:
- ".git"
- restore_cache:
key: v2-npm-{{ checksum "package-lock.json" }}
keys:
- v{{ .Environment.CIRCLE_CACHE_VERSION }}-deps-{{ .Branch }}-{{ .Environment.CIRCLE_JOB }}-{{ checksum "package.json" }}
- v{{ .Environment.CIRCLE_CACHE_VERSION }}-deps-master-{{ .Environment.CIRCLE_JOB }}-{{ checksum "package.json" }}
- v2-npm-
- run: node --version
- run: npm --version
- run:
name: Install deps
command: |
Expand All @@ -44,15 +60,15 @@ jobs:
- run:
name: List deps
command: |
npm ls
npm list
pdftops -v
- save_cache:
key: v2-npm-{{ checksum "package-lock.json" }}
paths:
- node_modules
key: v{{ .Environment.CIRCLE_CACHE_VERSION }}-deps-{{ .Branch }}-{{ .Environment.CIRCLE_JOB }}-{{ checksum "package.json" }}
- run:
name: Test
command: npm test
command: npm run test
- run:
name: Coverage
command: npm run coverage
Expand All @@ -62,38 +78,50 @@ jobs:
test-node-v6:
<<: *base
docker:
- image: circleci/node:6.10.3-browsers
- image: circleci/node:6-browsers

test-node-v8:
<<: *base
docker:
- image: circleci/node:8.4.0-browsers
- image: circleci/node:8-browsers
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@etpinard My rationale, is that this software is made to run on many people's personal computers, with diverse versioning profiles, rather than servers for which Plotly has complete control to use only one very precise version.

I am happy to revert this change if Plotly wishes me to do so, otherwise I vote to keep this change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer locking down the image versions. I don't think this project is likely to break from one minor release of node.js to another.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I have made this change. How's it look?


docker-build:
docker:
- image: circleci/node:6.10.3-browsers

- image: circleci/node:8-browsers
steps:
- setup_remote_docker:
reusable: true

- restore_cache:
key: v1-source-{{ .Branch }}-{{ .Revision }}
keys:
- v1-source-{{ .Branch }}-
- v1-source-
- checkout

- save_cache:
key: v1-source-{{ .Branch }}-{{ .Revision }}
paths:
- ".git"
- restore_cache:
keys:
- v2-npm-{{ checksum "package-lock.json" }}
- run:
name: Build Docker image
command: |
docker build -f deployment/Dockerfile -t quay.io/plotly/image-exporter:$CIRCLE_SHA1 .

- run:
name: Smoke test Docker image
command: |
docker run -d -p 9091:9091/tcp --name imageserver quay.io/plotly/image-exporter:$CIRCLE_SHA1
docker run --network container:imageserver quay.io/plotly/wget wget --retry-connrefused --waitretry=1 -t 60 -O /dev/null --progress=dot http://localhost:9091/ping

- save_cache:
paths:
- node_modules
key: v2-npm-{{ checksum "package-lock.json" }}

docker-push:
docker:
- image: circleci/node:6.10.3-browsers
- image: circleci/node:8-browsers

steps:
- setup_remote_docker:
Expand All @@ -106,3 +134,37 @@ jobs:
docker tag quay.io/plotly/image-exporter:$CIRCLE_SHA1 quay.io/plotly/image-exporter:$CIRCLE_BRANCH
docker push quay.io/plotly/image-exporter:$CIRCLE_SHA1
docker push quay.io/plotly/image-exporter:$CIRCLE_BRANCH

electron-pack-and-release:
docker:
- image: circleci/node:8-browsers
steps:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add the same steps to save and restore caches for .git and node_modules as those in the job test-node-v8.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@n-riesco good call. Okay, I have added these caches: https://circleci.com/gh/plotly/image-exporter/1201#config/containers/0. I ran the builds twice after this change, to ensure it worked with and without a primed cache—all good. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JamesCropcho {{ checksum "yarn.lock" }} is only good for node_modules. I'd remove .git from paths.

In Falcon we use 2 caches:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct, @n-riesco . I have changed the CircleCI caching strategy as per your recommendations, and tested with fresh and primed caches.

- restore_cache:
key: v1-source-{{ .Branch }}-{{ .Revision }}
keys:
- v1-source-{{ .Branch }}-
- v1-source-
- checkout
- save_cache:
key: v1-source-{{ .Branch }}-{{ .Revision }}
paths:
- ".git"
- restore_cache:
key: v2-npm-{{ checksum "package-lock.json" }}
keys:
- v2-npm-
- run: node --version
- run: npm --version
- run: npm install
- run:
name: Electron-Builder pack
command: npm run pack
- save_cache:
key: v2-npm-{{ checksum "package-lock.json" }}
paths:
- node_modules
- run: mkdir artifacts
- run: mv release/*.yml artifacts || true
- run: mv release/image-exporter* artifacts
- store_artifacts:
path: artifacts
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ coverage
.nyc_output

/build
dist/
/release

deployment/*.retry

/yarn-error.log
31 changes: 31 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
matrix:
include:
- os: osx
osx_image: xcode9.0
language: node_js
node_js: "8"
env:
- ELECTRON_CACHE=$HOME/.cache/electron
- ELECTRON_BUILDER_CACHE=$HOME/.cache/electron-builder

cache:
directories:
- node_modules
- $HOME/.cache/electron
- $HOME/.cache/electron-builder

install:
- npm install

script:
- npm run pack
- zip mac-release.zip release/Plotly\ Graph\ Exporter*

before_cache:
- rm -rf $HOME/.cache/electron-builder/wine

addons:
artifacts:
s3_region: us-east-1
paths: mac-release.zip
debug: true
33 changes: 33 additions & 0 deletions appveyor.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
version: 0.1.{build}

environment:
matrix:
- nodejs_version: 8

platform:
- x64

#services:

cache:
- node_modules
- '%LOCALAPPDATA%\electron\Cache'
- '%LOCALAPPDATA%\electron-builder\cache'

init:
- git config --global core.autocrlf input

install:
- ps: Install-Product node $env:nodejs_version $env:platform

build_script:
- npm run pack

after_build:
- 7z a release.zip release/latest.yml release/*.exe

artifacts:
- path: release.zip
name: release

test: off
7 changes: 6 additions & 1 deletion bin/args.js
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,12 @@ const SERVER_MINIMIST_CONFIG = {
}

exports.getExporterArgs = function () {
return minimist(process.argv.slice(2), EXPORTER_MINIMIST_CONFIG)
// https://electronjs.org/docs/api/process#processdefaultapp
// https://github.com/electron/electron/issues/4690#issuecomment-217435222
// https://github.com/tj/commander.js/issues/512
const sliceBegin = process.defaultApp ? 2 : 1

return minimist(process.argv.slice(sliceBegin), EXPORTER_MINIMIST_CONFIG)
}

exports.getServerArgs = function () {
Expand Down