chore: upload & promote builds to s3 with new key structure #458

RasPhilCo · 2020-12-07T20:00:55Z

This PR:

removes all publish commands
creates the upload command
upload uses the CLI version + git sha to push build artifacts to s3 at versions/<version>/<sha>/<artifact>
creates the promote command
promote takes channel, version & sha flags and copies the corresponding build manifests/builds to channel/<channel>/<artifact> for the update command to then pick up

RasPhilCo · 2020-12-19T01:16:20Z

See also: #347

amphro

We reviewed this in person. Looks good minus changes that will be made as we debug it's use with the Salesforce CLI.

src/commands/promote.ts

Bumps [oclif](https://github.com/oclif/oclif) from 1.16.1 to 1.16.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/oclif/oclif/releases">oclif's releases</a>.</em></p> <blockquote> <h2>v1.16.2</h2> <ul> <li>chore: upload & promote builds to s3 with new key structure (<a href="https://github.com/oclif/oclif/issues/458">#458</a>) 7b4cd51</li> <li>chore(deps): bump tslib from 2.0.3 to 2.1.0 (<a href="https://github.com/oclif/oclif/issues/489">#489</a>) 78957c7</li> <li>chore(deps): bump github-slugger from 1.2.1 to 1.3.0 (<a href="https://github.com/oclif/oclif/issues/469">#469</a>) d4f314a</li> <li>chore(deps-dev): bump @types/lodash from 4.14.165 to 4.14.167 (<a href="https://github.com/oclif/oclif/issues/483">#483</a>) 5d34bd0</li> <li>chore(deps-dev): bump ts-node from 9.0.0 to 9.1.1 (<a href="https://github.com/oclif/oclif/issues/467">#467</a>) a7a7565</li> <li>chore(deps-dev): bump @types/node from 14.14.10 to 14.14.20 (<a href="https://github.com/oclif/oclif/issues/484">#484</a>) 89a82cf</li> <li>chore(deps-dev): bump fancy-test from 1.4.9 to 1.4.10 (<a href="https://github.com/oclif/oclif/issues/471">#471</a>) ab49541</li> <li>chore(deps-dev): bump @types/mocha from 8.0.4 to 8.2.0 (<a href="https://github.com/oclif/oclif/issues/473">#473</a>) 86387a6</li> <li>chore(deps-dev): bump @oclif/dev-cli from 1.24.2 to 1.26.0 (<a href="https://github.com/oclif/oclif/issues/470">#470</a>) da42bfc</li> <li>chore(deps-dev): bump aws-sdk from 2.802.0 to 2.821.0 (<a href="https://github.com/oclif/oclif/issues/485">#485</a>) 7841b84</li> <li>chore(deps): bump @oclif/plugin-help from 3.2.0 to 3.2.1 (<a href="https://github.com/oclif/oclif/issues/472">#472</a>) b4faa56</li> <li>chore(deps): bump cli-ux from 5.3.2 to 5.5.1 (<a href="https://github.com/oclif/oclif/issues/466">#466</a>) be28fad</li> <li>chore(deps-dev): bump @types/fs-extra from 8.1.1 to 9.0.6 (<a href="https://github.com/oclif/oclif/issues/479">#479</a>) 52db244</li> <li>chore(deps-dev): bump eslint from 7.14.0 to 7.16.0 (<a href="https://github.com/oclif/oclif/issues/481">#481</a>) 2072a3e</li> <li>chore: sync dependabot.yml (<a href="https://github.com/oclif/oclif/issues/463">#463</a>) adec1b5</li> <li>chore(deps): bump ini from 1.3.5 to 1.3.7 (<a href="https://github.com/oclif/oclif/issues/459">#459</a>) 1b65cbb</li> <li>chore: port dev-cli as hidden cmds (<a href="https://github.com/oclif/oclif/issues/457">#457</a>) 00c9b75</li> <li>chore(deps-dev): bump mocha from 5.2.0 to 8.2.1 (<a href="https://github.com/oclif/oclif/issues/444">#444</a>) f32f6ab</li> <li>chore(deps-dev): bump @oclif/dev-cli from 1.24.0 to 1.24.2 (<a href="https://github.com/oclif/oclif/issues/456">#456</a>) 01d5b05</li> <li>chore(deps-dev): bump @types/yeoman-generator from 3.0.2 to 3.1.4 (<a href="https://github.com/oclif/oclif/issues/455">#455</a>) 858a6ed</li> <li>chore(deps-dev): bump eslint from 7.13.0 to 7.14.0 (<a href="https://github.com/oclif/oclif/issues/454">#454</a>) fe653da</li> <li>chore(deps-dev): bump eslint from 7.6.0 to 7.13.0 (<a href="https://github.com/oclif/oclif/issues/449">#449</a>) 6da9519</li> <li>chore(deps): bump bl from 3.0.0 to 3.0.1 (<a href="https://github.com/oclif/oclif/issues/428">#428</a>) 8698ac5</li> <li>chore(deps-dev): bump @oclif/dev-cli from 1.23.0 to 1.24.0 (<a href="https://github.com/oclif/oclif/issues/452">#452</a>) 8f84645</li> <li>chore(deps-dev): bump @types/lodash from 4.14.158 to 4.14.165 (<a href="https://github.com/oclif/oclif/issues/450">#450</a>) 249fb7c</li> <li>chore(deps): bump debug from 4.1.1 to 4.3.1 (<a href="https://github.com/oclif/oclif/issues/448">#448</a>) ba6686e</li> <li>chore(deps-dev): bump execa from 4.0.3 to 4.1.0 (<a href="https://github.com/oclif/oclif/issues/443">#443</a>) 14effe3</li> <li>chore(deps): bump tslib from 2.0.1 to 2.0.3 (<a href="https://github.com/oclif/oclif/issues/436">#436</a>) a38d4fb</li> <li>chore(deps): bump lodash from 4.17.19 to 4.17.20 (<a href="https://github.com/oclif/oclif/issues/420">#420</a>) 3aa02ff</li> <li>chore(deps): bump @oclif/config from 1.16.0 to 1.17.0 (<a href="https://github.com/oclif/oclif/issues/410">#410</a>) c3df0dc</li> <li>chore: sync dependabot.yml (<a href="https://github.com/oclif/oclif/issues/417">#417</a>) 899f841</li> <li>chore(deps): bump tree-kill from 1.2.1 to 1.2.2 (<a href="https://github.com/oclif/oclif/issues/430">#430</a>) 6eccde3</li> <li>chore(deps): bump @oclif/errors from 1.3.3 to 1.3.4 (<a href="https://github.com/oclif/oclif/issues/446">#446</a>) 13adfe1</li> <li>chore(deps-dev): bump @oclif/dev-cli from 1.22.2 to 1.23.0 (<a href="https://github.com/oclif/oclif/issues/447">#447</a>) e255e1d</li> <li>chore(deps-dev): bump ts-node from 8.10.2 to 9.0.0 (<a href="https://github.com/oclif/oclif/issues/425">#425</a>) 966def2</li> <li>chore(deps): bump tslib from 2.0.0 to 2.0.1 (<a href="https://github.com/oclif/oclif/issues/414">#414</a>) 09fe33b</li> <li>chore(deps): bump @oclif/command from 1.7.0 to 1.8.0 (<a href="https://github.com/oclif/oclif/issues/413">#413</a>) 0dd9ec4</li> <li>chore(deps): bump @oclif/plugin-help from 3.1.0 to 3.2.0 (<a href="https://github.com/oclif/oclif/issues/416">#416</a>) c8104ae</li> <li>chore(deps-dev): bump eslint from 7.5.0 to 7.6.0 (<a href="https://github.com/oclif/oclif/issues/406">#406</a>) 2e3f095</li> <li>chore: sync dependabot.yml (<a href="https://github.com/oclif/oclif/issues/402">#402</a>) b91fd4d</li> <li>chore(deps-dev): bump @types/lodash from 4.14.157 to 4.14.158 (<a href="https://github.com/oclif/oclif/issues/401">#401</a>) 7522cce</li> <li>chore(deps-dev): bump eslint from 7.4.0 to 7.5.0 (<a href="https://github.com/oclif/oclif/issues/399">#399</a>) a99ad28</li> <li>chore(deps-dev): bump fancy-test from 1.4.4 to 1.4.9 (<a href="https://github.com/oclif/oclif/issues/400">#400</a>) 6d3f7b6</li> <li>chore(deps-dev): bump nps from 5.9.12 to 5.10.0 (<a href="https://github.com/oclif/oclif/issues/395">#395</a>) c039c6f</li> <li>chore(deps-dev): bump execa from 4.0.2 to 4.0.3 (<a href="https://github.com/oclif/oclif/issues/396">#396</a>) 600defc</li> <li>chore(deps-dev): bump semantic-release from 15 to 17 (<a href="https://github.com/oclif/oclif/issues/397">#397</a>) af585f9</li> <li>chore(deps): bump lodash from 4.17.15 to 4.17.19 (<a href="https://github.com/oclif/oclif/issues/394">#394</a>) 37448a9</li> <li>chore(deps-dev): bump eslint from 7.3.1 to 7.4.0 (<a href="https://github.com/oclif/oclif/issues/392">#392</a>) de47c28</li> <li>chore(deps): bump @oclif/config from 1.15.1 to 1.16.0 (<a href="https://github.com/oclif/oclif/issues/391">#391</a>) ee15e49</li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/oclif/oclif/commit/de7c59dc41433cda0cb0f16dc5e263f191345914"><code>de7c59d</code></a> v1.16.2</li> <li><a href="https://github.com/oclif/oclif/commit/7b4cd51b3549dc9e4fc720a9bf483c30ecd4afef"><code>7b4cd51</code></a> chore: upload & promote builds to s3 with new key structure (<a href="https://github.com/oclif/oclif/issues/458">#458</a>)</li> <li><a href="https://github.com/oclif/oclif/commit/78957c7d2d81b4dc0e78077547df48effb09292b"><code>78957c7</code></a> chore(deps): bump tslib from 2.0.3 to 2.1.0 (<a href="https://github.com/oclif/oclif/issues/489">#489</a>)</li> <li><a href="https://github.com/oclif/oclif/commit/d4f314aa8474cec3f84f6007b0ec785e2c2548cc"><code>d4f314a</code></a> chore(deps): bump github-slugger from 1.2.1 to 1.3.0 (<a href="https://github.com/oclif/oclif/issues/469">#469</a>)</li> <li><a href="https://github.com/oclif/oclif/commit/5d34bd06e74afeddc895bcb94c3a1591ff67b0f8"><code>5d34bd0</code></a> chore(deps-dev): bump @types/lodash from 4.14.165 to 4.14.167 (<a href="https://github.com/oclif/oclif/issues/483">#483</a>)</li> <li><a href="https://github.com/oclif/oclif/commit/a7a7565297b16fd1c75157fc1d9047c05b4493d9"><code>a7a7565</code></a> chore(deps-dev): bump ts-node from 9.0.0 to 9.1.1 (<a href="https://github.com/oclif/oclif/issues/467">#467</a>)</li> <li><a href="https://github.com/oclif/oclif/commit/89a82cf6ebfc924b39b6a5e5f0ae41df08dec2bb"><code>89a82cf</code></a> chore(deps-dev): bump @types/node from 14.14.10 to 14.14.20 (<a href="https://github.com/oclif/oclif/issues/484">#484</a>)</li> <li><a href="https://github.com/oclif/oclif/commit/ab49541a4f49e7246fc1d4211a4442028e5bf30e"><code>ab49541</code></a> chore(deps-dev): bump fancy-test from 1.4.9 to 1.4.10 (<a href="https://github.com/oclif/oclif/issues/471">#471</a>)</li> <li><a href="https://github.com/oclif/oclif/commit/86387a6994914a22ee339408cd066fc542a7aab6"><code>86387a6</code></a> chore(deps-dev): bump @types/mocha from 8.0.4 to 8.2.0 (<a href="https://github.com/oclif/oclif/issues/473">#473</a>)</li> <li><a href="https://github.com/oclif/oclif/commit/da42bfc314d86ef87c6fa5104ca9efdb11a64d3b"><code>da42bfc</code></a> chore(deps-dev): bump @oclif/dev-cli from 1.24.2 to 1.26.0 (<a href="https://github.com/oclif/oclif/issues/470">#470</a>)</li> <li>Additional commits viewable in <a href="https://github.com/oclif/oclif/compare/v1.16.1...v1.16.2">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=oclif&package-manager=npm_and_yarn&previous-version=1.16.1&new-version=1.16.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

RasPhilCo added 2 commits December 9, 2020 11:54

chore: publish to s3 with new key strutcture

550fc7b

dont forget the files

1791357

RasPhilCo force-pushed the publish-s3-keys branch from c3e69f3 to 1791357 Compare December 9, 2020 19:54

RasPhilCo added 13 commits January 13, 2021 11:10

new upload publish

71ea5d6

new upload publish:*

e5fd2c9

rename publish -> upload

779b841

reuse gitSha fn

9aecd68

dont reuse async gitSha

22fd6bb

add promote cmd

0827fac

update build manifest with new keys

627f868

readability edits

b20b898

add darwin,win,deb to promote

0b93125

make build manifest file more clear

9f3d0ab

refactor utils

8d25e28

add cli action to promote

abab9fa

refactor local vs cloud s3Key templating

900992d

RasPhilCo mentioned this pull request Jan 15, 2021

feat: support oclif upload/promote cmds new s3 key structure oclif/plugin-update#204

Merged

RasPhilCo requested a review from amphro January 15, 2021 00:24

amphro approved these changes Jan 21, 2021

View reviewed changes

RasPhilCo and others added 10 commits January 27, 2021 16:19

fix guards

befc3f2

fix bin/run

517d904

fix s3key calls

62f0ffd

rm build base tarball

918a024

promote refactor

d944159

mk target flags enum

36edfa5

promote guards

816af5a

pkg

4731318

Merge branch 'master' into publish-s3-keys

7d4303a

rm github cmd

fb62dff

RasPhilCo added 8 commits January 27, 2021 17:34

add oclif.upload.s3.folder

3e63ece

rename s3ShortKey

9b7c3b7

missing args

983bddc

bunch-o-edits

ab12a6c

fix: manifest lookup & channels url

3b4939a

fix git rename

53ae231

fix git rename

c88627e

fix git rename

d2a7b84

RasPhilCo changed the title ~~chore: publish to s3 with new key structure~~ chore: upload & promote builds to s3 with new key structure Feb 3, 2021

RasPhilCo added 8 commits February 4, 2021 16:01

edits

8f7316c

lint

f2e3aed

default xz to true

d747ff9

fix upload xz

f0f3f57

fix: promote logging

2001e81

feat: promote max-age

be2cb47

fix: promote logging

8c88c7a

fix: version

18b1b1e

amphro reviewed Feb 11, 2021

View reviewed changes

src/commands/promote.ts Show resolved Hide resolved

RasPhilCo and others added 6 commits February 12, 2021 13:52

promote unversioned tarballs too

560877e

fix: typo in keys (#499)

2ee7534

fix: macos uploading & promoting

8de3385

feat: add windows singing to pack:win

97f6325

fix: add a note about skipping macos singing

06cfd08

Merge branch 'master' into publish-s3-keys

396a676

RasPhilCo merged commit 7b4cd51 into master Mar 11, 2021

RasPhilCo deleted the publish-s3-keys branch March 11, 2021 19:24

dependabot bot mentioned this pull request Mar 12, 2021

Bump oclif from 1.16.1 to 1.16.2 pravinbashyal/epubinator#96

Merged

mdonnalley mentioned this pull request Feb 16, 2022

Fix tarballs based commands not considering config from package.json #799

Merged

gaelmotte pushed a commit to gaelmotte/oclif that referenced this pull request Oct 24, 2022

chore: upload & promote builds to s3 with new key structure (oclif#458)

9c7f895

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: upload & promote builds to s3 with new key structure #458

chore: upload & promote builds to s3 with new key structure #458

RasPhilCo commented Dec 7, 2020 •

edited

RasPhilCo commented Dec 19, 2020

amphro left a comment

chore: upload & promote builds to s3 with new key structure #458

chore: upload & promote builds to s3 with new key structure #458

Conversation

RasPhilCo commented Dec 7, 2020 • edited

RasPhilCo commented Dec 19, 2020

amphro left a comment

Choose a reason for hiding this comment

RasPhilCo commented Dec 7, 2020 •

edited