electron-builder updater not using authorization (or any headers) in 'generic' provider #6192
Comments
|
Tbh, I still don't understand the "custom" provider That being said, I actually just wrote an official Publisher and Provider for Keygen #6167 electron-builder/packages/electron-updater/src/AppUpdater.ts Lines 97 to 101 in a755c82 For creating an updater, you only need to use prop electron-builder/packages/electron-updater/src/main.ts Lines 18 to 30 in dc359de I'm not sure why your provider headers are not working though unless there's an authorization header being provided elsewhere which I guess is very unlikely electron-builder/packages/electron-updater/src/providers/Provider.ts Lines 69 to 75 in dc359de I see Bintray provider has header logic though, maybe try copying that to your provider and setting the auth header through there? Hope that helps! (and that I didn't misunderstand any parts) |
|
@mmaietta Thanks for this. I'm just digging into the src code because I need to implement publish/auto-update with Atlassian products (so bitbucket) and private repos. And I'm having a hard time following the trail of what core code calls and where to hook in custom code. Also, We don't want to monkey-patch or fork this repo to add custom code to packages. Is that what you've done? I'm asking because of the file path you listed in under electron-builder. Or is there a way to drop updater code somewhere that core code will know to grab like the Publisher does with |
|
@develar is there someone from the core team that can be commissioned 💰 to create/update documentation on how to create both custom Publisher and Updater/Provider? The end goal specifically is to implement Bitbucket.org private repo integration for a private internal company project. |
@jbool24 I'm not sure I follow. The is a monorepo, so each module resides under the path Would you be interested in contributing to this project for electron-builder to officially support bitbucket? I'd be more than happy to assist in getting it integrated. It'd be cool to have a BitbucketPublisher and BitbucketProvider |
|
@mmaietta if we were to implement custom code we didn't want to have to keep a local package to inject into 'electron-updater' and we def did not want to modify the package after download (monkey-patch). Since we will need to do it either way I wouldn't mind making it a Bitbucket provider. I however I'd need a good alley oop since that part of the code is hard for me to follow. I looked at your Keygen a bit but still confused at which functions are hooked by the core and which are your internals. Also the current Publisher above is not actually publishing to the repo but doesn' throw errors in code or network response side. So that still needs work also. |
|
Happy to help! For the publisher, only For the provider only these are needed For integration testing, here are two skeletons you could easily reuse electron-builder/test/src/updater/nsisUpdaterTest.ts Lines 49 to 58 in 0880d1b electron-builder/test/src/ArtifactPublisherTest.ts Lines 128 to 142 in 0880d1b Notes:
|
|
@mmaietta OK, I'll start to dive in and go from there. For my publisher it looks as though @develar Can you provide an background or any suggestions/ideas?? |
|
If you extend HttpPublisher for simplicity, then yes, electron-builder/packages/electron-publish/src/publisher.ts Lines 102 to 108 in 0880d1b Ref: https://github.com/electron-userland/electron-builder/search?q=doUpload |
|
@mmaietta Ok. I found that spot. I will certainly need to override {
"type": "error",
"error": {
"fields": {
"files": [
"This field is required."
]
},
"message": "files: This field is required."
}
}Here is a snippet of that code. I'm wondering, has the request already started the stream with data elsewhere before I start piping so that the request will be malformed? Currently I've only been testing the const FormData = require('form-data')
//...
class CustomPublisher extends HttpPublisher {
//...
// Override base class function
async upload(task) {
const fileName = (this.useSafeArtifactName ? task.safeArtifactName : null) || basename(task.file);
if (task.fileContent != null) {
console.log('upload with content')
const form = new FormData();
form.append('files', task.fileContent, { fileName })
this.options.headers = { ...this.options.headers, ...form.getHeaders() }
await this.doUpload(
fileName,
task.arch || Arch.x64,
task.fileContent.length,
(request, reject) => {
form.on('end', () => console.log('ALL FORM DATA READ'))
form.pipe(request);
},
);
return;
}
//...
} |
|
After further inspection I found that With NetCat listening on port 8888 of local host nc -l localhost 8888This is output from curl to netcat. Importantly observe FormData output between boundry curl -X POST localhost:8888 -v -H "Authorization: Basic ***REDACTED***" -v -F files=@dist/latest-linux.yml POST / HTTP/1.1
Host: localhost:8888
User-Agent: curl/7.68.0
Accept: */*
Authorization: Basic ***REDACTED***
Content-Length: 620
Content-Type: multipart/form-data; boundary=------------------------39f4731450e9571b
--------------------------39f4731450e9571b
Content-Disposition: form-data; name="files"; filename="latest-linux.yml"
Content-Type: application/octet-stream
version: 21.9.1-1630512963907
files:
- url: TESTAPP-21.9.1-1630512963907.AppImage
sha512: 01ReKGhqGcoF4iQRGZvq8s60qCSHuFCGQqMYBAojMH0uhqUIzEON5Y/cTtEsLxpA45/olIQ71QiJKtJJzHYR6w==
size: 84799057
blockMapSize: 89701
path: TESTAPP-21.9.1-1630512963907.AppImage
sha512: 01ReKGhqGcoF4iQRGZvq8s60qCSHuFCGQqMYBAojMH0uhqUIzEON5Y/cTtEsLxpA45/olIQ71QiJKtJJzHYR6w==
releaseDate: '2021-09-01T16:16:11.019Z'
--------------------------39f4731450e9571b--
Now here is spying on the stream with a passthrough Transform stream to print the contents of FormData async upload(task) {
const fileName = (this.useSafeArtifactName ? task.safeArtifactName : null) || basename(task.file);
if (task.fileContent != null) {
console.log('upload with content')
const form = new FormData();
form.append(
'files',
task.fileContent,
{
filename: fileName,
header: `\r\n--${form.getBoundary()}\r\n` +
`Content-Disposition: form-data; name="files"; filename="${fileName}"\r\n` +
`Content-Type: ${mime.getType(fileName)}\r\n`,
},
);
this.options.headers = { ...this.options.headers, ...form.getHeaders() };
// for debugging the stream
const passthrough = new Transform({
transform: (chunk, encoding, cb) => {
console.log(chunk.toString())
cb(null, chunk)
}
})
await this.doUpload(
fileName,
task.arch || Arch.x64,
task.fileContent.length,
(request, reject) => {
request.on('end', () => console.log('ALL DONE'))
request.on('pipe', (src) => console.log('Pipe from', src.constructor.name))
request.on('error', (e) => {console.error(e); reject(e)})
passthrough.on('end', () => { console.log('FORM SENT'); request.end(); });
form.pipe(passthrough).pipe(request, {end: false}); // manually calling end for request stream writable
// form.pipe(request);
},
);
return;
}and here is the output upload with content
{
hostname: 'api.bitbucket.org',
path: '{{REDACTED PRIVATE REPO PATH}}',
protocol: 'https:',
method: 'POST',
headers: {
Host: 'api.bitbucket.org',
Accept: '*/*',
'Content-Length': 411,
'content-type': 'multipart/form-data; boundary=--------------------------581743681975947702003689',
authorization: 'Basic {{**REDACTED**}}',
'User-Agent': 'electron-builder',
'Cache-Control': 'no-cache'
}
}
----------------------------581743681975947702003689
Content-Disposition: form-data; name="files"; filename="latest-linux.yml"
Content-Type: text/yaml
version: 21.9.1-1630515079265
files:
- url: TESTAPP-21.9.1-1630515079265.AppImage
sha512: vs5TFNf3zt+CzQAxPt+NimNDUKHHafcXmgkYQ7IyrhtYcK5r9x8rNyNqjLUALkHV1zhH9AQH3uYwteC1SpsSnQ==
size: 84799057
blockMapSize: 89701
path: TESTAPP-21.9.1-1630515079265.AppImage
sha512: vs5TFNf3zt+CzQAxPt+NimNDUKHHafcXmgkYQ7IyrhtYcK5r9x8rNyNqjLUALkHV1zhH9AQH3uYwteC1SpsSnQ==
releaseDate: '2021-09-01T16:51:26.899Z'
----------------------------581743681975947702003689--
Pipe from Transform
FORM SENT
Status Code: 400
Message: 400 Bad Request
{
"type": "error",
"error": {
"fields": {
"files": [
"This field is required."
]
},
"message": "files: This field is required."
}
}
Headers: {
"server": "nginx",
"vary": "Authorization, Origin",
"cache-control": "no-cache, no-store, must-revalidate, max-age=0",
"content-type": "application/json; charset=utf-8",
"x-b3-traceid": "66c1242c919e4503",
"x-oauth-scopes": "repository:write",
"x-usage-output-ops": "0",
"x-dc-location": "Micros",
"strict-transport-security": "max-age=31536000; includeSubDomains; preload",
"date": "Wed, 01 Sep 2021 16:51:27 GMT",
"x-usage-user-time": "0.120622",
"x-usage-system-time": "0.001988",
"expires": "Wed, 01 Sep 2021 16:51:27 GMT",
"x-served-by": "ab58f15e475f",
"x-view-name": "bitbucket.apps.downloads.api.v20.handlers.DownloadsHandler",
"x-static-version": "12fbe411e51b",
"x-credential-type": "apppassword",
"x-render-time": "0.0714159011841",
"x-accepted-oauth-scopes": "repository:write",
"connection": "close",
"x-usage-input-ops": "0",
"x-request-count": "3833",
"x-frame-options": "SAMEORIGIN",
"x-version": "12fbe411e51b",
"content-length": "123"
} If I try to send the request to netcat by changing the url in electron-configs to http://localhost:8888 I get errors about the self signed certificate. This should not happen for {
hostname: 'localhost',
path: '/',
protocol: 'http:',
method: 'POST',
headers: {
Host: 'localhost:8888',
Accept: '*/*',
'Content-Length': 411,
'content-type': 'multipart/form-data; boundary=--------------------------375342512532304869982889',
authorization: 'Basic ***REDACTED***',
'User-Agent': 'electron-builder',
'Cache-Control': 'no-cache'
}
}
----------------------------375342512532304869982889
Content-Disposition: form-data; name="files"; filename="latest-linux.yml"
Content-Type: text/yaml
version: 21.9.1-1630525623110
files:
- url: TESTAPP-21.9.1-1630525623110.AppImage
sha512: VD+S1hnQ5KQOQOr5OcugG2Q6oJ5fX4YJB5p/q8U741exMw5m9ifD6A84n6+T5HimEVlNoOt8LNLtoaAJzXuQgA==
size: 84799039
blockMapSize: 89683
path: TESTAPP-21.9.1-1630525623110.AppImage
sha512: VD+S1hnQ5KQOQOr5OcugG2Q6oJ5fX4YJB5p/q8U741exMw5m9ifD6A84n6+T5HimEVlNoOt8LNLtoaAJzXuQgA==
releaseDate: '2021-09-01T19:47:10.117Z'
----------------------------375342512532304869982889--
Pipe from Transform
FORM SENT
Pipe from Transform
Error: self signed certificate
at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)
at TLSSocket.emit (events.js:315:20)
at TLSSocket._finishInit (_tls_wrap.js:932:8)
at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12) {
code: 'DEPTH_ZERO_SELF_SIGNED_CERT'
}
Status Code: undefined
Message: self signed certificate
Done in 7.56s. |
|
Do you have a sample branch that I could checkout and can you provide any instructions for me to get set up locally? This is a bit outside of my league, per se, but maybe I can debug it locally and help you out in an easier manner. |
|
@mmaietta I haven't forked this project yet. I'm currently only using 'custom' to validate the publisher logic in our own application repo. The custom code just gets dropped into the // in electron-builder.config.js
//...
publish: [
{
provider: "custom",
publisher: "Bitbucket.org",
channel: "latest",
auth: 'Basic ***REDACTED***',
url: `https://api.bitbucket.org/2.0/repositories/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}/downloads`
// url: 'http://localhost:8888/'
},
],
// ...You'll need to create an app password token for your personal account in Bitbucket to replace the Basic Auth header This is where I'm at currently. This code is in a file called const { basename } = require('path');
const { Transform } = require('stream');
const mime = require('mime');
const FormData = require('form-data');
const { Arch } = require('builder-util');
const { HttpPublisher } = require('electron-publish/out/publisher');
const { httpExecutor } = require('builder-util/out/nodeHttpExecutor');
const { configureRequestOptions } = require('builder-util-runtime');
const { stat } = require('fs-extra');
class CustomPublisher extends HttpPublisher {
constructor(context, info) {
super(context, false);
console.log(info);
this.options = info;
}
get providerName() {
return this.options.publisher || 'Custom Publisher';
}
// Create string for cli output publisher type build step
toString() {
const version = this.context.packager.appInfo.buildVersion;
return `Custom (publish: ${new URL(this.options.url).host}, version: ${version})`;
}
// Override base class upload function
async upload(task) {
const fileName =
(this.useSafeArtifactName ? task.safeArtifactName : null) ||
basename(task.file);
if (task.fileContent != null) {
console.log('upload with content');
const form = new FormData();
form.append(
'files',
task.fileContent,
{
filename: fileName,
contentType: mime.getType(fileName),
header: `\r\n--${form.getBoundary()}\r\n` +
`Content-Disposition: form-data; name="files"; filename="${fileName}"\r\n` +
`Content-Type: ${mime.getType(fileName)}\r\n`,
},
);
this.options.headers = { ...this.options.headers, ...form.getHeaders() };
console.log(form);
const passthrough = new Transform({
transform: (chunk, encoding, cb) => {
console.log(chunk.toString());
cb(null, chunk);
},
});
await this.doUpload(
fileName,
task.arch || Arch.x64,
task.fileContent.length,
(request, reject) => {
request.on('end', () => console.log('ALL DONE'));
request.on('pipe', (src) => console.log('Pipe from', src.constructor.name));
request.on('error', (e) => {console.error(e); reject(e); });
passthrough.on('end', () => { console.log('FORM SENT'); request.end(); });
form.pipe(passthrough).pipe(request, {end: false});
// form.pipe(request);
},
);
return;
}
return
// const fileStat = await stat(task.file);
// const progressBar = this.createProgressBar(fileName, fileStat.size);
// console.log('upload from filepath')
// await this.doUpload(
// fileName,
// task.arch || Arch.x64,
// fileStat.size,
// (request, reject) => {
// if (progressBar != null) {
// // reset (because can be called several times (several attempts)
// progressBar.update(0);
// }
// const form = new FormData();
// form.append(
// 'files',
// task.fileContent,
// {
// filename: fileName,
// contentType: mime.getType(fileName),
// header: `\r\n--${form.getBoundary()}\r\n` +
// `Content-Disposition: form-data; name="files"; filename="${fileName}"\r\n` +
// `Content-Type: ${mime.getType(fileName)}\r\n`,
// },
// );
// return this.createReadStreamAndProgressBar(task.file, fileStat, progressBar, reject).pipe(request);
// },
// );
}
async doUpload(fileName, arch, dataLength, requestProcessor) {
const headers = this.options.headers || {};
const url = new URL(this.options.url);
const opts = configureRequestOptions(
{
hostname: url.hostname,
path: url.pathname,
protocol: url.protocol,
method: 'POST',
headers: {
'Host': url.host,
'Accept': '*/*',
// 'X-File-Name': fileName,
// 'Content-Type': mime.getType(fileName) || 'application/octet-stream',
'Content-Length': dataLength,
...headers,
},
},
this.options.auth,
);
console.log(opts)
return await httpExecutor
.doApiRequest(opts, this.context.cancellationToken, requestProcessor)
.catch(e => {
console.error(`Status Code: ${e.statusCode}`);
console.error(`Message: ${e.message}`);
});
}
}
module.exports = {
default: CustomPublisher,
}; |
|
From what I've read, don't set the headers, it's automatically created through [EDIT] I was able to get it publishing + integration test passing. Here's the draft PR #6228 |
|
Added Auto-Updater functionality, |
|
@mmaietta Awesome!! YOU ROCK FOR THIS!! But what did you mean for updates using Bearer token? Consumers of this new Provider in general or our specific use case? We chose to use Bitbucket App tokens over oauth flow because the electron app is distributed internally only but to many workstations. We can control revoke and re-creation an app token for distribution without bothering end-users to authorize updates for electron app. We will just do it for them any time there is a update published. But does that effect community users of this Publisher if they would rather use OAuth and Bearer tokens? Also, whats the approval workflow for this project? Who reviews the pull requests? |
|
Oh, none to worry about then. You're welcome to choose whatever auth mechanism you want. Just need to set the Authorization header via It's just develar and myself as maintainers in terms of workflow. I always do encourage community contributors and PR reviewers though 🙂 |
|
@mmaietta Thanks so much for assistance with this! As my workload reduces I can totally stay more on top of this project and potentially chip in where I can now that you have helped orient me to this code. So what needs to happen to get this merged and published to npm? Anything from me? |
|
Merged. Released as part of electron-builder 22.14.0 and electron-updater 4.6.0 @ next |
|
Seems like there is a dependency issue on dmg-license package for linux on that release of [1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info fsevents@2.3.2: The platform "linux" is incompatible with this module.
info "fsevents@2.3.2" is an optional dependency and failed compatibility check. Excluding it from installation.
error dmg-license@1.0.9: The platform "linux" is incompatible with this module.
error Found incompatible module.
|
|
Should be fixed via #6244. Can you try with 22.14.1 now? |
|
@mmaietta confirming on Linux still receiving that same error with 22.14.1 |
|
I've deprecated 22.14.0 and 22.14.1 to prevent impact on other users. Working on fixing build asap. Seems there's some mixup with optional dependencies |
|
Please try the |
|
@mmaietta Well, the dependencies are good now. No more errors on forced packages. But getting 401 unauthorized on the Bitbucket API POST request. I have |
|
Hmmm are you just using an App Password? That's supposed to be the BITBUCKET_TOKEN, no transforms or "Basic" prefix modifications. electron-builder/test/src/ArtifactPublisherTest.ts Lines 145 to 153 in a9ec90d |
Yup the value I have stored in the Env variable is just a 20 char string of alphanumerics. No prefixes. And my config options objects looks the same as your test except i specify a |
|
@mmaietta Also, do you think we should supply a export class BitbucketPublisher extends HttpPublisher {
readonly providerName = "bitbucket"
readonly hostname = this.info.hostname || "api.bitbucket.org" // <-- might be a self hosted domain
private readonly info: BitbucketOptions
private readonly auth: string
private readonly basePath: string
constructor(context: PublishContext, info: BitbucketOptions) {
super(context)
// So this instead of this
// const token = process.env.BITBUCKET_TOKEN
const token = this.info.token // <-- use this instead
// Or at least check the config value first then default to checking for ENV.
const token = (this.info.token || process.env.BITBUCKET_TOKEN) || null
if (isEmptyOrSpaces(token)) {
throw new InvalidConfigurationError(`Bitbucket token is not set in publisher config options or environment variable 'BITBUCKET_TOKEN' (see https://www.electron.build/configuration/publish#BitbucketOptions)`)
}
this.info = info
this.auth = BitbucketPublisher.convertAppPassword(this.info.owner, token)
this.basePath = `/2.0/repositories/${this.info.owner}/${this.info.slug}/downloads`
}That way we can transparently set it in an config file {
provider: "bitbucket",
channel: "latest",
// hostname: 'api.self-hosted-domain.com', (optionally)
token: BITBUCKET_TOKEN,
owner: BITBUCKET_REPO_OWNER,
slug: BITBUCKET_REPO_SLUG,
}, |
Can you confirm what permissions you have set on the token? I think all I have on mine are Re: Token via Config files The whole point of having it as an env var is so that security vulnerabilities aren't inadvertently created by unsuspecting users who have:
electron-builder/packages/builder-util-runtime/src/publishOptions.ts Lines 108 to 111 in f4c2a19 Even the comment for github says not to store it in the config. Rather, it was provided as a manner with which to programmatically set the token when using setFeedUrl. IMO, the token property should be removed and we require all users to utilize AppUpdater::addAuthHeader instead for setting the token, but that's a breaking semver change and might be unnecessary for the interim.
Basically, the blanket rule is to not commit tokens in a config or env file. Side note, I like the idea of allowing a self-hosted Bitbucket domain to be provided. Great callout. Happy to review a PR if you're interested in adding it 😉 |
Yes the token I have has proper permissions (I just re verified this moment) both a request with
I politely disagree that there is a very good reason to have the use case as in the GithubOptions object. Yes, I acknowledge and agree that committing configs with secrets is irresponsible. Which is why I use a factory function to compose the configs with Env variables and deep freeze such objects at runtime. My configs are NEVER added to any repository with secrets and this is not the use case I'm suggesting. The builder cmd allows for a None of what I've suggested should imply committing token values (or any secrets) to a repository directly. If a developer consuming this library does that!? it's on them and their team, not the fault of library code or the authors. Having the choice to use the field value in programmatic files allows a project to communicate to other team members that a value dependency needs to be there without having to dive into library source or rely on error message instructions from library source. Anyone on the team can see its usage inside the config. |
|
@mmaietta Where is this new code located in the packages? I cannot find the bitbucket provider or any of the new stuff. Also did testing exporting the value directly and still not working. I have verified this particular token is valid, has admin rights, and certain it works on all GET|POST|PUT|DELETE requests. This has been tested with Postman and curl. # 201 in this case BITBUCKET_TOKEN is prehashed using user:passkey
[ $BITBUCKET_TOKEN == `echo -n "${OWNER}:${APP-TOKEN}" | base64` ] # true
# comes out to something like SkJlbGxlcstf45gsdfkROTM2c0xTZkRuS1FqNEo=
BITBUCKET_TOKEN={{token_value}}; curl -iX GET https://api.bitbucket.org/2.0/repositories/{{OWNER}}/{{SLUG}}/downloads -H "Authorization: Basic ${BITBUCKET_TOKEN}
HTTP/2 201
server: nginx
vary: Authorization, Origin
cache-control: no-cache, no-store, must-revalidate, max-age=0
content-type: text/html; charset=utf-8
...# this fails 401 -- In this case the BITBUCKET_TOKEN is just the passkey value direct from bitbucket since your code uses the helper static to hash the string.
BITBUCKET_TOKEN={{token_value}}; yarn electron-build --config electron-builder.config.js --publish always
⨯ 401 Unauthorized
"method: POST url: https://api.bitbucket.org/2.0/repositories/[[REDACTED_OWNER]]/[[REDACTED_SLUG]]/downloads\n\n Data:\n \n "
Headers: {
"server": "nginx",
"vary": "Origin",
"www-authenticate": "Basic realm=\"Bitbucket.org HTTP\"",
"cache-control": "no-cache, no-store, must-revalidate, max-age=0",
"content-type": "text/html; charset=utf-8",
"x-b3-traceid": "5237834340b72510",
"x-usage-output-ops": "0",
"x-dc-location": "Micros",I think something is wrong in the final URL string that gets used in the request. 🤷 Regarding the token as an optional config input field... Another use case would be a very strict corporate setting with very tight security policies. Development teams may not have access to actual values to export and so they might need to employ client code from a platform service such as Google::Secrets Manger or AWS::Hey Management Service or Hashicorp::Vault. In those cases, only the client code would have access to the token key at runtime so a config script would need to inject the private token value that way. If this all happens during a CI pipeline it makes the |
|
@jbool24 it's in the latest electron-builder currently pulls the Maybe I should rename the env vars to Also, I'm not sure I understand your description of
|
Yes I understood that and I think it makes sense to leave the name as you have it. I was explaining above that I had to hash the user:token myself in order to test with
I'm requesting that we keep it (and add optional 'token' field to the schema for BitbucketOptions) so that Enterprise consumers who have a specific use-case where all secrets must be stored in Key Management Services that are controlled by other departments outside the development team can use client-code inside a config.js file and let the client code supply the token via options that way. Does that make sense? Its a very specific scenario but exactly the scenario we have 😄 In Pre-Production CI, I have no control over what 3rd party tokens/keys were created and stored (SecOps does this) so we let the config script pull in the value using Google/AWS SDKs to pass the returned key into our pre-build operation. We would need the line below added to the BitbucketPublisher as well as the field added to schema validator template to allow for this use-case. This is exactly how I provided a token for GithubPublisher before migrating our repository to bitbucket. // BitbucketPublisher.ts @ line 20
const token = (this.info.token || process.env.BITBUCKET_TOKEN) || null |
|
@mmaietta So, can we get line 20 changed to the previous mentioned? // BitbucketPublisher.ts @ line 20 and 21
const token = (this.info.token || process.env.BITBUCKET_TOKEN) || null
const username = (this.info.username || process.env.BITBUCKET_USERNAME) || nullI hope its understood what this would allow me to do in configs. What I would like to have the flexibility to do is run > electron-builder --config=electron-builder.config.js -p alwayswith the config as below // electron-config.js
const package = require("./package.json");
const now = new Date();
const buildVersion = `${now.getFullYear() - 2000}.${now.getMonth() +
1}.${now.getDate()}-${now.getTime()}`;
const BITBUCKET_API_USER = process.env.BITBUCKET_API_USERNAME;
const BITBUCKET_API_SECRET = process.env.BITBUCKET_API_PASSWORD; // <-- notice still secret
// but user defined env var name. Not forced to use name BITBUCKET_TOKEN
const BITBUCKET_REPO_OWNER = process.env.BITBUCKET_REPO_OWNER;
const BITBUCKET_REPO_SLUG = process.env.BITBUCKET_REPO_SLUG;
/**
* @type {import('electron-builder').Configuration}
* @see https://www.electron.build/configuration/configuration
*/
const config = {
appId: `com.example.${package.name}`,
productName: "CoolApp",
copyright: `Copyright © year ${new Date().getFullYear()}`,
publish: [
{
provider: "bitbucket",
channel: "latest",
owner: BITBUCKET_REPO_OWNER,
slug: BITBUCKET_REPO_SLUG,
token: BITBUCKET_API_SECRET, //<-- I would rather do this. Now I could use a user defined env var name
username: BITBUCKET_API_USER, //<--- Here Too
},
],
//... rest of the config
}This setup makes the builder config (publish settings) transparent when someone else on the team needs to see what is required in the ENV for CI builder and Orchestrated Deployments. ALSO, this way I now would have the ability to use a Secrets Manager service client library in cloud CI/CD environments (AWS, Google, Azure) |
|
Fair points. Happy to accept a PR 🙂 |
|
@mmaietta OK. I'm also going to toss in one more thing to the PR. Using |
|
@mmaietta still some issues on the Updater runtime side. Getting the following: Error: Error: Unable to find latest version on Bitbucket (owner: {{REDACTED}}, slug: {{REDACTED}}, channel: latest), please ensure release exists: HttpError: 403
"method: GET url: https://api.bitbucket.org/2.0/repositories/{{REDACTED}}/{{REDACTED}}/downloads/latest-linux.yml?noCache=1fs50trpr
Data:
Access denied. You must have write or admin access." Looks like we need that ApiKey bundled with the other values in app-update.yml in order for the Provider to use on GET request. How does the Github provider handle this since it also needs the key at client runtime? |
|
Hi @jbool24 , apologies for the delayed response. Regarding the token. You can add an authentication header to the appUpdater via electron-builder/packages/electron-updater/src/AppUpdater.ts Lines 104 to 111 in 86e6d15 Usage in the docs: https://github.com/electron-userland/electron-builder/blob/b01d5225631115f6f301cb113b044fd10ebb5256/docs/auto-update.md#custom-options-instantiating-updater-directly |
|
@mmaietta Awesome I can confirm that worked. Thank you for that. BUT now I get another error because the Bitbucket API redirects to an Amazon s3 bucket for object storage and there seems to be a problem with the underlying request made by the Updater to follow the redirect to s3. The link that prints out in my console does works if I open in a browser so I can confirm that its a valid link and I can download the latest.yml file from Bitbucket's s3 bucket. But I get something like below at runtime during the Update check flow... Again, if I manually follow that link that prints it does download that file so it does exist. Any Ideas?? |
|
Maybe there are some headers missing? I don't have a test environment to work with currently. Can you trace the network request and compare web browser vs electron-updater's request? |
|
@mmaietta not sure how to capture the trace from electron-updater. How can I do that? I can do |
|
@mmaietta Any Ideas here? I don't think whatever client electron-updater is using is passing the Auth headers on through the 301 redirect to AWS s3. This is the only trace I can see from the console /tmp/.mount_PushPaVnJtgc/resources/app.asar/packages/main/dist/updater.js [UPDATER] Provider Options: {
provider: 'bitbucket',
owner: '{{REDACTED}}',
slug: '{{REDACTED}},
channel: 'latest',
requestHeaders: { Authorization: 'Basic {{REDACTED}}' }
}
# ...
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InvalidArgument</Code><Message>Only one auth mechanism allowed; only the X-Amz-Algorithm query parameter, Signature query string parameter or the Authorization header should be specified</Message><ArgumentName>Authorization</ArgumentName><ArgumentValue>Basic {{REDACTED}}</ArgumentValue><RequestId>XFZ7KE661TCYTGD0</RequestId><HostId>CneBQ7oyQIk9PciPQsWzuF65kU5DpY1y6eM7OjL3W5ayRaEu0XrQ5g0D61RlQ8ZMmmeA2BDEoJE=</HostId></Error>\n "
Headers: {
"x-amz-request-id": "XFZ7KE661TCYTGD0",
"x-amz-id-2": "CneBQ7oyQIk9PciPQsWzuF65kU5DpY1y6eM7OjL3W5ayRaEu0XrQ5g0D61RlQ8ZMmmeA2BDEoJE=",
"content-type": "application/xml",
"transfer-encoding": "chunked",
"date": "Fri, 25 Feb 2022 15:58:38 GMT",
"server": "AmazonS3",
"connection": "close"
}
at ye (/tmp/.mount_PushPaVnJtgc/resources/app.asar/packages/main/dist/updater.js:1:9819)
at IncomingMessage.<anonymous> (/tmp/.mount_PushPaVnJtgc/resources/app.asar/packages/main/dist/updater.js:1:12648)
at IncomingMessage.emit (node:events:394:28)
at endReadableNT (node:internal/streams/readable:1343:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
at Object.e.newError (/tmp/.mount_PushPaVnJtgc/resources/app.asar/packages/main/dist/updater.js:2:15129)
at ms.getLatestVersion (/tmp/.mount_PushPaVnJtgc/resources/app.asar/packages/main/dist/updater.js:2:109242)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async uc.getUpdateInfoAndProvider (/tmp/.mount_PushPaVnJtgc/resources/app.asar/packages/main/dist/updater.js:2:122415)
at async uc.doCheckForUpdates (/tmp/.mount_PushPaVnJtgc/resources/app.asar/packages/main/dist/updater.js:2:122701)
|
|
This looks like a solid culprit
Also, I think you may have left an additional auth token in the error response still needed to be edited out ( My best guess is that the authorization header is supposed to be removed on redirects? |
Thanks 🙏! Removed although these tokens are not for production and get cleared often. 😄 Hmm so what do you suggest I can do to fudge around with the updater? Can you point me to where I might dig in to source code. There is a lot of inheritance happening spread across modules withing electron-builder so I'm having a little difficulty sourcing the responsible functionality 😵💫 I have to figure out how to make the updater work with bitbucket asap to complete deployment of a project so any help you can provide is crazy appreciated 🙏 🙏 🙏 |
|
This is how the auth header is being added in the integration test: electron-builder/test/src/updater/nsisUpdaterTest.ts Lines 64 to 74 in bf0382e Could you confirm that you're converting your app password before adding it to the header? The integration test passes for me with my bitbucket token for the updater's |
|
@mmaietta yes I can confirm the password is encoded as Basic Auth base64 string. Right now for testing in fact I have the string already converted and hard-coded in. const options = {
provider: 'bitbucket',
owner: import.meta.env.VITE_BITBUCKET_REPO_OWNER,
slug: import.meta.env.VITE_BITBUCKET_REPO_SLUG,
channel: 'latest',
requestHeaders: {
Authorization: 'Basic {{REDACTED}}',
},
} as BitbucketOptions;Where what has been redacted looks like let autoUpdater: AppUpdater;
const options = {
provider: 'bitbucket',
owner: import.meta.env.VITE_BITBUCKET_REPO_OWNER,
slug: import.meta.env.VITE_BITBUCKET_REPO_SLUG,
channel: 'latest',
} as BitbucketOptions;
console.log(`${__filename} [UPDATER] Provider Options: `, options);
if (process.platform === 'win32') {
autoUpdater = new NsisUpdater(options);
} else if (process.platform === 'darwin') {
autoUpdater = new MacUpdater(options);
} else {
autoUpdater = new AppImageUpdater(options);
}
// Again using the hardcoded for testing but will change to the class method `convertAppPassword` for PROD
autoUpdater.addAuthHeader('Basic {{REDACTED}}'); <-- should this function be used instead of passing in args to contructor?
//...
|
I think technically both routes would work. Passing an auth header electron-builder/packages/electron-updater/src/AppUpdater.ts Lines 194 to 196 in 6fcd477 electron-builder/packages/electron-updater/src/AppUpdater.ts Lines 108 to 110 in 6fcd477 I'm legitimately confused on this. I can't even find where the error is being through, as it'd be great to have the error also log what the URL it's trying to download from. I'm wondering if the full download URL is just the blockmap URL with Side note: I think you can just use electron-builder/packages/electron-updater/src/main.ts Lines 18 to 30 in 6fcd477 |
|
Can you try applying this patch so that we can get more data on where it's failing? |
Updated my code to your suggestion above. After patching electron-builder, I did the following
yarn build && yarn electron-builder --config electron-builder.config.js -p always
This is the output at runtime > ./dist/{{REDACTED}}-22.3.2-1646245898359.AppImage
Loading Printer Module
[1321323:0302/134632.836679:ERROR:sandbox_linux.cc(376)] InitializeSandbox() called with multiple threads in process gpu-process.
Loading autoupdater
Checking for update
[1321287:0302/134634.091647:ERROR:nss_util.cc(286)] After loading Root Certs, loaded==false: NSS error code: -8018
Found version 22.3.2-1646246571369 (url: {{REDACTED}}-22.3.2-1646246571369.AppImage)
Downloading update from {{REDACTED}}-22.3.2-1646246571369.AppImage
updater cache dir: /home/justin/.cache/{{REDACTED}}-updater
{
version: '22.3.2-1646246571369',
files: [
{
url: '{{REDACTED}}-22.3.2-1646246571369.AppImage',
sha512: 'N4/h4doVl0diYq3qtQUEYbjYTxXYia/hB2fczLmynTSXdtylN8P/zqNX1Ncf/p9a1Kjbd6N+z+XnMS9mb2YxZg==',
size: 143058296,
blockMapSize: 148876
}
],
path: '{{REDACTED}}-22.3.2-1646246571369.AppImage',
sha512: 'N4/h4doVl0diYq3qtQUEYbjYTxXYia/hB2fczLmynTSXdtylN8P/zqNX1Ncf/p9a1Kjbd6N+z+XnMS9mb2YxZg==',
releaseDate: '2022-03-02T18:44:49.044Z'
}
Cached update sha512 checksum doesn't match the latest available update. New update must be downloaded. Cached: wIhWTc1w2DnfM11NWhE89w4lFDxg8zzzwaJPBj2ZMqii7nyNTnkK6+HuCONiSrszYGFosGw0BiFPgS8WomLLTA==, expected: N4/h4doVl0diYq3qtQUEYbjYTxXYia/hB2fczLmynTSXdtylN8P/zqNX1Ncf/p9a1Kjbd6N+z+XnMS9mb2YxZg==. Directory for cached update will be cleaned
{
headers: {
accept: '*/*',
authorization: 'Basic {{REDACTED}}',
'User-Agent': 'electron-builder',
'Cache-Control': 'no-cache'
},
protocol: 'https:',
hostname: 'api.bitbucket.org',
path: '/2.0/repositories/{{REDACTED}}/{{REDACTED}}/downloads/{{REDACTED}}-22.3.2-1646246571369.AppImage'
}
Cannot download differentially, fallback to full download: HttpError: 400 Bad Request
Headers: {
"x-amz-request-id": "{{REDACTED}}",
"x-amz-id-2": "{{REDACTED}}",
"content-type": "application/xml",
"transfer-encoding": "chunked",
"date": "Wed, 02 Mar 2022 18:46:34 GMT",
"server": "AmazonS3",
"connection": "close"
}
at Object.zl (/tmp/.mount_PushPaDhLcjI/resources/app.asar/packages/main/dist/index.cjs:8:200550)
at Object.Wy [as checkIsRangesSupported] (/tmp/.mount_PushPaDhLcjI/resources/app.asar/packages/main/dist/index.cjs:9:132644)
at ClientRequest.<anonymous> (/tmp/.mount_PushPaDhLcjI/resources/app.asar/packages/main/dist/index.cjs:9:140536)
at ClientRequest.emit (node:events:394:28)
at SimpleURLLoaderWrapper.<anonymous> (node:electron/js2c/browser_init:101:6816)
at SimpleURLLoaderWrapper.emit (node:events:394:28)
New version 22.3.2-1646246571369 has been downloaded to /home/justin/.cache/{{REDACTED}}-updater/pending/{{REDACTED}}-22.3.2-1646246571369.AppImage
Auto install update on quit
Install: isSilent: true, isForceRunAfter: false
./dist/{{REDACTED}}-22.3.2-1646245898359.AppImage 8.90s user 3.58s system 2% cpu 8:21.50 total
I think it worked but I do not see any of those logging statements patched in. Does that mean that code never executed? Is that a problem? |
|
Maybe this needs to be added in order for the logging to appear? |
Electron Version: 11.4.10
Electron Type (current, beta, nightly):
Which version of electron-updater are you using (if applicable)?
electron-updater@4.3.9
Issues:
The 'custom' publisher also completes without reporting errors but does not upload the file. The Basic Auth credentials were tested with curl and worked correctly. But the files do not upload even though electron-publish reports it did.
Both the custom Publisher and the Updater setup are below...
Here are the configs for setting up the Updater in the main process
Here is the publisher for build time
**NOTE: I would be willing to sponsor updating of docs for more clear and correct instructions on custom extending Publisher and Updater if in fact this is not an issue and the custom features are actually working and this is my development error for incorrectly implementing.
The text was updated successfully, but these errors were encountered: