New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixes source token expiration by token refresh #5198
Changes from 4 commits
236446f
08f87c0
6409aba
0935361
af6a82d
f3b4729
101425e
303bc30
3c5f2a8
493e215
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -1,37 +1,64 @@ | ||||||||||||
import { FirebaseError } from "../../../error"; | ||||||||||||
import { logger } from "../../../logger"; | ||||||||||||
|
||||||||||||
type TokenFetchState = "NONE" | "FETCHING" | "VALID"; | ||||||||||||
|
||||||||||||
/** | ||||||||||||
* GCF v1 deploys support reusing a build between function deploys. | ||||||||||||
* This class will return a resolved promise for its first call to tokenPromise() | ||||||||||||
* and then will always return a promise that is resolved by the poller function. | ||||||||||||
*/ | ||||||||||||
export class SourceTokenScraper { | ||||||||||||
private firstCall = true; | ||||||||||||
private resolve!: (token: string) => void; | ||||||||||||
private tokenValidDurationMs; | ||||||||||||
private resolve!: (token?: string) => void; | ||||||||||||
private promise: Promise<string | undefined>; | ||||||||||||
private expiry: number | undefined; | ||||||||||||
private fetchState: TokenFetchState; | ||||||||||||
|
||||||||||||
constructor() { | ||||||||||||
constructor(validDurationMs = 1500000) { | ||||||||||||
this.tokenValidDurationMs = validDurationMs; | ||||||||||||
this.promise = new Promise((resolve) => (this.resolve = resolve)); | ||||||||||||
this.fetchState = "NONE"; | ||||||||||||
} | ||||||||||||
|
||||||||||||
async getToken(): Promise<string | undefined> { | ||||||||||||
if (this.fetchState === "NONE") { | ||||||||||||
this.fetchState = "FETCHING"; | ||||||||||||
return undefined; | ||||||||||||
} else if (this.fetchState === "FETCHING") { | ||||||||||||
return this.promise; // wait until we get a source token | ||||||||||||
} else if (this.fetchState === "VALID") { | ||||||||||||
if (this.isTokenExpired()) { | ||||||||||||
this.fetchState = "FETCHING"; | ||||||||||||
this.promise = new Promise((resolve) => (this.resolve = resolve)); | ||||||||||||
return undefined; | ||||||||||||
} | ||||||||||||
return this.promise; | ||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we throw an error here instead of silently succeeding? Can see argument for either. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. By throwing an error, do you mean throwing an error and then catching it in an enclosing There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I mean that in theory we should never reach this codepath - fetchState can only be in one of three states, and we have if condition for all of them. So we could throw (which could happen if we misunderstood this code for somee reason) or just return the promise (which might leave the CLI in an undefined state). I have slight preference for the former. |
||||||||||||
} | ||||||||||||
} | ||||||||||||
|
||||||||||||
// Token Promise will return undefined for the first caller | ||||||||||||
// (because we presume it's this function's source token we'll scrape) | ||||||||||||
// and then returns the promise generated from the first function's onCall | ||||||||||||
tokenPromise(): Promise<string | undefined> { | ||||||||||||
if (this.firstCall) { | ||||||||||||
this.firstCall = false; | ||||||||||||
return Promise.resolve(undefined); | ||||||||||||
isTokenExpired(): boolean { | ||||||||||||
if (this.expiry === undefined) { | ||||||||||||
throw new FirebaseError("failed to check expiry: no token exists"); | ||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This error message makes sense to you and me but won't to almost all users trying to deploy their function. I think this is a good example of a user facing error message - describe the situation in coarsely and suggest next steps: firebase-tools/src/deploy/functions/prepareFunctionsUpload.ts Lines 36 to 40 in 2ebefe2
In our case, hitting this if statement should never happen and would be considered a bug. Maybe we can suggest that they file a issue. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ah gotcha. working on a more descriptive message. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
What about something like this? |
||||||||||||
} | ||||||||||||
return this.promise; | ||||||||||||
return Date.now() >= this.expiry; | ||||||||||||
} | ||||||||||||
|
||||||||||||
get poller() { | ||||||||||||
return (op: any) => { | ||||||||||||
console.log(`${op.response.name}: polling`) | ||||||||||||
taeold marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||
if (op.metadata?.sourceToken || op.done) { | ||||||||||||
const [, , , /* projects*/ /* project*/ /* regions*/ region] = | ||||||||||||
op.metadata?.target?.split("/") || []; | ||||||||||||
logger.debug(`Got source token ${op.metadata?.sourceToken} for region ${region as string}`); | ||||||||||||
console.log( | ||||||||||||
taeold marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||
`${op.response.name}: Got source token ${op.metadata?.sourceToken} for region ${ | ||||||||||||
region as string | ||||||||||||
}` | ||||||||||||
); | ||||||||||||
this.resolve(op.metadata?.sourceToken); | ||||||||||||
this.fetchState = "VALID"; | ||||||||||||
this.expiry = Date.now() + this.tokenValidDurationMs; | ||||||||||||
} | ||||||||||||
}; | ||||||||||||
} | ||||||||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,14 +1,30 @@ | ||
/** Measures the time taken from construction to the call to stop() */ | ||
export class Timer { | ||
private readonly start: bigint; | ||
private readonly start: bigint; // nanoseconds | ||
taeold marked this conversation as resolved.
Show resolved
Hide resolved
|
||
private readonly expiry?: bigint; // nanoseconds | ||
|
||
constructor() { | ||
/** | ||
* Constructor for Timer object. | ||
* @param expirationSeconds timer expiration in minutes | ||
*/ | ||
constructor(expirationMs?: number) { | ||
this.start = process.hrtime.bigint(); | ||
if (expirationMs !== undefined) { | ||
this.expiry = BigInt(expirationMs) * BigInt(1e6); | ||
} | ||
} | ||
|
||
stop(): number { | ||
expired(): boolean { | ||
const elapsedNanos = this.elapsedNanos(); | ||
return this.expiry !== undefined && elapsedNanos >= this.expiry; | ||
} | ||
|
||
elapsedNanos(): bigint { | ||
const stop = process.hrtime.bigint(); | ||
const elapsedNanos = stop - this.start; | ||
return Number(elapsedNanos / BigInt(1e6)); | ||
return stop - this.start; | ||
} | ||
|
||
stop(): number { | ||
return Number(this.elapsedNanos() / BigInt(1e6)); | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,23 +2,23 @@ import { expect } from "chai"; | |
|
||
import { SourceTokenScraper } from "../../../../deploy/functions/release/sourceTokenScraper"; | ||
|
||
describe("SourcTokenScraper", () => { | ||
describe("SourceTokenScraper", () => { | ||
it("immediately provides the first result", async () => { | ||
const scraper = new SourceTokenScraper(); | ||
await expect(scraper.tokenPromise()).to.eventually.be.undefined; | ||
await expect(scraper.getToken()).to.eventually.be.undefined; | ||
}); | ||
|
||
it("provides results after the firt operation completes", async () => { | ||
it("provides results after the first operation completes", async () => { | ||
const scraper = new SourceTokenScraper(); | ||
// First result comes right away; | ||
await expect(scraper.tokenPromise()).to.eventually.be.undefined; | ||
await expect(scraper.getToken()).to.eventually.be.undefined; | ||
|
||
let gotResult = false; | ||
const timeout = new Promise((resolve, reject) => { | ||
setTimeout(() => reject(new Error("Timeout")), 10); | ||
}); | ||
const getResult = (async () => { | ||
await scraper.tokenPromise(); | ||
await scraper.getToken(); | ||
gotResult = true; | ||
})(); | ||
await expect(Promise.race([getResult, timeout])).to.be.rejectedWith("Timeout"); | ||
|
@@ -31,14 +31,63 @@ describe("SourcTokenScraper", () => { | |
it("provides tokens from an operation", async () => { | ||
const scraper = new SourceTokenScraper(); | ||
// First result comes right away | ||
await expect(scraper.tokenPromise()).to.eventually.be.undefined; | ||
await expect(scraper.getToken()).to.eventually.be.undefined; | ||
|
||
scraper.poller({ | ||
metadata: { | ||
sourceToken: "magic token", | ||
target: "projects/p/locations/l/functions/f", | ||
}, | ||
}); | ||
await expect(scraper.tokenPromise()).to.eventually.equal("magic token"); | ||
await expect(scraper.getToken()).to.eventually.equal("magic token"); | ||
}); | ||
|
||
it("refreshes token after timer expires", async () => { | ||
const scraper = new SourceTokenScraper(10); | ||
await expect(scraper.getToken()).to.eventually.be.undefined; | ||
scraper.poller({ | ||
metadata: { | ||
sourceToken: "magic token", | ||
target: "projects/p/locations/l/functions/f", | ||
}, | ||
}); | ||
await expect(scraper.getToken()).to.eventually.equal("magic token"); | ||
const timeout = (duration: number): Promise<void> => { | ||
return new Promise<void>((resolve) => setTimeout(resolve, duration)); | ||
}; | ||
await timeout(50); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In the future, try to instrument code without sleeping in your test. You can inject a clock for example. |
||
await expect(scraper.getToken()).to.eventually.be.undefined; | ||
scraper.poller({ | ||
metadata: { | ||
sourceToken: "magic token #2", | ||
target: "projects/p/locations/l/functions/f", | ||
}, | ||
}); | ||
await expect(scraper.getToken()).to.eventually.equal("magic token #2"); | ||
}); | ||
|
||
it("concurrent requests for source token", async () => { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nice test!! |
||
const scraper = new SourceTokenScraper(); | ||
|
||
const promises = []; | ||
for (let i = 0; i < 3; i++) { | ||
promises.push(scraper.getToken()); | ||
} | ||
scraper.poller({ | ||
metadata: { | ||
sourceToken: "magic token", | ||
target: "projects/p/locations/l/functions/f", | ||
}, | ||
}); | ||
|
||
let successes = 0; | ||
const tokens = await Promise.all(promises); | ||
for (const tok of tokens) { | ||
if (tok === "magic token") { | ||
successes++; | ||
} | ||
} | ||
expect(tokens.includes(undefined)).to.be.true; | ||
expect(successes).to.equal(2); | ||
}); | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit - stray comment?