Fuse the calls to searchForIssuesUsingJql when possible + memoization + cleanups
#1456
Conversation
ypc-faros
changed the title
searchForIssuesUsingJql when possible + memoization + cleanups
| @@ -52,6 +54,9 @@ export interface JiraConfig extends AirbyteConfig { | |||
| readonly api_url?: string; | |||
| readonly api_key?: string; | |||
| readonly graph?: string; | |||
| readonly requestedStreams?: Set<string>; | |||
There was a problem hiding this comment.
feels like a leaky abstraction
There was a problem hiding this comment.
Yeah maybe I over optimized there...
Another way we can do this is use config.run_mode for this purpose
If RunMode.WebhookSupplement, faros_issues is NOT requested (since it's not offered)
Otherwise (Full), assume both faros_issues and faros_issue_pull_requests are requested
99/100 times syncs will run with all supported streams, so this optimization is valuable in a small number of cases (the user would have to purposely disable one of these streams)
There was a problem hiding this comment.
I like more the option of using the run_mode. We also avoid adding yet another config
| return issueTransformer.toIssue(item); | ||
| }, | ||
| 'issues' | ||
| ); | ||
| } | ||
|
|
||
| private memoizeIssue(item: any, jql: string) { |
There was a problem hiding this comment.
is there a way we can generalize this? We kinda have the same need for Sprint and Sprint Reports so wondering if we would also need to do the same for that
There was a problem hiding this comment.
Maybe..., let's chat offline
|
Splits the former
getIssuesinto 3 functions:getIssuesKeys: only fetches issuekey(s) => used only by thefaros_board_issuesstreamgetIssues: fetches all the fields needed for thefaros_issuesstream (if requested) and all the fields needed for thefaros_issue_pull_requests(if requested). This function also stores a projection of the issue (only the fields needed forfaros_issue_pull_requests) in memory (seenIssues). I decided to use my own data structure vs@Memoizedso that the memory usage was minimized.getIssuesCompact: fetches the fields needed forfaros_issue_pull_requests(from memory or from the api)Aside from the above, a couple of minor changes:
jql(s) forgetIssuesandgetIssuesCompactmatch, by storingstart_dateandend_datein the "global" config.