Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(operations): split the operations-per-run option between queries and mutations #501

Closed
20 changes: 17 additions & 3 deletions .github/workflows/test.yml
Expand Up @@ -7,14 +7,28 @@ on: # rebuild any PRs and main branch changes
- 'releases/*'

jobs:
build: # make sure build/ci work properly
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- run: |
npm ci
npm run all:ci
test: # make sure the action works on a clean machine without building
npm run build && npm run pack
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- run: |
npm ci
npm run lint:all
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- run: |
npm ci
npm run test:only-errors
e2e: # make sure the action works on a clean machine without building
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
Expand Down
32 changes: 19 additions & 13 deletions README.md
Expand Up @@ -53,7 +53,8 @@ Every argument is optional.
| [any-of-labels](#any-of-labels) | Only issues/PRs with ANY of these labels are checked | |
| [any-of-issue-labels](#any-of-issue-labels) | Override [any-of-labels](#any-of-labels) for issues only | |
| [any-of-pr-labels](#any-of-pr-labels) | Override [any-of-labels](#any-of-labels) for PRs only | |
| [operations-per-run](#operations-per-run) | Max number of operations per run | `30` |
| [query-operations-per-run](#query-operations-per-run) | Max number of query operations per run (GitHub read) | `30` |
| [mutation-operations-per-run](#mutation-operations-per-run) | Max number of mutation operations per run (GitHub write) | `200` |
| [remove-stale-when-updated](#remove-stale-when-updated) | Remove stale label from issues/PRs on updates | `true` |
| [remove-issue-stale-when-updated](#remove-issue-stale-when-updated) | Remove stale label from issues on updates/comments | |
| [remove-pr-stale-when-updated](#remove-pr-stale-when-updated) | Remove stale label from PRs on updates/comments | |
Expand Down Expand Up @@ -299,7 +300,7 @@ Override [any-of-labels](#any-of-labels) but only to process the pull requests t

Default value: unset

#### operations-per-run
#### query-operations-per-run

_Context:_
This action performs some API calls to GitHub to fetch or close issues and pull requests, set or update labels, add comments, delete branches, etc.
Expand All @@ -308,18 +309,23 @@ GitHub has a [rate limit](https://docs.github.com/en/rest/overview/resources-in-
This option helps you to stay within the GitHub rate limits, as you can use this option to limit the number of operations for a single run.

_Purpose:_
This option aims to limit the number of operations made with the GitHub API to avoid reaching the [rate limit](https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting).
This option limit the amount of API calls made to GitHub in order to fetch the issues and pull requests as well as comments.

Based on your project, your GitHub business plan and the date of the cron job you set for this action, you can increase this limit to a higher number.
If you are not sure which is the right value for you or if the default value is good enough, you could enable the logs and look at the end of the stale action.
If you reached the limit, you will see a warning message in the logs, telling you that you should increase the number of operations.
If you choose not to increase the limit, you might end up with unprocessed issues or pull requests after a stale action run.

When [debugging](#Debugging), you can set it to a much higher number like `1000` since there will be fewer operations made with the GitHub API.
Only the [actor](#repo-token) and the batch of issues (100 per batch) will consume the operations.

Default value: `30`

#### mutation-operations-per-run

Same as [query-operations-per-run](#query-operations-per-run) except that the purpose of this option is to limit the API calls to GitHub to add or remove labels, add comments, delete branches, etc.

When [debugging](#Debugging), you can set it to a much higher number like `5000` since the operations count will still works but no mutations will happen.

Default value: `200`

#### remove-stale-when-updated

Automatically remove the stale label when the issues or the pull requests are updated (based on [GitHub issue](https://docs.github.com/en/rest/reference/issues) field `updated_at`) or commented.
Expand Down Expand Up @@ -350,7 +356,7 @@ Default value: unset

A comma delimited list of labels to remove when a stale issue or pull request receives activity and has the [stale-issue-label](#stale-issue-label) or [stale-pr-label](#stale-pr-label) removed from it.

Warning: each label results in a unique API call which can drastically consume the limit of [operations-per-run](#operations-per-run).
Warning: each label results in a unique API call which can drastically consume the limit of [mutation-operations-per-run](#mutation-operations-per-run).

Default value: unset
Required Permission: `pull-requests: write`
Expand All @@ -370,7 +376,7 @@ Change the order used to fetch the issues and pull requests from GitHub:
- `true` is for ascending.
- `false` is for descending.

It can be useful if your repository is processing so many issues and pull requests that you reach the [operations-per-run](#operations-per-run) limit.
It can be useful if your repository is processing so many issues and pull requests that you reach the [query-operations-per-run](#query-operations-per-run) limit.
Based on the order, you could prefer to focus on the new content or on the old content of your repository.

Default value: `false`
Expand Down Expand Up @@ -734,18 +740,18 @@ There are many logs, so this can be very helpful!
**Statistics:**
If the logs are enabled, you can also enable the statistics log which will be visible at the end of the logs once all issues were processed.
This is very helpful to have a quick understanding of the whole stale workflow.
Set `enable-statistics` to `true` in your workflow configuration file.
Set [enable-statistics](#enable-statistics) to `true` in your workflow configuration file.

**Dry-run:**
You can run this action in debug only mode (no actions will be taken on your issues and pull requests) by passing `debug-only` to `true` as an argument to the action.
You can run this action in debug only mode (no actions will be taken on your issues and pull requests) by passing [debug-only](#debug-only) to `true` as an argument to the action.

**More operations:**
You can increase the maximum number of operations per run by passing `operations-per-run` to `1000` for example which will help you to handle more operations in a single stale workflow run.
If the `debug-only` option is enabled, this is very helpful because the workflow will (almost) never reach the GitHub API rate, and you will be able to deep-dive into the logs.
You can increase the maximum number of operations per run by passing [query-operations-per-run](#query-operations-per-run) and [mutation-operations-per-run](#mutation-operations-per-run) to `1000` for example which will help you to handle more operations in a single stale workflow run.
If the [debug-only](#debug-only) option is enabled, this is very helpful because the workflow will still count and limit the number of operations per run to avoid to reach the GitHub API rate, but will not perform the calls.

**Job frequency:**
You could change the cron job frequency in the stale workflow to run the stale workflow more often.
Usually, this is not very helpful though.
Usually, we recommended running the workflow once per day nonetheless you could use a more frequent job and use a higher [query-operations-per-run](#query-operations-per-run) count to process all the issues in your repository and a lower [mutation-operations-per-run](#mutation-operations-per-run) count to only perform a few operations which reduce the risk in case of bad configuration.

### Contributing

Expand Down
3 changes: 2 additions & 1 deletion __tests__/constants/default-processor-options.ts
Expand Up @@ -24,7 +24,8 @@ export const DefaultProcessorOptions: IIssuesProcessorOptions = Object.freeze({
anyOfLabels: '',
anyOfIssueLabels: '',
anyOfPrLabels: '',
operationsPerRun: 100,
queryOperationsPerRun: 100,
mutationOperationsPerRun: 100,
debugOnly: true,
removeStaleWhenUpdated: false,
removeIssueStaleWhenUpdated: undefined,
Expand Down
44 changes: 22 additions & 22 deletions __tests__/operations-per-run.spec.ts
Expand Up @@ -17,9 +17,9 @@ describe('operations-per-run option', (): void => {
sut.staleIn(10).newIssue().updated(20);
});

describe('when the operations per run option is set to 1', (): void => {
describe('when the mutation operations per run option is set to 1', (): void => {
beforeEach((): void => {
sut.operationsPerRun(1);
sut.mutationOperationsPerRun(1);
});

it('should consume 1 operation (stale label)', async () => {
Expand All @@ -29,7 +29,7 @@ describe('operations-per-run option', (): void => {

expect(sut.processor.staleIssues).toHaveLength(1);
expect(
sut.processor.operations.getConsumedOperationsCount()
sut.processor.operations.getConsumedMutationOperationsCount()
).toStrictEqual(1);
});
});
Expand All @@ -40,9 +40,9 @@ describe('operations-per-run option', (): void => {
sut.staleIn(10).commentOnStale().newIssue().updated(20);
});

describe('when the operations per run option is set to 2', (): void => {
describe('when the mutation operations per run option is set to 2', (): void => {
beforeEach((): void => {
sut.operationsPerRun(2);
sut.mutationOperationsPerRun(2);
});

it('should consume 2 operations (stale label, comment)', async () => {
Expand All @@ -52,15 +52,15 @@ describe('operations-per-run option', (): void => {

expect(sut.processor.staleIssues).toHaveLength(1);
expect(
sut.processor.operations.getConsumedOperationsCount()
sut.processor.operations.getConsumedMutationOperationsCount()
).toStrictEqual(2);
});
});

// Special case were we continue the issue processing even if the operations per run is reached
describe('when the operations per run option is set to 1', (): void => {
describe('when the mutation operations per run option is set to 1', (): void => {
beforeEach((): void => {
sut.operationsPerRun(1);
sut.mutationOperationsPerRun(1);
});

it('should consume 2 operations (stale label, comment)', async () => {
Expand All @@ -70,7 +70,7 @@ describe('operations-per-run option', (): void => {

expect(sut.processor.staleIssues).toHaveLength(1);
expect(
sut.processor.operations.getConsumedOperationsCount()
sut.processor.operations.getConsumedMutationOperationsCount()
).toStrictEqual(2);
});
});
Expand All @@ -83,9 +83,9 @@ describe('operations-per-run option', (): void => {
sut.newIssue().updated(20);
});

describe('when the operations per run option is set to 3', (): void => {
describe('when the mutation perations per run option is set to 3', (): void => {
beforeEach((): void => {
sut.operationsPerRun(3);
sut.mutationOperationsPerRun(3);
});

it('should consume 4 operations (stale label, comment)', async () => {
Expand All @@ -95,14 +95,14 @@ describe('operations-per-run option', (): void => {

expect(sut.processor.staleIssues).toHaveLength(2);
expect(
sut.processor.operations.getConsumedOperationsCount()
sut.processor.operations.getConsumedMutationOperationsCount()
).toStrictEqual(4);
});
});

describe('when the operations per run option is set to 2', (): void => {
describe('when the mutation operations per run option is set to 2', (): void => {
beforeEach((): void => {
sut.operationsPerRun(2);
sut.mutationOperationsPerRun(2);
});

it('should consume 2 operations (stale label, comment) and stop', async () => {
Expand All @@ -112,15 +112,15 @@ describe('operations-per-run option', (): void => {

expect(sut.processor.staleIssues).toHaveLength(1);
expect(
sut.processor.operations.getConsumedOperationsCount()
sut.processor.operations.getConsumedMutationOperationsCount()
).toStrictEqual(2);
});
});

// Special case were we continue the issue processing even if the operations per run is reached
describe('when the operations per run option is set to 1', (): void => {
describe('when the mutation operations per run option is set to 1', (): void => {
beforeEach((): void => {
sut.operationsPerRun(1);
sut.mutationOperationsPerRun(1);
});

it('should consume 2 operations (stale label, comment) and stop', async () => {
Expand All @@ -130,7 +130,7 @@ describe('operations-per-run option', (): void => {

expect(sut.processor.staleIssues).toHaveLength(1);
expect(
sut.processor.operations.getConsumedOperationsCount()
sut.processor.operations.getConsumedMutationOperationsCount()
).toStrictEqual(2);
});
});
Expand Down Expand Up @@ -169,15 +169,15 @@ class SUT {
return this;
}

operationsPerRun(count: number): SUT {
mutationOperationsPerRun(count: number): SUT {
this._updateOptions({
operationsPerRun: count
mutationOperationsPerRun: count
});

return this;
}

async test(): Promise<number> {
async test(): Promise<void> {
return this._setTestIssueList()._setProcessor();
}

Expand All @@ -202,7 +202,7 @@ class SUT {
return this;
}

private async _setProcessor(): Promise<number> {
private async _setProcessor(): Promise<void> {
this.processor = new IssuesProcessorMock(
this._opts,
async p => (p === 1 ? this._testIssueList : []),
Expand Down
4 changes: 2 additions & 2 deletions __tests__/updates-reset-stale.spec.ts
Expand Up @@ -658,7 +658,7 @@ class SUT {
return this;
}

async test(): Promise<number> {
async test(): Promise<void> {
return this._setTestIssueList()._setProcessor();
}

Expand All @@ -683,7 +683,7 @@ class SUT {
return this;
}

private async _setProcessor(): Promise<number> {
private async _setProcessor(): Promise<void> {
this.processor = new IssuesProcessorMock(
this._opts,
async p => (p === 1 ? this._testIssueList : []),
Expand Down
8 changes: 6 additions & 2 deletions action.yml
Expand Up @@ -108,10 +108,14 @@ inputs:
description: 'Only pull requests with all of these labels are checked if stale. Defaults to `[]` (disabled) and can be a comma-separated list of labels. Override "only-labels" option regarding only the pull requests.'
default: ''
required: false
operations-per-run:
description: 'The maximum number of operations per run, used to control rate limiting (GitHub API CRUD related).'
query-operations-per-run:
description: 'The maximum number of query operations per run, used to control rate limiting (GitHub API CRUD related).'
default: '30'
required: false
mutation-operations-per-run:
description: 'The maximum number of mutation operations per run, used to control rate limiting (GitHub API CRUD related).'
default: '200'
required: false
remove-stale-when-updated:
description: 'Remove stale labels from issues and pull requests when they are updated or commented on.'
default: 'true'
Expand Down