Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): allow disabling parallel asset publishing #22579

Merged
merged 2 commits into from Oct 26, 2022

Conversation

rix0rrr
Copy link
Contributor

@rix0rrr rix0rrr commented Oct 20, 2022

Once again, any change to anything anywhere broke someone. In this particular case, parallel asset publishing (#19367, and in particular building) broke a use case where someone uses a tool to build assets that is not concurrency-safe. So -- now we need to make that configurable.

Command line:

cdk deploy --no-asset-parallelism

cdk.json:

{
  "assetParallelism": false
}

Environment variables:

export CDK_ASSET_PARALLELISM=false

All Submissions:

Adding new Unconventional Dependencies:

  • This PR adds new unconventional dependencies following the process described here

New Features

  • Have you added the new feature to an integration test?
    • Did you use yarn integ to deploy the infrastructure and generate the snapshot (i.e. yarn integ without --dry-run)?

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Once again, any change to anything anywhere broke someone. In this
particular case, parallel asset publishing (#19367, and in particular
building) broke a use case where someone uses a tool to build assets
that is not concurrency-safe. So -- now we need to make that
configurable.

Command line:

```
cdk deploy --no-asset-parallelism
```

cdk.json:

```
{
  "assetParallelism": false
}
```

Environment variables:

```
export CDK_ASSET_PARALLELISM=false
```
@rix0rrr rix0rrr requested a review from a team October 20, 2022 14:26
@rix0rrr rix0rrr self-assigned this Oct 20, 2022
@gitpod-io
Copy link

gitpod-io bot commented Oct 20, 2022

@github-actions github-actions bot added the p2 label Oct 20, 2022
@aws-cdk-automation aws-cdk-automation requested a review from a team October 20, 2022 14:27
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Oct 20, 2022
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

@rix0rrr rix0rrr added pr-linter/exempt-readme The PR linter will not require README changes pr-linter/exempt-test The PR linter will not require test changes pr-linter/exempt-integ-test The PR linter will not require integ test changes pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested labels Oct 24, 2022
@aws-cdk-automation aws-cdk-automation dismissed their stale review October 26, 2022 13:18

✅ Updated pull request passes all PRLinter validations. Dissmissing previous PRLinter review.

@mergify
Copy link
Contributor

mergify bot commented Oct 26, 2022

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 32d408a
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 69981ac into main Oct 26, 2022
@mergify mergify bot deleted the huijbers/no-parallel-assets branch October 26, 2022 14:08
@mergify
Copy link
Contributor

mergify bot commented Oct 26, 2022

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

mergify bot pushed a commit that referenced this pull request Nov 4, 2022
Version 2.48.0 introduced a flag to enable/disable parallelism when
building and publishing assets (see #22579 ). The flag is supposed to
be set via either environment variable (`CDK_ASSET_PARALLELISM`), command
line argument (`-[no]-asset-parallelism`), or the `cdk.json` file. However,
the last method does not work. We are not passing the option from that
file properly.

----

### All Submissions:

* [X] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Naumel pushed a commit that referenced this pull request Nov 4, 2022
Version 2.48.0 introduced a flag to enable/disable parallelism when
building and publishing assets (see #22579 ). The flag is supposed to
be set via either environment variable (`CDK_ASSET_PARALLELISM`), command
line argument (`-[no]-asset-parallelism`), or the `cdk.json` file. However,
the last method does not work. We are not passing the option from that
file properly.

----

### All Submissions:

* [X] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@@ -155,7 +155,8 @@ async function parseCommandLineArguments() {
"'true' by default, use --no-logs to turn off. " +
"Only in effect if specified alongside the '--watch' option",
})
.option('concurrency', { type: 'number', desc: 'Maximum number of simultaneous deployments (dependency permitting) to execute.', default: 1, requiresArg: true }),
.option('concurrency', { type: 'number', desc: 'Maximum number of simultaneous deployments (dependency permitting) to execute.', default: 1, requiresArg: true })
.option('asset-parallelism', { type: 'boolean', desc: 'Whether to build/publish assets in parallel' }),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason you did not add the default param for this option?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. p2 pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested pr-linter/exempt-integ-test The PR linter will not require integ test changes pr-linter/exempt-readme The PR linter will not require README changes pr-linter/exempt-test The PR linter will not require test changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants