(cli): allow asset publishing (or at least the asset generation) to run in sequence

[huyphan]

Describe the feature

Currently, the CDK CLI always publishes assets in parallel. I would like to be able to change this behaviour via a command line parameter so that they run in sequence. This behaviour is actually configurable via the publishInParallel prop of the AssetPublishing class

aws-cdk/packages/cdk-assets/lib/publishing.ts

Line 33 in 74318c7

readonly publishInParallel?: boolean;

but when it is used, the value is hard-coded to true

aws-cdk/packages/aws-cdk/lib/util/asset-publishing.ts

Line 47 in 5cc0d35

publishInParallel: true,

Use Case

In our CDK packages, assets are configured to be generated during publishing step using the source.executable property. Example of a manifest file:

{
  "version": "20.0.0",
  "files": {
    "<artifact-name>": {
      "source": {
        "executable": ['some', 'internal', 'command']
      },
      "destinations": {
        "<s3-bucket>": {
          "bucketName": "...",
          "objectKey": "...",
          "region": "us-west-2"
        }
      }
    }
  },
  "dockerImages": {}
}

If we have more than one assets, their publishing steps are run in parallel, the generation steps are also executed in the same fashion. Our internal tool has a unavoidable limitation where it can fail when running in parallel in some edge cases. We need the ability to tell CDK to run the asset publishing steps (or at least the asset generation steps) in sequence.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

N/A

Environment details (OS name and version, etc.)

N/A

[misterjoshua]

Agreed that publishInParallel could be easily configurable. The CLI arg sounds like a good idea, but I wonder whether we should consider a cdk.json option as well.

[huyphan]

cdk.json works too!

[eidanspiegel]

We are also affected by this

[rix0rrr]

Fixed in #22579

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.