feat(s3): add EventBridge bucket notifications #18150
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
|
Title does not follow the guidelines of Conventional Commits. Please adjust title before merge. |
yerzhan7
changed the title
otaviomacedo
approved these changes
Jan 10, 2022
|
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
|
@otaviomacedo , thanks for reviewing this PR. However, as I mentioned in the description above, this feature is not working yet due to old AWS SDK version in Lambda. Let's revert and wait for SDK update? |
|
Is there an expected release date for the SDK update? I want to use this for StepFunctions but CloudFormation does not like it |
|
Hi team, I have the same problem as orekav: |
rix0rrr
added a commit
that referenced
this pull request
Jan 26, 2022
In #18150, a change was merged that blew up the size of the inline Lambda beyond its limit of 4096 characters. This change was not detected because the Lambda constructs being used there didn't use the regular `aws-lambda` module, but escape hatches that bypass the regular validation (released in 1.139.0, 2.8.0). Because this effectively broke S3 notifications, it was rolled back in #18507 (released in 1.140.0, not yet released in 2.x line). In this PR, add validation to make sure an event like this doesn't happen again. This will be relevant for #18614.
mergify bot
pushed a commit
that referenced
this pull request
Jan 26, 2022
In #18150, a change was merged that blew up the size of the inline Lambda beyond its limit of 4096 characters. This change was not detected because the Lambda constructs being used there didn't use the regular `aws-lambda` module, but escape hatches that bypass the regular validation (released in 1.139.0, 2.8.0). Because this effectively broke S3 notifications, it was rolled back in #18507 (released in 1.140.0, not yet released in 2.x line). In this PR, add validation to make sure an event like this doesn't happen again. This will be relevant for #18614. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
TikiTDO
pushed a commit
to TikiTDO/aws-cdk
that referenced
this pull request
Feb 21, 2022
### **Description** Adds EventBridge bucket notification configuration. See https://aws.amazon.com/blogs/aws/new-use-amazon-s3-event-notifications-with-amazon-eventbridge/ ### **Implementation** - Added new Bucket property to enable this feature (`eventBridgeEnabled: true`) - Added EventBridge config to `S3BucketNotifications` Custom Resource - Added unit tests - Added integration test (currently fails, see below for more info) - Fixed dependent integration tests Closes aws#18076 ### **FAQ** 1. **Why not simply expose EventBridge Cfn property via S3 BucketProps?** Currently CDK manages `NotificationConfigurations `via CustomResource. If we were to expose that way, then e.g. SNS configuration would override EventBridge config. 2. **Why not create new `IBucketNotificationDestination` class for EventBridge?** We can, but there is no need. Usually we create a subclass to `IBucketNotificationDestination` in order to adjust resource permissions, however in this case there is no need to adjust permissions: [default EventBridge does not require any additional permissions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ev-permissions.html) unlike SQS/SNS/Lambda destinations. Additionally, enabling this feature via bucket props is much cleaner/simpler API than creating new dummy object of type `IBucketNotificationDestination` for customers. However, if you still think that we need to create new `IBucketNotificationDestination` subclass for EventBridge for consistency, let me know and I will refactor. ---- **BLOCKED ON LAMBDA RUNTIME SDK UPDATE TO BOTOCORE >= v1.23.16 (Integration test currently fails as current version (v1.21.55) does not contain EventBridge configuration)** Check latest version here: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
TikiTDO
pushed a commit
to TikiTDO/aws-cdk
that referenced
this pull request
Feb 21, 2022
In aws#18150, a change was merged that blew up the size of the inline Lambda beyond its limit of 4096 characters. This change was not detected because the Lambda constructs being used there didn't use the regular `aws-lambda` module, but escape hatches that bypass the regular validation (released in 1.139.0, 2.8.0). Because this effectively broke S3 notifications, it was rolled back in aws#18507 (released in 1.140.0, not yet released in 2.x line). In this PR, add validation to make sure an event like this doesn't happen again. This will be relevant for aws#18614. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
2 tasks
mergify bot
pushed a commit
that referenced
this pull request
Apr 1, 2022
## Duplicate of #18150 ## ~~Blocked on Lambda runtime SDK update to Botocore >= v1.23.16 (Integration test currently fails as current version (v1.21.55) does not contain EventBridge configuration)~~ ## ~~Check latest version here: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html~~ ### **Description** Adds EventBridge bucket notification configuration. See https://aws.amazon.com/blogs/aws/new-use-amazon-s3-event-notifications-with-amazon-eventbridge/ ### **Implementation** - Added new Bucket property to enable this feature (`eventBridgeEnabled: true`) - Added EventBridge config to `S3BucketNotifications` Custom Resource - Added unit tests - Added integration test (currently fails, see below for more info) - Fixed dependent integration tests Closes #18076 ### **FAQ** 1. **Why not simply expose EventBridge Cfn property via S3 BucketProps?** Currently CDK manages `NotificationConfigurations `via CustomResource. If we were to expose that way, then e.g. SNS configuration would override EventBridge config. 2. **Why not create new `IBucketNotificationDestination` class for EventBridge?** We can, but there is no need. Usually we create a subclass to `IBucketNotificationDestination` in order to adjust resource permissions, however in this case there is no need to adjust permissions: [default EventBridge does not require any additional permissions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ev-permissions.html) unlike SQS/SNS/Lambda destinations. Additionally, enabling this feature via bucket props is much cleaner/simpler API than creating new dummy object of type `IBucketNotificationDestination` for customers. However, if you still think that we need to create new `IBucketNotificationDestination` subclass for EventBridge for consistency, let me know and I will refactor. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
StevePotter
pushed a commit
to StevePotter/aws-cdk
that referenced
this pull request
Apr 27, 2022
## Duplicate of aws#18150 ## ~~Blocked on Lambda runtime SDK update to Botocore >= v1.23.16 (Integration test currently fails as current version (v1.21.55) does not contain EventBridge configuration)~~ ## ~~Check latest version here: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html~~ ### **Description** Adds EventBridge bucket notification configuration. See https://aws.amazon.com/blogs/aws/new-use-amazon-s3-event-notifications-with-amazon-eventbridge/ ### **Implementation** - Added new Bucket property to enable this feature (`eventBridgeEnabled: true`) - Added EventBridge config to `S3BucketNotifications` Custom Resource - Added unit tests - Added integration test (currently fails, see below for more info) - Fixed dependent integration tests Closes aws#18076 ### **FAQ** 1. **Why not simply expose EventBridge Cfn property via S3 BucketProps?** Currently CDK manages `NotificationConfigurations `via CustomResource. If we were to expose that way, then e.g. SNS configuration would override EventBridge config. 2. **Why not create new `IBucketNotificationDestination` class for EventBridge?** We can, but there is no need. Usually we create a subclass to `IBucketNotificationDestination` in order to adjust resource permissions, however in this case there is no need to adjust permissions: [default EventBridge does not require any additional permissions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ev-permissions.html) unlike SQS/SNS/Lambda destinations. Additionally, enabling this feature via bucket props is much cleaner/simpler API than creating new dummy object of type `IBucketNotificationDestination` for customers. However, if you still think that we need to create new `IBucketNotificationDestination` subclass for EventBridge for consistency, let me know and I will refactor. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adds EventBridge bucket notification configuration.
See https://aws.amazon.com/blogs/aws/new-use-amazon-s3-event-notifications-with-amazon-eventbridge/
Implementation
eventBridgeEnabled: true)S3BucketNotificationsCustom ResourceCloses #18076
FAQ
Currently CDK manages
NotificationConfigurationsvia CustomResource. If we were to expose that way, then e.g. SNS configuration would override EventBridge config.IBucketNotificationDestinationclass for EventBridge?We can, but there is no need. Usually we create a subclass to
IBucketNotificationDestinationin order to adjust resource permissions, however in this case there is no need to adjust permissions: default EventBridge does not require any additional permissions unlike SQS/SNS/Lambda destinations. Additionally, enabling this feature via bucket props is much cleaner/simpler API than creating new dummy object of typeIBucketNotificationDestinationfor customers.However, if you still think that we need to create new
IBucketNotificationDestinationsubclass for EventBridge for consistency, let me know and I will refactor.BLOCKED ON LAMBDA RUNTIME SDK UPDATE TO BOTOCORE >= v1.23.16 (Integration test currently fails as current version (v1.21.55) does not contain EventBridge configuration)
Check latest version here: https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license