(aws-cdk/aws-s3-deployment): Directory deployment doesn't work

[pavelhlushchanka]

Hello,

It looks like that BucketDeployment doesn't work for directories. I have an non-empty directory with some files which i want to deploy. When I deploy, i receive an error Uploaded file must be a non-empty zip. However the documentation states that source of the BucketDeployment can be a directory.

Reproduction Steps

Create a resource similar to this one:

        s3_deployment.BucketDeployment(
            self,
            "MyResource",
            sources=[s3_deployment.Source.asset("./mydir")],
            destination_bucket=workspace_bucket,
            destination_key_prefix="/mydir",
            vpc=vpc,
            vpc_subnets=ec2.SubnetSelection(subnets=vpc.private_subnets)
        )

where mydir is a non empty directory.

What did you expect to happen?

mydir folder is synced with with destination folder.

What actually happened?

I receive an error Uploaded file must be a non-empty zip

Environment

• CDK CLI Version : 1.95.1
• Framework Version: 1.95.1
• Node.js Version: v15.12.0
• OS : Mac OS 11.1
• Language (Version): Python 3.9.2

---

This is 🐛 Bug Report

[pavelhlushchanka]

One more note. I tried to zip my folder and provide a path to the zip file. And i still get the same error Uploaded file must be a non-empty zip. So basically BucketDeployment doesn't work at all.

[iliapolo]

@codenamestif I wasn't able to reproduce this.

Can you double check you are using the CDK version you specified in the issue? We had this exact issue a few releases back and it was resolved: #12536

[pavelhlushchanka]

I confirm the versions. It's reproducible on my side. I even reproduced it with java lib. Since my cdk is installed with brew, i also tried to reinstall it with npm with no luck. As a next step i will try to deploy from a docker container.

[pavelhlushchanka]

@iliapolo i prepared a reproducible example https://github.com/codenamestif/s3-deployment-issue. You just need to provide credentials and destination bucket/key with env variables.

[iliapolo]

@codenamestif Thanks for creating this repo. I followed your instructions and still wasn't able to reproduce:

❯ docker run -e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} -e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} -e AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN} -e S3_BUCKET=${S3_BUCKET} -e S3_KEY=${S3_KEY} deployment-issue cdk deploy --require-approval 'never' --progress 'events'
deployment-issue: deploying...
[0%] start: Publishing e9882ab123687399f934da0d45effe675ecc8ce13b40cb946f3e1d6141fe8d68:current
[33%] success: Published e9882ab123687399f934da0d45effe675ecc8ce13b40cb946f3e1d6141fe8d68:current
[33%] start: Publishing c24b999656e4fe6c609c31bae56a1cf4717a405619c3aa6ba1bc686b8c2c86cf:current
[66%] success: Published c24b999656e4fe6c609c31bae56a1cf4717a405619c3aa6ba1bc686b8c2c86cf:current
[66%] start: Publishing 2fef5f29e3cf0166c4b6434cb3a021bbae413c135453a1b036a0d575b7a1f355:current
[100%] success: Published 2fef5f29e3cf0166c4b6434cb3a021bbae413c135453a1b036a0d575b7a1f355:current
deployment-issue: creating CloudFormation changeset...
 0/7 | 9:22:21 AM | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack  | deployment-issue User Initiated
 0/7 | 9:22:27 AM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack  | deployment-issue User Initiated
 1/7 | 9:22:54 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata          | Main/CDKMetadata/Default (CDKMetadata) 
 1/7 | 9:22:54 AM | CREATE_IN_PROGRESS   | AWS::Lambda::LayerVersion   | Main/Deployment/AwsCliLayer (DeploymentAwsCliLayerB82B26A3) 
 1/7 | 9:22:54 AM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) 
 1/7 | 9:22:55 AM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) Resource creation Initiated
 1/7 | 9:22:56 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata          | Main/CDKMetadata/Default (CDKMetadata) Resource creation Initiated
 1/7 | 9:22:56 AM | CREATE_COMPLETE      | AWS::CDK::Metadata          | Main/CDKMetadata/Default (CDKMetadata) 
 2/7 | 9:23:02 AM | CREATE_IN_PROGRESS   | AWS::Lambda::LayerVersion   | Main/Deployment/AwsCliLayer (DeploymentAwsCliLayerB82B26A3) Resource creation Initiated
 2/7 | 9:23:02 AM | CREATE_COMPLETE      | AWS::Lambda::LayerVersion   | Main/Deployment/AwsCliLayer (DeploymentAwsCliLayerB82B26A3) 
 3/7 | 9:23:08 AM | CREATE_COMPLETE      | AWS::IAM::Role              | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) 
 3/7 | 9:23:10 AM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) 
 3/7 | 9:23:11 AM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) Resource creation Initiated
 4/7 | 9:23:23 AM | CREATE_COMPLETE      | AWS::IAM::Policy            | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) 
 5/7 | 9:23:25 AM | CREATE_IN_PROGRESS   | AWS::Lambda::Function       | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) 
 5/7 | 9:23:27 AM | CREATE_IN_PROGRESS   | AWS::Lambda::Function       | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) Resource creation Initiated
 5/7 | 9:23:28 AM | CREATE_COMPLETE      | AWS::Lambda::Function       | Main/Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) 
 5/7 | 9:23:30 AM | CREATE_IN_PROGRESS   | Custom::CDKBucketDeployment | Main/Deployment/CustomResource/Default (DeploymentCustomResource47E8B2E6) 
 7/7 | 9:23:59 AM | CREATE_IN_PROGRESS   | Custom::CDKBucketDeployment | Main/Deployment/CustomResource/Default (DeploymentCustomResource47E8B2E6) Resource creation Initiated
 7/7 | 9:23:59 AM | CREATE_COMPLETE      | Custom::CDKBucketDeployment | Main/Deployment/CustomResource/Default (DeploymentCustomResource47E8B2E6) 
 7/7 | 9:24:00 AM | CREATE_COMPLETE      | AWS::CloudFormation::Stack  | deployment-issue 
arn:aws:cloudformation:us-east-1:<account-id-redacted>:stack/deployment-issue/6e189400-9139-11eb-a6f1-12be25e2835f

 ✅  deployment-issue

Can you confirm you are running this in a clean environment? my suspicion is that you might have a corrupted asset that was deployed with a version affected by #12536. Note that the corrupted asset is not the one you specify in the BucketDeployment construct. It is actually our internal asset that contains the lambda code for the handler.

Have you tried clearing up the bootstrap bucket as instructed here?: #12536 (comment)

Can you also share the full deployment log?

Thanks

[pavelhlushchanka]

@iliapolo Thanks for the tip. I have cleaned up the bootstrap bucket and it works now. I'm closing the issue.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.