Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): add volumes-from option to docker run command for bundling #22829

Merged
merged 30 commits into from Dec 12, 2022
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
9a18004
WIP add volumesFrom
webratz Aug 9, 2022
020b825
WIP add volumesFrom
webratz Aug 9, 2022
6bcb4ee
adding tests and usage instruction
webratz Aug 18, 2022
8473d45
disable failing test for now
webratz Sep 13, 2022
abdb9b5
add missing type
webratz Sep 13, 2022
658e1dd
Merge branch 'main' into master
mergify[bot] Oct 2, 2022
a7fc600
Merge branch 'aws:main' into master
webratz Oct 4, 2022
74aae68
cleanup
webratz Oct 4, 2022
2ee951b
re-add adjusted test
webratz Oct 5, 2022
9dbbd18
test variant to allow the call to fail
webratz Oct 5, 2022
1fb4ada
remove failing test
webratz Oct 5, 2022
96c5614
whitespace cleanup
webratz Oct 5, 2022
d6b0bd4
Merge branch 'main' into master
webratz Nov 2, 2022
18073db
adding fixed test
webratz Nov 8, 2022
c287929
Merge branch 'aws:main' into master
webratz Nov 8, 2022
1f5c667
add integration test
webratz Nov 8, 2022
f5e5ea0
Merge branch 'master' of https://github.com/webratz/aws-cdk
webratz Nov 8, 2022
08fd008
check if adding helper container works
webratz Nov 8, 2022
aa528ca
update integration tests
webratz Nov 9, 2022
13c7e8b
test sync spawn of docker container
webratz Nov 9, 2022
bce18f2
Merge branch 'main' into master
webratz Nov 9, 2022
e032d03
Merge branch 'main' into master
webratz Nov 11, 2022
f1d9a16
Merge branch 'main' into master
webratz Nov 11, 2022
5a5d3f7
Apply suggestions from code review
webratz Dec 9, 2022
89bb614
WIP allow multiple volumes to be defined
webratz Dec 9, 2022
2ed997a
add full integration test for all steps
webratz Dec 9, 2022
9754f03
remove of integration tests and additional options
webratz Dec 12, 2022
af81272
remove integration for python
webratz Dec 12, 2022
c637016
remove empty line
webratz Dec 12, 2022
6c21217
Merge branch 'main' into master
webratz Dec 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 12 additions & 0 deletions packages/@aws-cdk/aws-lambda-python/README.md
Expand Up @@ -145,6 +145,18 @@ new python.PythonFunction(this, 'function', {
});
```

In situations where files are needed for bundling that are from a different container, you can specify the container ID from which additional volumes should be mounted.

```ts
const entry = '/path/to/function';

new python.PythonFunction(this, 'function', {
entry,
runtime: Runtime.PYTHON_3_8,
bundling: { volumesFrom: ['777f7dc92da7'] },
});
```

webratz marked this conversation as resolved.
Show resolved Hide resolved
## Custom Bundling with Code Artifact

To use a Code Artifact PyPI repo, the `PIP_INDEX_URL` for bundling the function can be customized (requires AWS CLI in the build environment):
Expand Down
8 changes: 8 additions & 0 deletions packages/@aws-cdk/aws-lambda-python/lib/bundling.ts
Expand Up @@ -59,6 +59,7 @@ export class Bundling implements CdkBundlingOptions {
public readonly image: DockerImage;
public readonly command: string[];
public readonly environment?: { [key: string]: string };
public readonly volumesFrom?: string[];

constructor(props: BundlingProps) {
const {
Expand All @@ -68,6 +69,7 @@ export class Bundling implements CdkBundlingOptions {
outputPathSuffix = '',
image,
poetryIncludeHashes,
customCommands,
} = props;

const outputPath = path.posix.join(AssetStaging.BUNDLING_OUTPUT_DIR, outputPathSuffix);
Expand All @@ -77,6 +79,7 @@ export class Bundling implements CdkBundlingOptions {
inputDir: AssetStaging.BUNDLING_INPUT_DIR,
outputDir: outputPath,
poetryIncludeHashes,
customCommands: customCommands,
});

this.image = image ?? DockerImage.fromBuild(path.join(__dirname, '../lib'), {
Expand All @@ -88,6 +91,7 @@ export class Bundling implements CdkBundlingOptions {
});
this.command = ['bash', '-c', chain(bundlingCommands)];
this.environment = props.environment;
this.volumesFrom = props.volumesFrom;
}

private createBundlingCommand(options: BundlingCommandOptions): string[] {
Expand All @@ -99,6 +103,9 @@ export class Bundling implements CdkBundlingOptions {
if (packaging.dependenciesFile) {
bundlingCommands.push(`python -m pip install -r ${DependenciesFile.PIP} -t ${options.outputDir}`);
}
if (options.customCommands) {
bundlingCommands.push(...options.customCommands);
}
return bundlingCommands;
}
}
Expand All @@ -108,6 +115,7 @@ interface BundlingCommandOptions {
readonly inputDir: string;
readonly outputDir: string;
readonly poetryIncludeHashes?: boolean;
readonly customCommands?: string[]
}

/**
Expand Down
13 changes: 13 additions & 0 deletions packages/@aws-cdk/aws-lambda-python/lib/types.ts
Expand Up @@ -86,4 +86,17 @@ export interface BundlingOptions {
* @default - Based on `assetHashType`
*/
readonly assetHash?: string;

/**
* Where to mount the specified volumes from
* @see https://docs.docker.com/engine/reference/commandline/run/#mount-volumes-from-container---volumes-from
* @default - no containers are specified to mount volumes from
*/
readonly volumesFrom?: string[];

/**
* Additional commands executed in the bundling container after running dependency installation
* @default - no additional custom commands are executed
*/
readonly customCommands?: string[]
webratz marked this conversation as resolved.
Show resolved Hide resolved
}
15 changes: 15 additions & 0 deletions packages/@aws-cdk/aws-lambda-python/test/bundling.test.ts
Expand Up @@ -278,6 +278,21 @@ test('Bundling with custom environment vars`', () => {
}));
});

test('Bundling with custom volumes', () => {
const entry = path.join(__dirname, 'lambda-handler');
Bundling.bundle({
entry: entry,
runtime: Runtime.PYTHON_3_7,
volumesFrom: ['777f7dc92da7'],
});

expect(Code.fromAsset).toHaveBeenCalledWith(entry, expect.objectContaining({
bundling: expect.objectContaining({
volumesFrom: ['777f7dc92da7'],
}),
}));
});

test('Do not build docker image when skipping bundling', () => {
const entry = path.join(__dirname, 'lambda-handler');
Bundling.bundle({
Expand Down