Skip to content

Commit

Permalink
feat(stepfunctions): add intrinsic functions (#22431)
Browse files Browse the repository at this point in the history 
Resolves #22068 and resolves #22629

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
@gkkachi
gkkachi committed Oct 31, 2022
1 parent ebba9e3 commit 8f85b08
Show file tree
Hide file tree
Showing 15 changed files with 2,054 additions and 12 deletions.
16 changes: 15 additions & 1 deletion packages/@aws-cdk/aws-stepfunctions/README.md
View file
Expand Up @@ -192,9 +192,23 @@ You can also call [intrinsic functions](https://docs.aws.amazon.com/step-functio
| Method | Purpose |
|--------|---------|
| `JsonPath.array(JsonPath.stringAt('$.Field'), ...)` | make an array from other elements. |
| `JsonPath.format('The value is {}.', JsonPath.stringAt('$.Value'))` | insert elements into a format string. |
| `JsonPath.arrayPartition(JsonPath.listAt('$.inputArray'), 4)` | partition an array. |
| `JsonPath.arrayContains(JsonPath.listAt('$.inputArray'), 5)` | determine if a specific value is present in an array. |
| `JsonPath.arrayRange(1, 9, 2)` | create a new array containing a specific range of elements. |
| `JsonPath.arrayGetItem(JsonPath.listAt('$.inputArray'), 5)` | get a specified index's value in an array. |
| `JsonPath.arrayLength(JsonPath.listAt('$.inputArray'))` | get the length of an array. |
| `JsonPath.arrayUnique(JsonPath.listAt('$.inputArray'))` | remove duplicate values from an array. |
| `JsonPath.base64Encode(JsonPath.stringAt('$.input'))` | encode data based on MIME Base64 encoding scheme. |
| `JsonPath.base64Decode(JsonPath.stringAt('$.base64'))` | decode data based on MIME Base64 decoding scheme. |
| `JsonPath.hash(JsonPath.objectAt('$.Data'), JsonPath.stringAt('$.Algorithm'))` | calculate the hash value of a given input. |
| `JsonPath.jsonMerge(JsonPath.objectAt('$.Obj1'), JsonPath.objectAt('$.Obj2'))` | merge two JSON objects into a single object. |
| `JsonPath.stringToJson(JsonPath.stringAt('$.ObjStr'))` | parse a JSON string to an object |
| `JsonPath.jsonToString(JsonPath.objectAt('$.Obj'))` | stringify an object to a JSON string |
| `JsonPath.mathRandom(1, 999)` | return a random number. |
| `JsonPath.mathAdd(JsonPath.numberAt('$.value1'), JsonPath.numberAt('$.step'))` | return the sum of two numbers. |
| `JsonPath.stringSplit(JsonPath.stringAt('$.inputString'), JsonPath.stringAt('$.splitter'))` | split a string into an array of values. |
| `JsonPath.uuid()` | return a version 4 universally unique identifier (v4 UUID). |
| `JsonPath.format('The value is {}.', JsonPath.stringAt('$.Value'))` | insert elements into a format string. |

## Amazon States Language

Expand Down
190 changes: 188 additions & 2 deletions packages/@aws-cdk/aws-stepfunctions/lib/fields.ts
View file
Expand Up @@ -103,6 +103,161 @@ export class JsonPath {
return new JsonPathToken(`States.Array(${values.map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayPartition expression
*
* Use this function to partition a large array. You can also use this intrinsic to slice the data and then send the payload in smaller chunks.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayPartition(array: any, chunkSize: number): string {
return new JsonPathToken(`States.ArrayPartition(${[array, chunkSize].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayContains expression
*
* Use this function to determine if a specific value is present in an array. For example, you can use this function to detect if there was an error in a Map state iteration.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayContains(array: any, value: any): string {
return new JsonPathToken(`States.ArrayContains(${[array, value].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayRange expression
*
* Use this function to create a new array containing a specific range of elements. The new array can contain up to 1000 elements.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayRange(start: number, end: number, step: number): string {
return new JsonPathToken(`States.ArrayRange(${[start, end, step].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayGetItem expression
*
* Use this function to get a specified index's value in an array.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayGetItem(array: any, index: number): string {
return new JsonPathToken(`States.ArrayGetItem(${[array, index].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayLength expression
*
* Use this function to get the length of an array.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayLength(array: any): string {
return new JsonPathToken(`States.ArrayLength(${renderInExpression(array)})`).toString();
}

/**
* Make an intrinsic States.ArrayUnique expression
*
* Use this function to get the length of an array.
* Use this function to remove duplicate values from an array and returns an array containing only unique elements. This function takes an array, which can be unsorted, as its sole argument.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayUnique(array: any): string {
return new JsonPathToken(`States.ArrayUnique(${renderInExpression(array)})`).toString();
}

/**
* Make an intrinsic States.Base64Encode expression
*
* Use this function to encode data based on MIME Base64 encoding scheme. You can use this function to pass data to other AWS services without using an AWS Lambda function.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static base64Encode(input: string): string {
return new JsonPathToken(`States.Base64Encode(${renderInExpression(input)})`).toString();
}

/**
* Make an intrinsic States.Base64Decode expression
*
* Use this function to decode data based on MIME Base64 decoding scheme. You can use this function to pass data to other AWS services without using a Lambda function.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static base64Decode(base64: string): string {
return new JsonPathToken(`States.Base64Decode(${renderInExpression(base64)})`).toString();
}

/**
* Make an intrinsic States.Hash expression
*
* Use this function to calculate the hash value of a given input. You can use this function to pass data to other AWS services without using a Lambda function.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static hash(data: any, algorithm: string): string {
return new JsonPathToken(`States.Hash(${[data, algorithm].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.JsonMerge expression
*
* Use this function to merge two JSON objects into a single object.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static jsonMerge(value1: any, value2: any): string {
return new JsonPathToken(`States.JsonMerge(${[value1, value2].map(renderInExpression).join(', ')}, false)`).toString();
}

/**
* Make an intrinsic States.MathRandom expression
*
* Use this function to return a random number between the specified start and end number. For example, you can use this function to distribute a specific task between two or more resources.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static mathRandom(start: number, end: number): string {
return new JsonPathToken(`States.MathRandom(${[start, end].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.MathAdd expression
*
* Use this function to return the sum of two numbers. For example, you can use this function to increment values inside a loop without invoking a Lambda function.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static mathAdd(num1: number, num2: number): string {
return new JsonPathToken(`States.MathAdd(${[num1, num2].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.StringSplit expression
*
* Use this function to split a string into an array of values. This function takes two arguments.The first argument is a string and the second argument is the delimiting character that the function will use to divide the string.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static stringSplit(inputString: string, splitter: string): string {
return new JsonPathToken(`States.StringSplit(${[inputString, splitter].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.UUID expression
*
* Use this function to return a version 4 universally unique identifier (v4 UUID) generated using random numbers. For example, you can use this function to call other AWS services or resources that need a UUID parameter or insert items in a DynamoDB table.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static uuid(): string {
return new JsonPathToken('States.UUID()').toString();
}

/**
* Make an intrinsic States.Format expression
*
Expand Down Expand Up @@ -298,14 +453,45 @@ export class FieldUtils {
}

function validateJsonPath(path: string) {
const intrinsicFunctionNames = [
// Intrinsics for arrays
'Array',
'ArrayPartition',
'ArrayContains',
'ArrayRange',
'ArrayGetItem',
'ArrayLength',
'ArrayUnique',
// Intrinsics for data encoding and decoding
'Base64Encode',
'Base64Decode',
// Intrinsic for hash calculation
'Hash',
// Intrinsics for JSON data manipulation
'JsonMerge',
'StringToJson',
'JsonToString',
// Intrinsics for Math operations
'MathRandom',
'MathAdd',
// Intrinsic for String operation
'StringSplit',
// Intrinsic for unique identifier generation
'UUID',
// Intrinsic for generic operation
'Format',
];
const intrinsicFunctionFullNames = intrinsicFunctionNames.map((fn) => `States.${fn}`);
if (path !== '$'
&& !path.startsWith('$.')
&& path !== '$$'
&& !path.startsWith('$$.')
&& !path.startsWith('$[')
&& ['Format', 'StringToJson', 'JsonToString', 'Array'].every(fn => !path.startsWith(`States.${fn}`))
&& intrinsicFunctionFullNames.every(fn => !path.startsWith(fn))
) {
throw new Error(`JSON path values must be exactly '$', '$$', start with '$.', start with '$$.', start with '$[', or start with an intrinsic function: States.Format, States.StringToJson, States.JsonToString, or States.Array. Received: ${path}`);
const lastItem = intrinsicFunctionFullNames.pop();
const intrinsicFunctionsStr = intrinsicFunctionFullNames.join(', ') + ', or ' + lastItem;
throw new Error(`JSON path values must be exactly '$', '$$', start with '$.', start with '$$.', start with '$[', or start with an intrinsic function: ${intrinsicFunctionsStr}. Received: ${path}`);
}
}

Expand Down
19 changes: 12 additions & 7 deletions packages/@aws-cdk/aws-stepfunctions/lib/private/json-path.ts
View file
Expand Up @@ -312,17 +312,22 @@ function pathFromToken(token: IResolvable | undefined) {
}

/**
* Render the string in a valid JSON Path expression.
* Render the string or number value in a valid JSON Path expression.
*
* If the string is a Tokenized JSON path reference -- return the JSON path reference inside it.
* Otherwise, single-quote it.
* If the value is a Tokenized JSON path reference -- return the JSON path reference inside it.
* If the value is a number -- convert it to string.
* If the value is a string -- single-quote it.
* Otherwise, throw errors.
*
* Call this function whenever you're building compound JSONPath expressions, in
* order to avoid having tokens-in-tokens-in-tokens which become very hard to parse.
*/
export function renderInExpression(x: string) {
const path = jsonPathString(x);
return path ?? singleQuotestring(x);
export function renderInExpression(x: any) {
const path = jsonPathFromAny(x);
if (path) return path;
if (typeof x === 'number') return x.toString(10);
if (typeof x === 'string') return singleQuotestring(x);
throw new Error('Unxexpected value.');
}

function singleQuotestring(x: string) {
Expand All @@ -341,4 +346,4 @@ function singleQuotestring(x: string) {
}
ret.push("'");
return ret.join('');
}
}

0 comments on commit 8f85b08

Please sign in to comment.