Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions): add intrinsic functions #22431

Merged
merged 11 commits into from Oct 31, 2022
16 changes: 15 additions & 1 deletion packages/@aws-cdk/aws-stepfunctions/README.md
Expand Up @@ -192,9 +192,23 @@ You can also call [intrinsic functions](https://docs.aws.amazon.com/step-functio
| Method | Purpose |
|--------|---------|
| `JsonPath.array(JsonPath.stringAt('$.Field'), ...)` | make an array from other elements. |
| `JsonPath.format('The value is {}.', JsonPath.stringAt('$.Value'))` | insert elements into a format string. |
| `JsonPath.arrayPartition(JsonPath.listAt('$.inputArray'), 4)` | partition an array. |
| `JsonPath.arrayContains(JsonPath.listAt('$.inputArray'), 5)` | determine if a specific value is present in an array. |
| `JsonPath.arrayRange(1, 9, 2)` | create a new array containing a specific range of elements. |
| `JsonPath.arrayGetItem(JsonPath.listAt('$.inputArray'), 5)` | get a specified index's value in an array. |
| `JsonPath.arrayLength(JsonPath.listAt('$.inputArray'))` | get the length of an array. |
| `JsonPath.arrayUnique(JsonPath.listAt('$.inputArray'))` | remove duplicate values from an array. |
| `JsonPath.base64Encode(JsonPath.stringAt('$.input'))` | encode data based on MIME Base64 encoding scheme. |
| `JsonPath.base64Decode(JsonPath.stringAt('$.base64'))` | decode data based on MIME Base64 decoding scheme. |
| `JsonPath.hash(JsonPath.objectAt('$.Data'), JsonPath.stringAt('$.Algorithm'))` | calculate the hash value of a given input. |
| `JsonPath.jsonMerge(JsonPath.objectAt('$.Obj1'), JsonPath.objectAt('$.Obj2'))` | merge two JSON objects into a single object. |
| `JsonPath.stringToJson(JsonPath.stringAt('$.ObjStr'))` | parse a JSON string to an object |
| `JsonPath.jsonToString(JsonPath.objectAt('$.Obj'))` | stringify an object to a JSON string |
| `JsonPath.mathRandom(1, 999)` | return a random number. |
| `JsonPath.mathAdd(JsonPath.numberAt('$.value1'), JsonPath.numberAt('$.step'))` | return the sum of two numbers. |
| `JsonPath.stringSplit(JsonPath.stringAt('$.inputString'), JsonPath.stringAt('$.splitter'))` | split a string into an array of values. |
| `JsonPath.uuid()` | return a version 4 universally unique identifier (v4 UUID). |
| `JsonPath.format('The value is {}.', JsonPath.stringAt('$.Value'))` | insert elements into a format string. |

## Amazon States Language

Expand Down
190 changes: 188 additions & 2 deletions packages/@aws-cdk/aws-stepfunctions/lib/fields.ts
Expand Up @@ -103,6 +103,161 @@ export class JsonPath {
return new JsonPathToken(`States.Array(${values.map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayPartition expression
*
* Use this function to partition a large array. You can also use this intrinsic to slice the data and then send the payload in smaller chunks.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayPartition(array: any, chunkSize: number): string {
return new JsonPathToken(`States.ArrayPartition(${[array, chunkSize].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayContains expression
*
* Use this function to determine if a specific value is present in an array. For example, you can use this function to detect if there was an error in a Map state iteration.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayContains(array: any, value: any): string {
return new JsonPathToken(`States.ArrayContains(${[array, value].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayRange expression
*
* Use this function to create a new array containing a specific range of elements. The new array can contain up to 1000 elements.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayRange(start: number, end: number, step: number): string {
return new JsonPathToken(`States.ArrayRange(${[start, end, step].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayGetItem expression
*
* Use this function to get a specified index's value in an array.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayGetItem(array: any, index: number): string {
return new JsonPathToken(`States.ArrayGetItem(${[array, index].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.ArrayLength expression
*
* Use this function to get the length of an array.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayLength(array: any): string {
return new JsonPathToken(`States.ArrayLength(${renderInExpression(array)})`).toString();
}

/**
* Make an intrinsic States.ArrayUnique expression
*
* Use this function to get the length of an array.
* Use this function to remove duplicate values from an array and returns an array containing only unique elements. This function takes an array, which can be unsorted, as its sole argument.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static arrayUnique(array: any): string {
return new JsonPathToken(`States.ArrayUnique(${renderInExpression(array)})`).toString();
}

/**
* Make an intrinsic States.Base64Encode expression
*
* Use this function to encode data based on MIME Base64 encoding scheme. You can use this function to pass data to other AWS services without using an AWS Lambda function.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static base64Encode(input: string): string {
return new JsonPathToken(`States.Base64Encode(${renderInExpression(input)})`).toString();
}

/**
* Make an intrinsic States.Base64Decode expression
*
* Use this function to decode data based on MIME Base64 decoding scheme. You can use this function to pass data to other AWS services without using a Lambda function.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static base64Decode(base64: string): string {
return new JsonPathToken(`States.Base64Decode(${renderInExpression(base64)})`).toString();
}

/**
* Make an intrinsic States.Hash expression
*
* Use this function to calculate the hash value of a given input. You can use this function to pass data to other AWS services without using a Lambda function.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static hash(data: any, algorithm: string): string {
return new JsonPathToken(`States.Hash(${[data, algorithm].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.JsonMerge expression
*
* Use this function to merge two JSON objects into a single object.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static jsonMerge(value1: any, value2: any): string {
return new JsonPathToken(`States.JsonMerge(${[value1, value2].map(renderInExpression).join(', ')}, false)`).toString();
}

/**
* Make an intrinsic States.MathRandom expression
*
* Use this function to return a random number between the specified start and end number. For example, you can use this function to distribute a specific task between two or more resources.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static mathRandom(start: number, end: number): string {
return new JsonPathToken(`States.MathRandom(${[start, end].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.MathAdd expression
*
* Use this function to return the sum of two numbers. For example, you can use this function to increment values inside a loop without invoking a Lambda function.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static mathAdd(num1: number, num2: number): string {
return new JsonPathToken(`States.MathAdd(${[num1, num2].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.StringSplit expression
*
* Use this function to split a string into an array of values. This function takes two arguments.The first argument is a string and the second argument is the delimiting character that the function will use to divide the string.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static stringSplit(inputString: string, splitter: string): string {
return new JsonPathToken(`States.StringSplit(${[inputString, splitter].map(renderInExpression).join(', ')})`).toString();
}

/**
* Make an intrinsic States.UUID expression
*
* Use this function to return a version 4 universally unique identifier (v4 UUID) generated using random numbers. For example, you can use this function to call other AWS services or resources that need a UUID parameter or insert items in a DynamoDB table.
*
* @see https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html
*/
public static uuid(): string {
return new JsonPathToken('States.UUID()').toString();
}

/**
* Make an intrinsic States.Format expression
*
Expand Down Expand Up @@ -298,14 +453,45 @@ export class FieldUtils {
}

function validateJsonPath(path: string) {
const intrinsicFunctionNames = [
// Intrinsics for arrays
'Array',
'ArrayPartition',
'ArrayContains',
'ArrayRange',
'ArrayGetItem',
'ArrayLength',
'ArrayUnique',
// Intrinsics for data encoding and decoding
'Base64Encode',
'Base64Decode',
// Intrinsic for hash calculation
'Hash',
// Intrinsics for JSON data manipulation
'JsonMerge',
'StringToJson',
'JsonToString',
// Intrinsics for Math operations
'MathRandom',
'MathAdd',
// Intrinsic for String operation
'StringSplit',
// Intrinsic for unique identifier generation
'UUID',
// Intrinsic for generic operation
'Format',
];
const intrinsicFunctionFullNames = intrinsicFunctionNames.map((fn) => `States.${fn}`);
if (path !== '$'
&& !path.startsWith('$.')
&& path !== '$$'
&& !path.startsWith('$$.')
&& !path.startsWith('$[')
&& ['Format', 'StringToJson', 'JsonToString', 'Array'].every(fn => !path.startsWith(`States.${fn}`))
&& intrinsicFunctionFullNames.every(fn => !path.startsWith(fn))
) {
throw new Error(`JSON path values must be exactly '$', '$$', start with '$.', start with '$$.', start with '$[', or start with an intrinsic function: States.Format, States.StringToJson, States.JsonToString, or States.Array. Received: ${path}`);
const lastItem = intrinsicFunctionFullNames.pop();
const intrinsicFunctionsStr = intrinsicFunctionFullNames.join(', ') + ', or ' + lastItem;
throw new Error(`JSON path values must be exactly '$', '$$', start with '$.', start with '$$.', start with '$[', or start with an intrinsic function: ${intrinsicFunctionsStr}. Received: ${path}`);
}
}

Expand Down
19 changes: 12 additions & 7 deletions packages/@aws-cdk/aws-stepfunctions/lib/private/json-path.ts
Expand Up @@ -312,17 +312,22 @@ function pathFromToken(token: IResolvable | undefined) {
}

/**
* Render the string in a valid JSON Path expression.
* Render the string or number value in a valid JSON Path expression.
*
* If the string is a Tokenized JSON path reference -- return the JSON path reference inside it.
* Otherwise, single-quote it.
* If the value is a Tokenized JSON path reference -- return the JSON path reference inside it.
* If the value is a number -- convert it to string.
* If the value is a string -- single-quote it.
* Otherwise, throw errors.
*
* Call this function whenever you're building compound JSONPath expressions, in
* order to avoid having tokens-in-tokens-in-tokens which become very hard to parse.
*/
export function renderInExpression(x: string) {
const path = jsonPathString(x);
return path ?? singleQuotestring(x);
export function renderInExpression(x: any) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than throwing an error here for other types and accepting any, x should be typed to the specific valid types:

Suggested change
export function renderInExpression(x: any) {
export function renderInExpression(x: string | number) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renderInExpression can receive string[] here, so it should at least accept string[]. (Note that listAt returns string[]. )
We could restrict the input type by creating another function like renderListInExpression, but I think it is not necessary because the function is a "private" function.

https://github.com/gkkachi/aws-cdk/blob/ab29124d26cf8a21a1399b0193122c7b8419d00d/packages/%40aws-cdk/aws-stepfunctions/lib/fields.ts#L113-L115
https://github.com/gkkachi/aws-cdk/blob/ab29124d26cf8a21a1399b0193122c7b8419d00d/packages/%40aws-cdk/aws-stepfunctions/test/fields.test.ts#L241

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it can accept string[], the logic isn't quite right, then. You throw an error if it's not string or number.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, please write tests against the various input types you'd expect here and at least one failure case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote tests for renderInExpression . d32fbf3

const path = jsonPathFromAny(x);
if (path) return path;
if (typeof x === 'number') return x.toString(10);
if (typeof x === 'string') return singleQuotestring(x);
throw new Error('Unxexpected value.');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra x in unexpected

}

function singleQuotestring(x: string) {
Expand All @@ -341,4 +346,4 @@ function singleQuotestring(x: string) {
}
ret.push("'");
return ret.join('');
}
}