Step Functions Made Easy in AWS and LocalStack

AWS Step Functions are incredibly powerful and useful for building workflows that can do things like automate multi-step processes, orchestrate microservices or create data processing pipelines. However, building these workflows (aka state machines) can become quite complex, in part because of the manner in which Step Functions passed and accessed state across various steps. In some cases, it might involve adding extraneous steps or writing complex ASL (Amazon States Language) to handle simple tasks like date/time formatting or basic mathematical calculations.

Well, I’ve got good news! AWS Step Functions just announced new support for variables and JSONata. In this post we’ll explore how this simplifies the overall developer experience of building Step Functions by making things like managing state and data transformations easier.

I’ve got even more good news! You can already use these new features in your local testing and automations that depend on LocalStack. With the support of the AWS Step Functions team, we are able to provide you with day zero access to these awesome new capabilities.

JSONata

The biggest change to Step Functions is the new support for JSONata in ASL. JSONata offers a bunch more capabilities than JSONPath and can help eliminate the need for things like Pass states that make simple modifications to the data passed between states. This can make your workflows simpler while also potentially reducing costs on AWS.

JSONPath is a simple query language for retrieving data within JSON. Based on XPath for XML, it allows you to write expressions to more easily reference data deep within a JSON response. For instance, you could reference data in a response from one state and assign it to the input of another.

However, JSONPath does not allow any kind of data manipulation. This meant that often simple tasks like formatting or mathematical calculations required intermediary states to transform the data into the format needed by the subsequent state or use intrinsic functions.

For example, we can get firstname and lastname easily…

$.orders.order[0].firstname
$.orders.order[0].lastname

…but if we need to combine them into a single string called name, it would require an additional intermediate step.

JSONata, on the other hand, supports an XPath-like querying alongside a wide range of functions and transformations. You can use JSONata’s built in operators and functions to manipulate data passing between states in your state machine without requiring any intermediary steps.

In this example, we’re concatenating the first and last name while also trimming any white space from the names.

$trim($orders.order[0].firstname) & ' ' & $trim($orders.order[0].lastname)

This is a very simple example but there’s so much more that you can do with JSONata. Check out the JSONata documentation for a full language guide. To enable JSONata, just set the new QueryLanguage key in your ASL at the state machine level.

"QueryLanguage": "JSONata",

Variables

Traditionally, state in step functions had to be passed in and out of step functions using input and output processing fields like InputPath, ResultPath, or OutputPath that allowed you to filter and assign values. This could sometimes be difficult to write and debug.

The new variables support in Step Functions eliminates the need for these processing fields and makes passing data into and out of a state much simpler. Variables work with both the new JSONata support and the traditional JSONPath syntax.

Start by using the new assign field in your ASL to assign values to one or more variables. Note that, if you’re using ASL with JSONata (as in the example below), you do not even need to append the trailing .$, further simplifying the syntax. JSONata expressions are identified with {%...%}.

"QueryLanguage": "JSONata",
...
"Assign": {
  "info": {
      "status": "{% $order.status% }",
      "completedItems": "{% $order.items[type='done'] %}"
  },
  "total": "{% $sum($charges) %}",
  "success": true,
  "message": "{% 'The total was ' & $states.result.total %}"
}

The variables that have been assigned can be referenced and evaluated throughout your ASL. You can see this in the assignment block above (variable references are preceded by the $). This makes it much easier to design the input (arguments) and output (output) structure of your state in a declarative manner without the need for complex filtering and assignment blocks.

Conclusion

If you’d like to learn more about the new features in AWS Step Functions, check out their documentation. LocalStack’s Step Functions support is available in our free and open-source Community Edition and offers a perfect opportunity to learn and experiment with these new features locally before deploying them to AWS . To try them out, update to LocalStack 4.0.1 by using localstack update all from the command line or pulling the latest image from DockerHub.