Data Mapping in Flows

Transform and Reshape Data for the Next Step.

You can use Data Mapper to reshape and transform data as it flows through your integration. Whether you're preparing inputs for an API call, structuring data for storage, or customizing outputs for downstream steps, the Data Mapper gives you full control. You can reference outputs from earlier steps, use flow variables, or inject secrets to build the exact data object you need, without writing custom code.

Getting Started

1. Log in to live.fastn.ai and navigate to "Flows." Select the flow type you need and name your new flow based on the task you want to perform.

2. Navigate to the side panel and from the list of available flow components, select "Data Mapper." This will allow you to reshape your data as needed for your specific flow requirements.

1. Inside the Data Mapper, define your data types such as int, string, or boolean. Once you've established a key for your value, you can choose to:

   • Hardcode a value directly.

   • Map it from a previous step, for example, {{input.text}}.

   • Concatenate values to create a new data format, like {{input.text}}_{{another.value}}.

Make sure your defined key type aligns with your headers or input in the terminal, otherwise it will throw an error.

1. Utilize AI-assisted suggestions in the Data Mapper for efficient schema mapping. Use the search function in the Flow data Map to quickly locate and map items:

   • Add headers to a new key you defined.

   • Incorporate an input for mapping.

   • Include a secret value that you have defined for flows in your project.

   • Assign steps to a key that navigate input from the previous steps.

   • An App Config for any environment where you wish to deploy your project.

   

2. You can run the Data Mapper in a debug mode. This allows you to verify the accuracy of mapping and ensure that the data transformations meet your expectations before proceeding to deployment.

Defining Field Data Types

Here's an example of defining different data types as shown in the image in JSON, including nested objects and arrays:

{
  "integerValue": 42,
  "floatValue": 3.14,
  "stringValue": "example",
  "booleanValue": true,
  "passwordValue": "s3cr3t",
  "decimalValue": 9.99,
  "objectValue": {
    "nestedInteger": 1,
    "nestedString": "nested"
  },
  "orderedObjectJSON": [
    {
      "order": 1,
      "description": "First item"
    },
    {
      "order": 2,
      "description": "Second item"
    }
  ],
  "Array": {
    "key": "value",
    "list": [1, 2, 3, 4]
  }
}

Value Configuration Options

1. Advanced Action

In the Advanced Action section, a variety of options are available to dynamically manipulate data:

• Filter: Extract specific elements from arrays or lists.

The filter function iterates through the elements and selects those matching specified criteria. The result is a new array containing only the filtered elements.

• UUID: Generate a unique identifier.

A UUID generates a unique identifier that is essential for ensuring data items are distinct within a dataset. This function helps in tracking and referencing specific data entries in complex data mapping scenarios.

• Array Size: Obtain the size of an array.

• Escape String: Escape special characters in a string.

Use this advanced action to automatically escape characters like quotation marks ("), backslashes (\), and others that may interfere with parsing or formatting in downstream systems. Useful when working with JSON, code snippets, or other structured data formats that require properly escaped strings.

• Loop: Iterate over elements to apply transformations.

• Substring: Extract a portion of a string.

• Trim White Spaces: Remove leading and trailing spaces.

• Math Expression: Apply mathematical calculations.

• Conditions: Implement logic branches based on conditions.

• Converter: Change data types or formats.

• Transcode: Transform data encoding.

• Hashing: Generate a hash value.

• Append To List: Add an element to a list.

• Remove From List: Delete elements from a list.

• Insert Item: Place an item at a specific position.

• Remove Item: Delete an item from a list.

• Date Source: Handle and manipulate date/time values.

• Find index: Determine the index of an element in a list.

2. Add Parent Key

Use: Encapsulate the field under a higher-level object key to create structured output.

Example:

• Original:

  jsonCopyEdit{ "id": "input" }

• With parent key "user" added:

  jsonCopyEdit{ 
    "user": { "id": "input" } 
  }

What Happens: The field becomes part of a nested JSON structure. This is useful when APIs or schemas expect data to be wrapped inside parent keys

3. Set Default Value

Use: Define what value should be used when the input is missing or not explicitly set.

Options:

• Null

  • Sets the field to null.

  • Example output:

    jsonCopyEdit{ "id": null }

  On every run, null is returned unless overridden. Useful when you want to mark a value as explicitly absent.

• Empty

  • Sets the field to an empty string or empty object (depending on context).

  • Example:

    jsonCopyEdit{ "id": "" }

  Ensures the field exists with an empty value. Used to clear or reset the field.

• Fallback Value

  • Provides a static value if nothing else is passed.

  • Example:

    jsonCopyEdit{ "id": "default-id-001" }

  If id is not set, this default will be used during execution.

• Required (fail if missing)

  • Enforces that a value must be present.

  If id is not provided at runtime, the system throws an error or blocks the workflow. Ideal for mandatory identifiers.

4. Import JSON

Use: Load and assign structured JSON data directly to the field.

Example:

• Imported JSON file:

  jsonCopyEdit{
    "id": "123",
    "type": "admin",
    "status": "active"
  }

• Result:

  jsonCopyEdit{
    "id": {
      "id": "123",
      "type": "admin",
      "status": "active"
    }
  }

The field is replaced by the full JSON structure from the import. This is useful for setting default templates or pre-populated data.

Code View & Expression Modes

View or author logic in different programming languages or expression styles.

Fastn data mapper offers a feature that allows you to switch between visual and code-based views for defining field values within a flow. It supports various programming languages like JavaScript, C#, and Python Lambda.

This flexibility enables users to manage their data structure according to their preferred coding or visual style, offering a more customizable approach to editing and mapping data. Additionally, the section mentions the functionality of adding states to manage dynamic data changes, like updating counters in response to API calls, which aids in maintaining accurate and updated data flow within the application.

• Form Visual editor for defining field values through UI components.

Switch between visual and code-based views of field logic:

• JavaScript

• C#

• Python Lambda (if applicable)

States & Mapping

You can add states in your flow before your data mapper to fetch values from previously defined states. Every time you call an API, you may want to increment a counter or update its count.

This can be achieved by storing the last state information and using it next time you run the flow.

For example:

• To increment a sum:

  state.inc.sum += 1

• To check if a state exists and perform an operation:

  if (isExist(state.inc.sum)) {
      state.inc.sum++;
  } else {
      state.inc.sum = 1; // Initialize sum if it does not exist
  }

Standard Error Patterns

DATA_VALIDATION_ERROR

This error often arises due to issues with data mapping and type conversion. Here are some corrective actions to resolve it:

1. Incorrect Type Mapping: Ensure you are not mapping a string to other data types such as boolean, number, decimal, or list. Strings must be mapped to string fields only.

2. Invalid Field Naming: Do not start mapping field names with numbers, for example, {{4444sss.tet}} is not valid.

3. Default Value Check: If a field's default value is empty, consider setting it to null.

4. Value Escaping: Properly escape values that contain special characters like " or / using the advanced action feature.

5. Advance Action Inputs: Review any advanced actions in the step to ensure that input values are not passing incorrect data.