How to Route JSON into Multiple Attributes in NiFi: A Step-by-Step Guide

Are you tired of struggling with routing JSON data into multiple attributes in Apache NiFi? Look no further! In this comprehensive guide, we’ll take you through the process of routing JSON data into multiple attributes with ease. By the end of this article, you’ll be a NiFi pro, effortlessly routing JSON data like a boss.

What You’ll Need

Before we dive in, make sure you have:

• A basic understanding of Apache NiFi and its components
• NiFi installed and running on your machine
• A JSON data source (e.g., a JSON file, API, or another NiFi processor)

Understanding JSON Data in NiFi

In NiFi, JSON data is treated as a string attribute. When you receive JSON data, NiFi stores it as a single attribute, making it challenging to extract specific values or route them to different attributes. But fear not! We’ll show you how to overcome this limitation using NiFi’s built-in processors.

The Problem: Routing JSON Data into Multiple Attributes

Let’s say you have a JSON data source that looks like this:

{
  "name": "John Doe",
  "age": 30,
  " occupation": "Software Engineer",
  "address": {
    "street": "123 Main St",
    "city": "Anytown",
    "state": "CA",
    "zip": "12345"
  }
}

You want to route this JSON data into multiple attributes, such as:

• name
• age
• occupation
• address_street
• address_city
• address_state
• address_zip

This is where the magic happens! We’ll use NiFi’s ExtractJsonPath processor to extract the desired values and route them to separate attributes.

Step 1: Add an ExtractJsonPath Processor

In your NiFi canvas, add an ExtractJsonPath processor by dragging and dropping it from the toolbar or by searching for it in the processor list.

Configure the processor as follows:

Property	Value
Destination	flowfile-attribute
JSON Path Expressions	$['name'], $['age'], $['occupation'], $['address']['street'], $['address']['city'], $['address']['state'], $['address']['zip']
Return Type	string

This configuration tells the processor to extract the specified JSON paths and store them as separate attributes in the flowfile.

Step 2: Route the Extracted Attributes

Now that we’ve extracted the desired attributes, we need to route them to their respective destinations. Add a RouteOnAttribute processor to your canvas and configure it as follows:

Property	Value
Routing Strategy	Route to separate relationships
Attribute Name	name, age, occupation, address_street, address_city, address_state, address_zip
Relationship	ROUTE_TO_NAME, ROUTE_TO_AGE, ROUTE_TO_OCCUPATION, ROUTE_TO_ADDRESS_STREET, ROUTE_TO_ADDRESS_CITY, ROUTE_TO_ADDRESS_STATE, ROUTE_TO_ADDRESS_ZIP

This configuration routes each extracted attribute to a separate relationship, allowing you to further process or store them as needed.

Step 3: Verify the Results

To verify that the attributes have been routed correctly, add a LogAttribute processor to each relationship and inspect the flowfile attributes.

2019-12-17 14:30:00,123 INFO  [processor.LogAttribute] name='John Doe'
2019-12-17 14:30:00,124 INFO  [processor.LogAttribute] age='30'
2019-12-17 14:30:00,125 INFO  [processor.LogAttribute] occupation='Software Engineer'
2019-12-17 14:30:00,126 INFO  [processor.LogAttribute] address_street='123 Main St'
2019-12-17 14:30:00,127 INFO  [processor.LogAttribute] address_city='Anytown'
2019-12-17 14:30:00,128 INFO  [processor.LogAttribute] address_state='CA'
2019-12-17 14:30:00,129 INFO  [processor.LogAttribute] address_zip='12345'

Voilà! You’ve successfully routed JSON data into multiple attributes in NiFi.

Conclusion

In this article, we’ve demonstrated how to route JSON data into multiple attributes in Apache NiFi using the ExtractJsonPath and RouteOnAttribute processors. By following these steps, you can easily extract and route JSON data to separate attributes, making it easier to process and store your data.

Tips and Variations

Here are some additional tips and variations to enhance your JSON routing experience:

• Use ExtractJsonPath with regex patterns to extract values from complex JSON structures.
• Combine ExtractJsonPath with other processors, such as SplitJson or JsonPathEvaluator, to handle more complex JSON scenarios.
• Use RouteOnAttribute with custom routing strategies to Route based on attribute values or other conditions.
• Explore NiFi’s other JSON-related processors, such as JsonReader and JsonWriter, to work with JSON data in different ways.

Keep exploring and experimenting with NiFi’s vast array of processors and capabilities. Happy data routing!

Frequently Asked Question

Getting stuck in routing JSON data into multiple attributes in Nifi? You’re not alone! Here are some frequently asked questions to help you navigate this challenge: