Enterprise Integration with SAP CPI (Part 3 of 4)

In this part, we learn about Routers, Content Modifiers, Content Enricher and a little more of Groovy Scripting as build our integration flow further

Hello technology hitchhikers!

Welcome to the third part of Enterprise Integration with SAP CPI. In the first two parts (read the first and second parts if you haven’t already) we established our integration scenario, divided the approach into 7 steps and also discussed the first three steps of the solution. We learned about Message events, Start Timers and some scheduling options, Request Reply shape, Groovy Script Shape, General and Iterative Splitters while creating our integration flow.

In this part, we are going to discuss the 4th step below:

1. Configure the Integration to run every day
2. Obtain the list of products
3. For each product in the list, identify if reordering is required
4. If product must be reordered, obtain supplier info
5. Create a list of the products to be reordered along with supplier info
6. Convert List into CSV format
7. Send List to Manager via Email

Till now, we have extracted individual products and identified if they require to be reordered or not. If a product needs to be reordered, we set message property needsReorder to true. Otherwise it is set to false.

Step 4: If a product needs to be reordered, obtain supplier info

(Technical concepts used: Message Router, Content Modifier, Gather, Content Enricher)

Now that we know which products do not need to be reordered, we can safely discard those products from this integration flow as they need no further processing. To separate those products from the ones that need reordering, we use a message router.

Very often, we come across scenarios where we want to separate messages flowing through our integration based on some condition and subject them to different treatment. Message router is a shape that’s used to send messages to different routes based on specified criteria. You can read about Routers here.

To add Message Router in our integration flow, click on Message Routing->Router.

Place the Router shape between Groovy Script shape and Message End shape.

Before we create different routes, we must decide how many routes are needed, which messages are to be sent on those routes and what operations are to be performed on them. As discussed earlier, we are going to create two routes- one for products which need to be reordered and one for those which don’t. We are aware of the actions to be taken on the products that need reordering. We also know that the products that don’t need reordering can be discarded/nullified.

We are going to use a content modifier shape to nullify the messages for products which do not need reordering. As the name suggests, a content modifier shape allows us to modify a message by changing its body, headers and properties. In this case, we want to change the message body to null.

To add a content modifier, click on Message Transformers->Content Modifier.

Place it between the message router and the end message shapes.

Now click on the content modifier, go to its ‘Message Body’ tab and set the value of Body as null.

If you save the integration flow at this point, you will see error marks on the shapes and connectors:

This is because we haven’t specified the route condition for this route and also because there is only one route present. Click on the connector labelled as ‘Route 1’ and under ‘Processing’ tab, select ‘Default Route’. This means that all messages must go down this route if they do not match any other route condition. The next logical step is to add a route for products which need to be reordered.

We had split the list of all products into individual products for assessment, but we need to send a consolidated list to the store manager at the end. For this purpose we use a Gather shape. Gather shape is used to collect messages on one or more routes in a manner specified in its configuration.

Click on Message Routing->Gather and place it on the canvas like this:

Click on the router shape and draw a connector to Gather, it’ll be named as ‘Route 2’ automatically.

Click on Route 2 and under Processing Tab, set Condition to : ${property.needsReorder} = ‘true’

Thus, we have specified that if message property needsReorder is set to true, the message has to follow this route, otherwise the default route is taken. Also, every route must end at some point. Draw a connector from Gather Shape to Message End shape to complete the route.

Now, we are going to change the connector between Content Modifier and End shape to connect to Gather shape instead. Why should we do this, you ask? Logically, we could have kept it as it was. We were nullifying message bodies for products that don’t need to be reordered, and we were ending the process for those messages after that.  But, this would have led to multiple messages reaching the End Shape. Suppose there were 80 products that didn’t need reordering, we would have had 80 messages reaching the End Shape apart from the consolidated list that we prepare for the products which need reordering. That could have led to problems in tracing these messages, or worse, 80 unnecessary Emails!

We need to configure the Gather shape and specify its Incoming Format and Aggregation Algorithm. We know that we wish to Gather messages containing details of products and also some messages with nullified body and that they should be concatenated to form a single message, hence we are going to set the Incoming Format to ‘Plain Text’ and Aggregation Algorithm to ‘Concatenate’. You can use other options as well based on their relevance and your preference. You can read more about Gather Shape here.

Even though we have added a gather shape, we have not added shapes for the operations to be performed on messages following route 2 before they are gathered. The goal is to add supplier information to each of these messages and this can be divided into the following tasks:

1. Obtain Product Id from Message
2. Obtain Supplier Info using Product Id
3. Form message bodies which are suitable for concatenation

1. Obtain Product Id from Message

For this purpose, we are going to use a Content Modifier. As discussed before, a content modifier shape is used to modify the message body, properties and headers. We have already used it to set the message body to null. Now, we are going to extract Product Id from message body and store it in message properties.

Add a content modifier between Router and Gather shapes.

We are going to use an XPath Expression to extract Product Id from Message body: /Products/Product/SupplierID

2. Obtain Supplier Info using Product Id

For this purpose, we are going to use a shape called ‘Content Enricher’, which gets data from an external source(Lookup Message) and adds it to the existing data(Original Message) body. In our case, we want to add Supplier information to Product details that we currently have in message body.

To add a Content Enricher, click on Call->External Call->Content Enricher and place it between Content Modifier and Gather Shapes:

Since we are going to get data from a remote source, we will add a receiver just like we had added in the first step of the integration flow to obtain product information. Add a receiver and then, an OData adapter from the receiver to the content enricher.

Click on the OData Adapter to configure the connection. The configuration is very similar to Step 2 discussed in the first part of this blog series, but instead of  ‘Products’, we obtain data from ‘Suppliers’.

Now, we select the aggregation algorithm for the content enricher. There are two options for aggregation algorithm:

1. Combine: On using this option, the Lookup Message is appended to the Original message.
2. Enrich: On using this option, the Lookup Message is merged with the Original message using a common field.

In our scenario, we want to merge Supplier details into Product Information using Supplier Id as the common field and hence, we are going to select Enrich as our aggregation algorithm. You can read more about Content Enricher here.

For both Enrichment and Original Message, we have to provide Path to Node where entity data is contained, like Product Details and Supplier Details in our case. Key element is the name of the element which must be matched for merging. The configuration looks like:

At this point, we have merged supplier information in the product information and the enriched data looks like:

<?xml version='1.0' encoding='UTF-8'?>
<Products>
	<Product>
		<CategoryID>8</CategoryID>
		<Discontinued>false</Discontinued>
		<SupplierID>13</SupplierID>
		<Supplier>
			<SupplierID>13</SupplierID>
			<CompanyName>Nord-Ost-Fisch Handelsgesellschaft mbH</CompanyName>
			<Address>Frahmredder 112a</Address>
			<Phone>(04721) 8713</Phone>
			<Region/>
			<PostalCode>27478</PostalCode>
			<Country>Germany</Country>
			<City>Cuxhaven</City>
			<ContactName>Sven Petersen</ContactName>
			<ContactTitle>Coordinator Foreign Markets</ContactTitle>
		</Supplier>
		<UnitPrice>25.8900</UnitPrice>
		<ProductName>Nord-Ost Matjeshering</ProductName>
		<QuantityPerUnit>10 - 200 g glasses</QuantityPerUnit>
		<UnitsOnOrder>0</UnitsOnOrder>
		<ProductID>30</ProductID>
		<ReorderLevel>15</ReorderLevel>
		<UnitsInStock>10</UnitsInStock>
	</Product>
</Products>

3. Form message bodies which are suitable for concatenation

We have product details along with the details of suppliers but the resulting XML is in nested format, where the supplier details are not at the same hierarchical level as the rest of the product details. It is important to have all the data at the same level so that the message can be effectively converted into CSV format. Hence, the following tasks must be performed on this data:

1. Get all relevant data at the same hierarchical level in the XML file.
2. Remove Duplicate data, if any.

To achieve this, we are going to use the following groovy script:

import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import groovy.util.XmlSlurper ;
import groovy.xml.XmlUtil;
def Message processData(Message message) {
    //Body 
       def body = message.getBody(java.lang.String) as String;
       def rootNode= new XmlSlurper().parseText(body);
       def product=rootNode.Product;
	   	   
	   product.SupplierID.replaceNode{
	   };
	   product.appendNode(product.Supplier.SupplierID);
	   product.appendNode(product.Supplier.ContactName);
	   product.appendNode(product.Supplier.ContactTitle);
	   product.appendNode(product.Supplier.CompanyName);
	   product.appendNode(product.Supplier.Address);
	   product.appendNode(product.Supplier.Phone);
	   product.appendNode(product.Supplier.PostalCode);
	   product.appendNode(product.Supplier.Country);
	
	   product.Supplier.replaceNode{};
	   def flatXML=XmlUtil.serialize(product);
       flatXML=flatXML-"""<?xml version="1.0" encoding="UTF-8"?>""";
       flatXML=flatXML-"""<?xml version='1.0' encoding='UTF-8'?>""";
       message.setBody(flatXML);
       return message;
       }

In this script, we first use XMLSlurper to read the XML body so as to extract relevant Supplier Information from Supplier Node added during Enrichment. We then add this information to Product Node so that we have all required information at the same hierarchical level. This is done using appendNode method of XMLSlurper. Once we have the required information in the format that we desire, we remove the redundant data nodes using replaceNode.

If you look at the Script, you will realize that we removed the node containing Supplier Id information before appending Supplier Data. This is done to avoid creation of redundant data. But you may be wondering why we added Supplier Id data if it was already present in the first place. When we append Node using XMLSlurper, the data is added at the end. In our case, all the supplier information is added to the end of Product Node, thus keeping all this data together. When we convert this data into CSV format at a later stage, it will be logical to have Supplier Information at the same place instead of being scattered throughout the Product Information. Please note that this is not the only way to achieve this result and we will discuss the other ways at a later stage.

That’s it! We’re going to pause this blog here so that you can fully understand and appreciate the topics we came across. In this blog post, we learned about Routers, Content Modifiers, Content Enricher and a little more of Groovy Scripting. In the next blog post, we will learn about XML to CSV Conversion and Mail Adapter among other things as we conclude our journey of Enterprise Integration using SAP CPI. Till then, goodbye and happy learning!