Dataweave transformations
Contents:
1) Iterating payload using Map Operator
2) Iterating Payload using MapObject Operator
3) Processing payload using Pluck Operator
4) Filtering the Payload
5) Using OrderBy operator
6) Using GroupBy operator
7) quiz
Enterprise systems are quite complex, they consist of multiple applications interact with each other. The messages those flown between these systems are complex in nature. Consists of multiple records information.
Example: Collection of orders, Collection of items, Collection of persons etc.
Scenario: Mr. Arun working as a software engineer, he is smart and handsome guy desperately looking for marriage alliances. He developed an application that will call the matrimony site and gets the girls details, Now Arun wants to choose his life partner. Let us understand how Dataweave helps him in this process.
The Data he got from matrimony service is as below
<?xml version="1.0" encoding="UTF-8"?>
<matrimony>
<brides>
<bride>
<name>Rakulpreet</name>
<age>25</age>
<educationQualification>Btech</educationQualification>
<profession>BankPO</profession>
<height>170</height>
<city>Hyderbabad</city>
</bride>
<bride>
<name>PoojaHegde</name>
<age>26</age>
<educationQualification>MBA</educationQualification>
<height>168</height>
<monthlyIncome>350000</monthlyIncome>
<city>Bangalore</city>
</bride>
<bride>
<name>Katrina</name>
<age>30</age>
<educationQualification>MA</educationQualification>
<siblings>2</siblings>
<height>171</height>
<city>Mumbai</city>
</bride>
<bride>
<name>Sonakshi</name>
<age>24</age>
<educationQualification>MBBS</educationQualification>
<height>167</height>
<monthlyIncome>250000</monthlyIncome>
<city>Mumbai</city>
</bride>
</brides>
</matrimony>
The transformation engine needs to Iterate the data and process each record. Dataweave map operator is used to process each record in the collection.
1) Iterating payload (Brides) using Map Operator:
Arun want to save brideName, age, qualifications to csv file. To save these details, need to pull them from payload and save to file
In this example using map operator we will iterate over each bride and retrieves the required data. Map operates needs to collection to iterate, so all brides should be given as input ( payload.matrimony.brides.*bride) .
here $ represents the bride object from which information getting pulled.
Ex: while retrieving the first bride details, the first bride data becomes $
Expand : $ means
The Flow code as below. Change the highlighted path according your machine
<flow name="basic_transformFlow1" doc:id="323d0dfb-1ed6-42b3-9b73-7f29ba5a1db9" >
<http:listener doc:name="Listener" doc:id="0548b13c-1729-424d-b325-5988be8652c9" config-ref="HTTP_Listener_config" path="/bride" outputMimeType="application/xml"/>
<ee:transform doc:name="Transform Message" doc:id="d2963243-4907-4def-9d2d-72b1bf317f1a" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/csv
---
payload.matrimony.brides.*bride map {
BrideName: $.name,
BrideAge: $.age,
Qualification: $.educationQualification
}]]></ee:set-payload>
</ee:message>
</ee:transform>
<file:write doc:name="Write" doc:id="9168432e-c2f6-4d2a-802a-4ebe42c027ab" path="C:\AnypointStudio\AnypointStudio\My-workspace\basic_transform\src\main\resources\brides.csv"/>
</flow>
Iterating payload (Brides) using MapObject Operator:
Arun wants to save the entire brides information to the csv file. He needs to define the column names in the csv file,
Column names should be derived from the Key’s of the payload.
mapObject operator is used to iterate over key-value pairs.
Here MapObject operator is used to iterate over a single bride (Key-value pair). whereas Map operator is used to iterate over collection of brides. In Below example
Expand image here : Map Object demo
The dataweave code as below:
%dw 2.0
output application/csv
---
payload.matrimony.brides.*bride map(
$ mapObject {
'$$' : $
}
)
3) Processing payload using Pluck Operator :
The details set provided by girls differs from one to another, for example some girls providing monthly income details and income field missing in some girl’s details, similarly details like profession, siblings etc. Now Arun wants to get the fields provided by the girls. Using pluck operator, he will fetch the details by passing girl name as input
Expand image : Pluck Example The Dataweave code as below:
%dw 2.0
output application/json skipNullOn="everywhere"
var brideName = attributes.queryParams.brideName
---
{
Details_Provided : payload.matrimony.brides.*bride map(if ($.name == brideName)
{ provided_details : $ pluck $$ }
else null)
}
4) Filtering the Payload:
Arun needs to take decision by girl’s qualification, age, city etc. Dataweave filter operator will help Arun to pick up his right choice. Let us see how he is going to apply filter operator to shortlist the bride
Arun has different kinds of preferences, so he will pass his preferences at runtime, Dataweave need to filter data according to his requirement. Example: he is looking for girl aged less than 26 years and belongs to Mumbai
The below code as various filter to show up all his requirements
The Dataweave code as below:
%dw 2.0
output application/json
var age = attributes.queryParams.age
var height = attributes.queryParams.height
var city = attributes.queryParams.city
var qualification = attributes.queryParams.qualification
---
{
ageQualified : if (age !=null) payload.matrimony.brides.*bride filter ($.age <= age) else 0,
heightQualified : if(height !=null) payload.matrimony.brides.*bride filter ($.height >= height) else 0,
preferredCity: if(city != null) payload.matrimony.brides.*bride filter ($.city== city) else 0,
qualification: if(qualification !=null) payload.matrimony.brides.*bride filter
($.educationQualification== qualification) else 0,
City_and_age_satisfied: if(age !=null and city !=null)
payload.matrimony.brides.*bride filter ($.age <= age and $.city == city) else 0
}
5) Using OrderBy operator:
Arun wants to sort the bride details on basis of age, salary to decide. Arun will send this order parameter through query param. this parameter would be collected through the dataweave variable.
The Dataweave code as follows
%dw 2.0
output application/json
var brides = payload.matrimony.brides.*bride
var param = attributes.queryParams.param
---
if(param=='name') brides orderBy $.name else
if(param=='age') brides orderBy $.age else brides
The request
http://localhost:8081/orderBy?param=name
returns the below response, ordered by bride name
[ { "name": "Katrina", "age": "30", "educationQualification": "MA", "siblings": "2", "height": "175", "city": "Mumbai" }, { "name": "PoojaHegde", "age": "26", "educationQualification": "MBA", "height": "168", "monthlyIncome": "350000", "city": "Bangalore" }, { "name": "Rakulpreet", "age": "25", "educationQualification": "Btech", "profession": "BankPO", "height": "170", "city": "Hyderbabad" }, { "name": "Sonakshi", "age": "24", "educationQualification": "MBBS", "height": "167", "monthlyIncome": "250000", "city": "Mumbai" } ]
please test this code using this url
http://localhost:8081/orderBy?param=age
6) Using GroupBy operator:
Arun want to see the brides grouped by city, qualification etc .
%dw 2.0
output application/json
var brides = payload.matrimony.brides.*bride
var param = attributes.queryParams.param
---
if(param=='city') brides groupBy $.city else
if(param=='qualification') brides groupBy $.educationQualification else brides
http://localhost:8081/groupBy?param=city
{
"Bangalore": [
{
"name": "PoojaHegde",
"age": "26",
"educationQualification": "MBA",
"height": "168",
"monthlyIncome": "350000",
"city": "Bangalore"
}
],
"Mumbai": [
{
"name": "Katrina",
"age": "30",
"educationQualification": "MA",
"siblings": "2",
"height": "175",
"city": "Mumbai"
},
{
"name": "Sonakshi",
"age": "24",
"educationQualification": "MBBS",
"height": "167",
"monthlyIncome": "250000",
"city": "Mumbai"
}
],
"Hyderbabad": [
{
"name": "Rakulpreet",
"age": "25",
"educationQualification": "Btech",
"profession": "BankPO",
"height": "170",
"city": "Hyderbabad"
}
]
}
Quiz:
1) If Matrimony service returns duplicate brides information then How Arun will get distinct brides, please provide the dataweave expression.
2) How to get average age of brides ?
3) How to get total number of brides returned from the service ?
4) How to retrieve brides who earning more than 100000
No comments:
Post a Comment