Last Updated on 30/03/2020 by Patryk Bandurski
At some point, you may have a service that returns a lot of fields and related objects. What if the service consumer doesn’t want all the fields all the time? In other words, he/she would like to have the response filtered. Here come the Content Filter to the rescue. After you read this article you will find some ideas about how to use it in your own scenarios. So let’s roll.
Content Filter
This is the opposite pattern to Enrchier Pattern – more about it you can find here. In this particular scenario, we want to simplify our response. Remove unnecessary information that the receiver doesn’t need.
As you can see in the diagram above, before filter we have a message with 3 elements and after the filter is applied we have only one element. The magic is happening in the black boxes called Content Filter. However, we as designers should know how this particular element works. So, how to define filter?
We may choose two ways of doing this:
- positive filter – we specify what we would like to have in the response
- negative filter – we specify what should be removed from the original payload
Filtering in Mule 4 using DataWeave 2.2
In order to filter out some fields, we have DataWeave language. So for filtering purposes, we will use the Transform Message component. We can filter properties from objects and arrays. So let see how to do the positive and negative filtering in DW.
Negative filtering
In DataWeave we have two operators – and –. The first one allows for removing a key-value pair by providing the key.
Okay, that way I can remove one field at a time. In order to remove two fields like in the example above I would need to do it in the following way:
%dw 2.0
output application/json
---
{
"id": "idy4234-2",
"title": "Meetup"
} - "id" - "title"The double dash operator (–) allows providing keys array. In other words, each key present in the array will be removed from the supplied object.
In the example above we have provided an array with title and id keys. As a result of applying — operator, we got the empty object.
Positive filtering
This is the opposite scenario in contrast to previously described negative filtering. This time we would like to define which properties should stay, not mentioned ones should be removed from the final outcome.
We don’t have a similar operator to perform that action. filterObject function is the best match for that case. This function iterates over each key and applies lambda expression. If the condition is evaluated to true, include the field in the output. Otherwise, omit it.
Below you can see a code snippet showing how to do a positive filter. We instruct filterObject to include the key-value pair only if the key is present in the array – here “id”.
%dw 2.0
output application/json
---
{
"id": "idy4234-2",
"title": "Meetup"
} filterObject ((value, key) -> ["id"] contains key as String)Filter fields in your REST service
I have looked for may publicly available services to see some good guidelines to do the filter. However, a lot of them are too complex or simple. Here what you can see, is some version in between. Below you can see a trait filter definition.
As you can see I have decided to introduce the default value *all indicating that all fields will be returned. To distinguish negative and positive filtering I have decided to introduce – (minus) prefix before fields to remove from the final response. Values are separated by the comma without any additional space.
Warning
We can’t mix the positive and negative filters. I have decided on this rule because it makes sense. Either we want to specify what should be returned or skipped.
Here is the link to the trait filterable. You can place it some common traits library or in specif API directly.
Use trait in your API specification
In order to use an externally defined trait, you need to include it in the API specification. You will do this like in the snippet below:
#%RAML 1.0
title: Consent Experience API
version: v1
baseUri: https://sfdc-consents.profit-online.pl/api/{version}/
mediaType: [ application/json ]
types:
consent: !include types/consent.raml
traits:
filterable: !include traits/filterable.raml
As you can see declared filterable trait in traits section. How to apply this to the resource?
You need to specify the array of traits using is, like in the second line on the snippet below.
/consents:
is: [filterable]
get:
responses:
200:
body: consent[]Example usage
The first case is the call to retrieve all the fields
GET /mongodb/sapi/consents HTTP/1.1
Host: ambassadorpatryk.comor
GET /mongodb/sapi/consents?filter=*all HTTP/1.1
Host: ambassadorpatryk.comThe second case is to retrieve only specified fields
GET /mongodb/sapi/consents?filter=id,name,title HTTP/1.1
Host: ambassadorpatryk.comThe third case is to retrieve response without the specified fields
GET /mongodb/sapi/consents?filter=-details,-id HTTP/1.1
Host: ambassadorpatryk.comSummary
I think that the presented solution for filtering fields gives some flexibility but isn’t overcomplicated. In many publicly, available APIs providers give the possibility to decide which fields are expected at the response. In the next article, I will introduce my filter DataWeave module. Stay tuned!
If you have any other ideas on how to introduce filtering fill free to share it. Let’s learn from each other.
Have a great day!