DataWeave Modules, new way to introduce reusability

Last Updated on 30/05/2021 by Patryk Bandurski

It is always desirable to reuse as much code as possible. We can do the same regarding DataWeave transformations and custom functions. In this article, I will describe how to reuse code in the newest DataWeave using modules. For those working with DataWeave 1.0, I will describe how to reuse code with readUrl function.

DataWeave Module

With Mule 4 we got a lot of new improvements. The same is with DataWeave 2. MuleSoft has introduced the modules concept. All functions prepared by the MuleSoft team have been divided into logical pieces and placed within modules as in the diagram below.

The Core module is a special one. This module is imported by default in your transformations. In contract, other modules you need to import explicitly. All modules and their functions are fully described on the MuleSoft documentation page. If you were reading my other articles, you may find that I have been using functions from different modules. For example, here I used the isEmpty function that is from the core module.

Implementation

Custom modules are separate dwl files. You place them in src/main/resources catalog in for example dw folder. However, you can use the package name whatever you like for example modules/json/utils.  

Your custom module may contain only:

• variable declaration – var
• functions – fun
• namespace declaration – ns
• custom types – type

In such a file, output directive is not permitted, as well as headers-body separator —.  Because the custom module file contains only the body. Below you can see an example DataWeave file.

%dw 2.0

ns blogns https://ambassadorpatryk.com/blog
var protocol = "https"

fun appendURL(url: String) =
	protocol ++ "://" ++ url ++ "/test"

Let’s see what is going on there. First, we have the DataWeave version declaration – %dw 2.0. This is the only directive with a percent sign. Then we have blogns namespace defined. In the fourth line, we have a protocol variable that is used later on in appendURL function. This function concatenates the string to a nice URL. Let’s compare it with standard transformation.

%dw 2.0
output application/json skipNullOn="everywhere"
---
{
	userAge: payload.age
}

In the transformation, we have an output directive. In contrast, the custom module doesn’t have such declaration.  What is more, in the transformation we have headers-body separation, whereas in the module we do not have.

Ok, so we have defined our own custom module. How to use it?

Usage

In order to use functions, variables, etc defined in any module, we have import directive. It can be used in two different ways:

• whole module import,
• import specific parts from the module.

Here, how it looks in code:

• import dw::core::Arrays
• import * from dw::core::Arrays
• import countBy, sumBy from dw::core::Arrays

First, two examples import the whole content defined in the Arrays module. The last one, on the other hand, imports only two functions by name. Imagine that I have put my filter.dwl file in src/main/resources/dw1/json/utils. How should I import it?

import dw1::json::utils::filter

As you can see the structure is the same as directory tree, but instead of slash we use double colons. 

After import, we would like to use it. When we did not specify what functions to import we refer to them by module name. Let’s see an example with the Arrays module.

%dw 2.0
output application/json skipNullOn="everywhere"
import dw::core::Arrays
---
Arrays.countBy($ > 2)

In case of importing specific function from the module we do not need to use module name. Like in the example below.

%dw 2.0
output application/json skipNullOn="everywhere"
import countBy, sumBy from dw::core::Arrays
---
countBy($ > 2)

Reuse in DataWeave 1.0

In the previous version of DataWeave reusability was possible by loading files from the classpath by using readUrl function. So, how to share DataWeave transformation?

Implementation

Unlike DataWeave 2.0, the file should have headers and body sections separated by three dashes. In the header section, you may implement functions, create variables, and declare a namespace.

%dw 1.0

%namespace blogns https://ambassadorpatryk.com/blog
%var protocol = "https"

%function appendURL(url)
	protocol ++ "://" ++ url ++ "/test"
---
{
	appendURL: appendURL,
	blogns: blogns
}

In the example above, in our library are defined:

• appendURL function,
• protocol variable with value “https”,
• namespace.

In the body section, we need to declare what will be accessible outside the library. We do this, by defining an object with properties and assigned references to the function/variable/namespace. 

Have you notice what is missing in the mapping file, above? Yes, you are right we do not provide %output directive.

Usage

In order to load the DataWeave file use readUrl function. It takes only one argument. It is a URI to the resource. URI contains two parts. First is the protocol. Mainly it is a classpath. The second part is the path to the dw file.

%dw 1.0
%output application/json skipNullOn="everywhere"

%var filterLib = readUrl("classpath://dwl/json/utils/emptiness_filter.dwl")

---
filterLib.filterEmptyObjects({"myprop1": "super value", "obj": { }})

In the example we are loading emptiness_filter.dwl file from classpath located in the dwl/json/utils folder.

Summary

I like the idea of Modules in DataWeave 2. Modules are similar to the Java packages concept and make managing code easier. We are able, to create custom modules. In contrast to DataWeave 1.0, we are not able to decide what should be accessible outside of the file. So module will publish its full content. In that case, some parts can be rewritten so that only desired parts are exported.

If you have reusable pieces of code. Extract it to an external file. If this is something that can be shared for more than one project, you can embed files into a jar and share them among projects. In that scenario, you use the code the same way as described above.