Kafka consumer transformation exception

Last Updated on 30/03/2020 by Patryk Bandurski

Transform Message is a very powerful component in Mule 4. It gives a lot, but with the new DataWeave 2.0 language, we need to be careful. Sometimes we lack metadata or metadata is insufficient and Transform Message component may behave differently as we expect. What is more we may receive transformation exception.

Case

In the screenshot below you can find my simple flow. As a listener, I have configured Kafka’s consumer. After I receive an event I try to transform the incoming payload into a JSON object. After that, it is passed for further processing.

My incoming event looks like:

{
  "customerId": 1234567, 
  "orders": [
     {
       "id": 1,
       "user": "USR-1"
     },
     {
       "id": 2,
       "user": "USR-XY"
     }
  ]
}

Success! My application is deployed.

Oh no, I received DataWeave execution error! You can see it below. It says that I tried to access customerId property on Binary object. However, the engine is not able to translate the binary object into a JSON object. It makes sense. Kafka connector creates an event message that does not have a content type. The engine could guess, what the content type is, based on the payload, however it could be error-prone.

Message: "You called the function 'Value Selector' with these arguments: 
  1: Binary ("{\n  \"customerId\": 1234567,\n  \"orders\": [...]"...)
  2: Name ("customerId")

But it expects one of these combinations:
  (Array, Name)
  (Array, String)
  (Date, Name)
  (DateTime, Name)
  (LocalDateTime, Name)
  (LocalTime, Name)
  (Object, Name)
  (Object, String)
  (Period, Name)
  (Time, Name)

5| 	customerId: payload.customerId,
              ^^^^^^^^^^^^^^^^

MIME Type solving the issue

So the question is how to set content type. We can drive that by setting incoming MIME type. Unfortunately, Kafka connector does not allow to set such metadata. By contrast, JMS connector does allow that. We need to add an additional event processor at the beginning of the flow.

The most convenient would be either Set Payload or Transform Message processor. Below you can see how I used set-payload to change MIME type. We need to set it by using the mimeType attribute.

<set-payload value="#[payload]" doc:name="Set Payload" mimeType="application/json"/>

Now our event will be processed without any problem at all.

Summary

Mule works best when he has metadata. When something is missing you may end up with a project that does not work. Like in the example above. In my opinion, Kafka connector’s team should add such configuration into their next release. However, we can simply bypass that by adding either Set Payload or Message Transformer event processor.