Last Updated on 09/01/2021 by Patryk Bandurski
MuleSoft and its Partners are publishing many connectors. It’s great as we can quickly and with less overhead start integrating with 3rd parties. However, there are situations where no connector exists. This may happen for smaller or bigger systems. I come across that challenge while trying to call AWS service through its REST API. This was not an obvious task. Authentication was a hard nut. In this article, I go with you on the journey of singing AWS API Request using Signature Version 4.
Signing AWS Request
In the AWS documentation, you can find thorough chapters on different ways of signing requests. Recommended is using Signature Version 4. The process is divided into four steps:
- Create a canonical request – a unified way of representing the request
- Create a string to sign – metadata and canonical request
- Calculate the signature
- Calculate the authorization header
In the next subsection, I will briefly explain what is happening at each stage. Alongside I will present part of the DataWeave that I have prepared. Full code you can find on my GitHub here.
Create a canonical request
The canonical request is a unified way of representing the request. AWS uses it to compute and compare the signature. Down below, you can see a simple canonical request for one of my POST requests. Let’s break it down, line by line.
POST
/
content-type:application/x-amz-json-1.1
host:ssm.us-east-1.amazonaws.com
x-amz-date:20210107T201016Z
x-amz-target:AmazonSSM.GetParametersByPath
content-type;host;x-amz-date;x-amz-target
283926535ca9b8eb8561d16c8cdc45983791ffd96d3dc0f0923076a34115a720
- HTTP Method like POST, GET, etc
- Canonical URI – I am calling https://ssm.us-east-1.amazonaws.com/, so in my case, just the forward slash.
- Canonical query string – not applicable for my POST method
- Canonical headers – sorted by a header name
- Canonical headers – sorted by a header name
- Canonical headers – sorted by a header name
- Canonical headers – sorted by a header name
- Line break
- Signed headers – keys from line 4 – 7 separated by the semicolon
- Signature
In my DataWeave Auth module it looks like this
fun canoncialRequest(method: String, uri: String, querystring: String, headers: Object, request: Binary) =
"$(method)\n$(canoncialUri(uri))\n$(canonicalQueryParameters(querystring))\n$(canoncialHeader(headers))\n\n$(headerKeys(headers))\n$(hash(request))"
This function takes all necessary input to compute the canonical request. As you can see, I am calling other helper functions like a hash that computes the signature.
Create a string to sign
At this step, we compute the string to sign. As an input serves metadata and canonical request. Let’s break it down as we did it previously. Line (1) is the hashing algorithm. In my case, that is AWS4-HMAC-SHA256. Next line (2) is the request’s date and time. In line 3, you can see so-called credentials scope. That is the date, the AWS region, the service, and fixed postfix aws4_request. The last line (4) is the signature from the previous step (Create a canonical request).
AWS4-HMAC-SHA256
20210107T204740Z
20210107/us-east-1/ssm/aws4_request
10f06b1502f5a4ce7a6717eed616cadbaba916295d84d09598c32cfb3f4db9f7
In the Auth module, the function responsible for computing string to sign looks like this:
fun stringToSign(date, dateStamp, region: String, service: String, canonicalRequest) =
"$(algorithm)\n$(date)\n$(dateStamp)/$(region)/$(service)/aws4_request\n$(hash(canonicalRequest))"
Calculate the signature
It is now time to calculate the signing key that we will use at the next stage and the string to sign. Here is the DataWeave function responsible for it. As you can see, it is an embedded callout. In line 5, we compute HMAC using AWS Secret Key and request date. The output is the parameter for the next HMAC function that is hashed using the region as a key (lines 4, 6). Next, the output is used to compute hash using AWS service name as a key (lines 3, 7). As the last step, the output is hashed using the aws4_request string as a key (lines 2, 8).
fun getSignatureKey(key, date, region, service) =
sign(
sign(
sign(
sign('AWS4$(key)', date),
region),
service),
"aws4_request")
Under the sign function is hidden simple following reusable piece
fun sign(key: Binary, msg: Binary): Binary =
HMACBinary(key, msg, "HmacSHA256")
Calcualte the authorization header
This is the simplest step. We compute the signature using the HMAC function with string to sing and signing key computed in two previous steps. Then, we can generate the Authorization header. Let’s see the example first
AWS4-HMAC-SHA256 Credential=AKIATC6AQYSJSFEG5X54/20210107/us-east-1/ssm/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=45f603fb75e3d7656024129ff800d87730087c37d4637f71b9ef33cccc4a55b9
We can compare it with template: algorithm
Credential=access key ID
/credential scope
, SignedHeaders=SignedHeaders
, Signature=signature
.
Using Auth module
As a starting point, you can use my Auth module computing authorization header. The provided module is a simplified version that can be easily extended. Currently, it accepts only requests with bodies like POST, PUT, etc. You should use the generateSecureAWSHeaders function with the following parameters:
- HTTP method
- AWS region
- AWS service
- Operation/Action
- URI
- query parameters
- request body
- AWS Access Key ID
- AWS Secret Key
I put this module Auth in resuable asset and the file Auth.dwl is stored under following directory dw/com/ambassadorpatryk/aws. Here is a simple callout
%dw 2.0
output application/json
import generateSecureAWSHeaders from dw::com::ambassadorpatryk::aws::Auth
---
generateSecureAWSHeaders(
'POST',
'us-east-1',
'ssm',
'AmazonSSM.GetParameter',
'/',
'',
payload,
Mule::p('secure::aws.accessKey'),
Mule::p('secure::aws.secretKey'))
Performance
I generated a thousand Authorization headers ten times. I used CloudHub, having just 0.1 vCore and 500 MB. The average to generate the header is five milliseconds. Down below you can see a chart with time to generate a thousand headers in each test series.
Summary
Although the number of MuleSoft connectors is rising, we still need to call services directly via REST or SOAP interfaces. It is true even for such big product platforms as AWS. Not every AWS service has a dedicated connector. For those who do not have it, we need to go through the API specification to get to know how to call it. I went through this to find out how to construct the authorization header. AWS splits it into four steps. All can be done using plain DataWeave language. In this article, you can find a reference to my DataWeave module implementing a simplified version.
Hope you enjoy it. Cheers
Hi, this was really helpful article. DO you have blog on how we can call rest aws api using the signature generated
Hi, sorry for the late reply. This I plan to put in one of my next articles. It is nearly ready
Do you have blog or steps laid out on how we can call rest aws api using the signature generated?
Hi Patryk,
should the presented apporach work online in dwlang.fun?
Sorry about very late reposne. Message landed in SPAM. Unfortunately Wojtek, DW playground have an issue resolving request.^raw 🙁
Hey Patryk!
I just ran into a use case where this was very helpful. Thanks for writing it up! It saved me a lot of time.
I had to make two updates to the code to make it work:
1) I was using a JSON payload, so request.^raw returned a null value. This was compounded by issues around using the write function, which by default adds in newline and indentation characters, which messes up the canonical request and hashes. I ended up using ‘write(payload, “application/json”,{“indent=false”})’ as my input to the ‘request’ parameter of generateSecureAWSHeaders, which worked for me.
2) The AWS service I was using (and from the documentation, all of them now) require an additional AMZ header, which is x-amz-content-sha256. If you are sending your data in multiple chunks, this is more difficult, but with a single chunk of data, this is as simple as adding the following code after line 25 in your Auth module:
“x-amz-content-sha256: hash(request),”
Hi,
it is great to hear that it was helpful :). Thanks for extra remarks regarding the DW code.
Hi,
is anyone defined the following two functions and how to pass the query parameters to the function from API URL/Function URL
/*
* TODO!
*/
fun canoncialUri(uri) =
‘/’
/*
* TODO!
*/
fun canonicalQueryParameters(queryParameters) = ”
Hi,
I did not implement these functions as they were not needed in my specific case. I was following Amazon documentation explaining how to construct target value – there is an example how query parameters should be combined (“Example: Request with authentication parameters in the query string”)