You might say I’m a late adopter of GraphQL. I’ve heard a lot of talk about it and somehow everybody that uses it seems to be extremely enthusiastic about it. But I never got around to implementing it for any of my projects. Recently that changed. I was working on a new application, read up on GraphQL and AppSync and decided to use it as the foundation of my project. And boy, did I wish I had done that sooner.
Over the last 15 years I’ve built many APIs. First in PHP on NginX, then on Python on NginX, then Python on Lambda and API Gateway. Like many before me I found that it is very hard to design and maintain a consistent, easy-to-use REST API. And don’t get me started on documentation. All of these problems have simply vanished by using GraphQL and frankly I don’t see myself using REST APIs for any CRUD-like application ever again.
AWS’ managed GraphQL service is called AppSync. Extending on all the benefits of bare-bones GraphQL, it provides the heavy lifting in hosting, scaling and integrating with other services. These services include IAM, Cognito, DynamoDB, Lambda, RDS and Elasticsearch.
In my project I implemented three features that I believe might be valuable for others as well. I will cover these in a three-part series, of which this is the first installment. The three features are:
- Restricting Access with OAuth Scopes and VTL
- Implementing a Generic String Filter in Python
- Minimizing Data Transfer on all Layers
These blog posts are supported by an example implementation in Python CDK. You can find it on GitHub. It has the following components:
- An AppSync API
- Four microservices (Python Lambda) in a shared codebase
- A DynamoDB Table with Single Table Design
- A Cognito User Pool with:
- One User Pool Client for human logins
- One User Pool Client for machine-to-machine communications with read permissions
- One User Pool Client for machine-to-machine communications with write permissions
All the code and implementations in these three blog posts can be found, reviewed and deployed through the GitHub project. And now, without further ado, the first feature: OAuth Scopes, VTL, and GraphQL.
Restricting Access with OAuth Scopes and VTL
Our GraphQL Playground mimics an inventory API. You can add items (books or cars) to the inventory, and you can query which books and cars have been stored in the database. The four operations to achieve this are AddCar, AddBook, GetCars and GetBooks.
In our example, human users are allowed to read and write. There are also external systems that can interact with our API without human intervention. Some of these systems are allowed to read, others are allowed to write, but none are allowed to do both.
To allow machine-to-machine clients to authorize with AppSync, we have implemented a Cognito User Pool Client with limited OAuth scopes. In the Cognito console this is displayed as follows:
As you can see, this user pool client is allowed to authenticate with client credentials, which are non-rotating simple keys. When a machine does authenticate through this user pool client, it will be assigned the scopes/items:read scope. If you have deployed the GraphQL Playground from GitHub you can try this by running the following command in terminal, replacing the placeholders with the values from your Cognito User Pool:
curl --silent -X POST \
"https://<your-prefix>.auth.<your-region>.amazoncognito.com/oauth2/token?grant_type=client_credentials" \
-H 'Content-Type: application/x-www-form-urlencoded' \
--user <client_id>:<client_secret> | jq -r '.access_token'
This will return a JWT Access Token, which you can use to authorize with the AppSync API. For example, you can call the whoami endpoint:
PLAYGROUND_ACCESS_KEY=`curl --silent -X POST \
"https://<your-prefix>.auth.<your-region>.amazoncognito.com/oauth2/token?grant_type=client_credentials" \
-H 'Content-Type: application/x-www-form-urlencoded' \
--user <client_id>:<client_secret> | jq -r '.access_token'
curl --silent -X POST \
https://<your-appsync-prefix>.appsync-api.eu-west-1.amazonaws.com/graphql \
-H "Authorization: $PLAYGROUND_ACCESS_TOKEN" \
-d '{"query": "{ whoami { sub username issuer claims { at_hash scopes token_use auth_time iss exp iat version jti client_id } sourceIp defaultAuthStrategy }}"}' | jq
This will tell you who you are, and which scopes have been assigned to you:
{
"data": {
"whoami": {
"sub": "2t0n1jrmqmp9fa2n8lb4g4n7bh",
"username": null,
"issuer": "https://cognito-idp.eu-west-1.amazonaws.com/eu-west-1_y9UV6TY1K",
"claims": {
"at_hash": null,
"scopes": [
"scopes/items:read"
],
"token_use": "access",
"auth_time": "1616447660",
"iss": "https://cognito-idp.eu-west-1.amazonaws.com/eu-west-1_y9UV6TY1K",
"exp": "1616451260",
"iat": "1616447660",
"version": "2",
"jti": "573692a1-ff5a-4e48-a8a2-f80bac4ca3a8",
"client_id": "2t0n1jrmqmp9fa2n8lb4g4n7bh"
},
"defaultAuthStrategy": "ALLOW"
}
}
}
Retrieve an access token with the other M2M client, and the result will look like this:
{
"data": {
"whoami": {
"sub": "992bhrfnp4mn350pvs5rcnikh",
"username": null,
"issuer": "https://cognito-idp.eu-west-1.amazonaws.com/eu-west-1_y9UV6TY1K",
"claims": {
"at_hash": null,
"scopes": [
"scopes/items:write"
],
"token_use": "access",
"auth_time": "1616449133",
"iss": "https://cognito-idp.eu-west-1.amazonaws.com/eu-west-1_y9UV6TY1K",
"exp": "1616452733",
"iat": "1616449133",
"version": "2",
"jti": "b4cd80ec-aaf3-4111-b753-390317bfc8eb",
"client_id": "992bhrfnp4mn350pvs5rcnikh"
},
"defaultAuthStrategy": "ALLOW"
}
}
}
About that Whoami call: VTL Magic
You might wonder how this Whoami call has been implemented. After all, I said there were only four Lambda functions, and Whoami does not seem to be one of them. So please allow me to introduce you to the wonderful world of the Apache Velocity Template Language, or VTL templates.
Every request received by AppSync can be parsed by a Request Template before it is sent off to its target destination. Likewise, any response from a backend database or other data source will be parsed by a Response Template before it is sent back to the user. The cool thing is that these templates can also generate a response without communicating with a backend at all - and that’s what’s happening here.
The whoami call has been implemented in CDK as a NoneDataSource:
# A NoneDataSource has no backing data. Processing is purely done in the templates.
who_am_i_data_source = appsync.NoneDataSource(
scope=self,
id='playground_who_am_i_datasource',
api=params['graphql_api']
)
# The resolver binds the data source to the whoami query in the GraphQL schema
who_am_i_data_source.create_resolver(
type_name='Query',
field_name='whoami',
request_mapping_template=appsync.MappingTemplate.from_file(
file_name=f'{request_templates_path}/whoami.vtl'
),
response_mapping_template=appsync.MappingTemplate.from_file(
file_name=f'{response_templates_path}/whoami.vtl'
),
)
It only has a request and response template, and it is the response template that’s interesting:
#set($sourceIps = [])
#foreach ($ip in $context.identity.sourceIp)
$util.qr($sourceIps.add($ip.trim()))
#end
#set($result = {
"sub": $context.identity.sub,
"username": $context.identity.username,
"issuer": $context.identity.issuer,
"sourceIp": $sourceIps,
"claims": {
"at_hash": $context.identity.claims.get("at_hash"),
"scopes": $context.identity.claims.get("scope").split(" "),
"token_use": $context.identity.claims.get("token_use"),
"auth_time": $context.identity.claims.get("auth_time"),
"iss": $context.identity.claims.get("iss"),
"exp": $context.identity.claims.get("exp"),
"iat": $context.identity.claims.get("iat"),
"version": $context.identity.claims.get("version"),
"jti": $context.identity.claims.get("jti"),
"client_id": $context.identity.claims.get("client_id")
},
"defaultAuthStrategy": $context.identity.defaultAuthStrategy
})
$util.toJson($result)
This template does nothing but parse the $context.identity value, which is available to any AppSync operation by default, and return it in a valid AppSync format. No Lambda, no Python, just VTL.
Back to the user pool clients
Now that we’ve seen how VTL templates work, and we have seen that the user pool client’s scopes are available to us in VTL, we can use this information to determine whether a client has access to a specific resource, e.g. GetBook or AddBook. To define which scopes are required for these resources, we use VTL Request Templates to determine if a user (or machine) is authorized to execute a Lambda function:
#set($requiredScopes = ['{required_scopes}'])
#set($userScopes = $context.identity.claims.get("scope").split(" "))
#foreach ($requiredScope in $requiredScopes)
#if(!$userScopes.contains($requiredScope))
$utils.error("Scope '$requiredScope' is required")
#end
#end
{
"version" : "2017-02-28",
"operation": "Invoke",
"payload": {
"arguments": $util.toJson($context.args),
"selectionSetList": $utils.toJson($context.info.selectionSetList)
}
}
This VTL template parses the request before sending it off to Lambda. And as you can tell, it will first compare the user’s scopes with the required scopes, and return an error when the required scopes are not present.
Let’s try it out on the AddCar and GetCars operations. Retrieve an access token for the write-only M2M client, and execute the AddCar call:
PLAYGROUND_ACCESS_KEY=`curl --silent -X POST \
"https://<your-prefix>.auth.<your-region>.amazoncognito.com/oauth2/token?grant_type=client_credentials" \
-H 'Content-Type: application/x-www-form-urlencoded' \
--user <writer_client_id>:<writer_client_secret> | jq -r '.access_token'
curl --silent -X POST https://<your-appsync-prefix>.appsync-api.eu-west-1.amazonaws.com/graphql -H "Authorization: $PLAYGROUND_ACCESS_TOKEN" -d '{
"query": "mutation($car:AddCarInput!) { addCar(car:$car) { car { id make model color continentOfOrigin countryOfOrigin color } } }",
"variables": {
"car": {
"make": "Tesla",
"model": "Model 3",
"color": "white",
"continentOfOrigin": "EUROPE"
}
}
}' | jq
This should return a valid response:
{
"data": {
"addCar": {
"car": {
"id": "7ba706b4-ac16-49c1-ba33-7ceb34f2025c",
"make": "Tesla",
"model": "Model 3",
"color": "white",
"continentOfOrigin": "EUROPE",
"countryOfOrigin": null
}
}
}
}
Now run the GetCars call, and you should get a failure:
➜ ~ curl --silent -X POST https://zefvgq774rhlxebz2wsn3trhzy.appsync-api.eu-west-1.amazonaws.com/graphql -H "Authorization: $PLAYGROUND_ACCESS_TOKEN" -d '{
"query": "{ getCars { items { id make model color continentOfOrigin countryOfOrigin color } } } "}' | jq
{
"data": null,
"errors": [
{
"path": [
"getCars"
],
"data": null,
"errorType": null,
"errorInfo": null,
"locations": [
{
"line": 1,
"column": 3,
"sourceName": null
}
],
"message": "Scope 'scopes/items:read' is required"
}
]
}
Now try the same operations with the read-only access token, and you will get the exact opposite results.
Conclusion
Here is what we have built in a diagram:
By reading the scopes from the identity context in the Request VTL Template, AppSync can detect whether a user is authorized to access a resource very early in the processing flow. This will allow it to return an error before the Lambda function is even executed, which saves cost and latency.
The example implementation in this post is deliberately simple. It only has items:read and items:write scopes. However, the concept can easily scale to dozens of scopes, and the clients do not have to be Cognito Clients. The definition of ‘scopes’ is a standardized OAuth component, and is found and used in many applications. For example GitHub (Scopes for OAuth Apps) and Slack (Scopes and permissions) make extensive use of OAuth scopes. I hope this blog post has shown how VTL Templates can be leveraged to easily implement the same mechanism in AppSync.
I share posts like these and smaller news articles on Twitter, follow me there for regular updates! If you have questions or remarks, or would just like to get in touch, you can also find me on LinkedIn.
