Forem: Kong

Installing Kong's API Gateway with Docker

Viktor Gamov — Fri, 10 Sep 2021 19:57:32 +0000

Kong Gateway is an API layer for managing APIs and microservices. The easiest way to get started running Kong Gateway is to use Docker.

In this video, I'll explain how to run Kong Gateway using Docker and Docker Compose.

8 Common Request Transformation Policies

Michael Heap — Tue, 27 Jul 2021 13:00:17 +0000

API gateway request transformation policies are incredibly powerful. There are many situations when an API developer can take advantage of request transformations to adjust the shape and values of a request to cleanly fit their API.

Let’s say you’re deprecating a certain endpoint for your API, but you still need to support the old specification for a transition period. Request transformations let you accept requests according to the old specification, transform them to fit the new specification and then forward them.

Or maybe you discovered that incorrect documentation on an endpoint’s query parameter specification has been in the wild for far too long, which is why so many users have been encountering errors. Rather than changing your API to fit bad documentation, you can use request transformations to modify requests as they come in, shaping them according to how your API is supposed to work.

Or perhaps it’s simply time to implement basic security best practices, like stripping or obfuscating sensitive data in the headers before letting them continue upstream.

For these and many other reasons, request transformations can be your go-to solution.

This article will look at how we might use Kong Gateway, coupled with the Request Transformer plugin, to configure, intercept and transform requests as they make their way to an upstream route. Setup and configuration are incredibly simple. We’ll cover several common use cases:

Copying an API Key from the query string to the header
Removing a query string value
Moving an API key from a query string to the header
Adding a version number to a query string
Modifying the header token to Bearer Auth
Moving JWT from header to body
Sanitizing the body
Changing the HTTP method

This should give you enough foundation to craft your own transformations, custom-tailored for your application situation.

Overview of Core Concepts

Before we dive in, let’s quickly cover the technologies we’ll be using for our walkthrough.

Kong Gateway

Kong Gateway is a thin-layer, front-line “gateway” sitting in front of your system’s upstream services. Whether those upstream services are API servers, web servers or any other cloud microservices, Kong Gateway handles traffic control, authentication, load balancing and more.

Request Transformer Plugin

The Request Transformer plugin for Kong Gateway comes built in. As requests come through Kong, you can configure the plugin to transform those requests—mutating headers, query string parameters, the request body and so on—before forwarding those requests to their final destination. Transformations are highly configurable and extremely powerful, giving you a lot of control over the precise shape of requests before they hit your endpoint.

Mockbin

Mockbin is an online custom endpoint generator used for testing HTTP requests and tracking responses. Mockbin is exactly what we need for our walkthrough—as we set up transformations, send requests and then inspect the result.

Our Walkthrough Approach

To start, we’ll set up our Mockbin endpoint as our “upstream service.” We only need a single endpoint, and we’ll hit it with GET and POST requests. Then, we’ll make sure Kong Gateway is installed and configured properly.

We’ll look at configuring the Request Transformer plugin for each transformation that we want to demonstrate. Then, we’ll send our request (using curl). Finally, we’ll inspect the request at Mockbin. We want to see what the request looked like when it arrived at Mockbin after passing through Kong and the Request Transformer plugin.

Are you ready? Roll out!

Want to set up a request transformation for your API gateway with clicks instead of code? Try Konnect for free >>

Set Up Mockbin Endpoint

Mockbin is a simple and easy-to-use test endpoint generator. On the website, click on the Create Bin link to get started.

And, just like that, your endpoint has been created! From here, you will need your endpoint URL. This is just https://mockbin.org/bin/{BIN-IDENTIFIER}. But, for good measure, you can right-click on Visit in Browser and copy the link address.

To test it out, you can open a new tab in your browser and visit the link you just copied. Doing so will send a GET request to your Mockbin endpoint. To see the details for this request, go back to the browser tab with your Mockbin details and click on View History. You’ll see a running log of requests to this endpoint. You should see the GET request that you just sent:

Our Mockbin endpoint is up and running and ready to go.

Set Up Kong Gateway

Spinning up Kong Gateway is quite straightforward. First, you’ll need to install Kong on your local machine. There are many different installation options, so you can choose whichever best fits your environment.

After installing Kong, create a project folder on your local machine. For simplicity, I’m creating a sub-folder called “project” in my home folder. Then, in that folder, run the command to generate a declarative configuration file.

~$ mkdir project
~$ cd project
~/project$ kong config init
~/project$ tree
.
└── kong.yml

0 directories, 1 file

For our use of the Request Transformer plugin, we can put all of our service and route setup and plugin configurations in a single file. Then, start Kong. Kong does not need to write to a database, and we don’t need to do any further on-the-fly configuration as we go along. This is Kong’s DB-less and Declarative Configuration, and it’s sufficient for what we need.

The kong.yml file is a starter template for a declarative configuration file. Let’s open it and modify it to look like the following:

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: untouched
  service: mockbin
  paths:
    - /untouched

Let’s briefly go over what we’ve done here. The _format_version line, which is required, specifies the version number of the declarative configuration syntax we’re using. Next, we declare an upstream service, and we give it the arbitrary name “mockbin” and specify the URL for this upstream service. Make sure you use your unique URL for the Mockbin endpoint you created earlier.

Next, we create a route, which we will arbitrarily name “untouched.” This route listens on the Kong Gateway for requests that go to the path /untouched, and then it forwards those requests to our upstream service called “mockbin.” In this first example, we are not adding a Request Transformer plugin. Requests will proceed through Kong Gateway untouched, continuing to Mockbin.

We’re almost ready! The last thing we need to do is tell Kong where to look for our declarative configuration file when it starts up. To do this, we need a kong.conf file, copied from the default template provided to us upon installation:

~/project$ cd /etc/kong
/etc/kong$ cp kong.conf.default kong.conf
~/project$ sudo vi kong.conf

In the kong.conf file, there are two lines that we need to edit. At around line 922, we need to tell Kong that we won’t be using a database. And at around line 1106, we need to provide the absolute path to our declarative configuration file. That’s the kong.conf file in your project folder.

# PATH: /etc/kong/kong.conf

...

# AROUND LINE 922
# inherited from the corresponding main connection config described above but
# may be optionally overwritten explicitly using the `pg_ro_*` config below.

database = off # Determines which of PostgreSQL or Cassandra
                              # this node will use as its datastore.

# AROUND LINE 1106
                              # This value is only used during
                              # migrations.

declarative_config = /PATH/TO/YOUR/PROJECT/FOLDER/kong.yml
                              # The path to the declarative configuration
                              # file which holds the specification of all
                              # entities (Routes, Services, Consumers, etc.)

With the file saved, we’re ready to start up Kong.

~/project$ sudo kong start

Now that Kong is running, let’s send a request to localhost:8000, which is the port where Kong is listening. We’ll send a GET request to the /untouched endpoint, and we’ll tack on a query string parameter:

~/project$ curl -X GET 'http://localhost:8000/untouched?hello=world'

In your browser, refresh the page with your Mockbin endpoint history log. You should see a new entry for a GET request.

As you view the Request Details, you can see that the queryString array has a pair that matches the parameters we sent to Kong. It looks like Kong successfully forwarded our (untouched) request to Mockbin!

Just for good measure, let’s also send a POST request to the same endpoint at Kong. We’ll set our “Content-Type” header and tack on a request body like so:

~/project$ curl -X POST \
           -H 'Content-Type: application/json' \
           -d '{"hello":"world"}' \
           'http://localhost:8000/untouched'

Again, we refresh the Mockbin log, and we see our POST request. Here, we see our Request Body, which matches what we sent to Kong.

If we poke around at the Request Details, we also see the “Content-Type” value that we set in our header, along with our POST body data.

It looks like we have finished our setup. We’ve connected all the pieces. It’s time to start playing around with some common request transformations.

Common Request Transformations

1. Copy API key from query string to header

Let’s start with a common scenario. Your API users have been adding their API key as a query string parameter, but you’d like to transition them toward attaching it as a custom header value instead. Until you’re certain that all of your API users have gotten the memo that the key should be in the header, you want to accommodate the stragglers. What you need is a request transformation that copies the value from the query string parameter to a new header value.

Here is how our kong.yml file would look:

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: copy-query-to-header
  service: mockbin
  paths:
    - /copy-q2h
plugins:
- name: request-transformer
  route: copy-query-to-header
  config:
    add:
      headers:
        - api-key:$(query_params.api_key)

Similar to our first example, we’ve created a service, along with a route that forwards to this service. For this route, Kong will listen on the path /copy-q2h. Next, we add a plugin. The name of this plugin ( not arbitrary) is request-transformer. The plugin is for our specific route.

What kinds of request transformations should this plugin do? It will add a header. The new key for this header will be api-key. The value will be copied from the query string parameter called api_key. This part uses the plugin’s templating feature to grab a value that came in the query string

Don’t forget: Whenever we modify the kong.yml file, we need to restart Kong. Let’s restart Kong and send our request.

~/project$ sudo kong restart
~/project$ curl -X GET 'http://localhost:8000/copy-q2h?api_key=THISISMYAPIKEY'

We’ve sent a GET request to our /copy-q2h endpoint. We tacked on a query string parameter api_key=THISISMYAPIKEY. Let’s see how this request was transformed by refreshing the Mockbin log.

As we scroll down the Request Details, we see in the headers that there is an api-key header with the value THISISMYAPIKEY. It looks like our copy transformation worked.

You will notice that the api_key=THISISMYAPIKEY query string parameter is still in the request, though. That’s okay for now. We’ll deal with that in a little bit.

2. Remove query string value

Another common scenario involves the need to remove certain query string parameters completely. Perhaps for privacy reasons or security reasons, sanitizing query parameters is a common use case for request transformations.

In the following kong.yml example, we want to remove the key-value pairs for the api_key and password query string parameters. Similar to our previous example, we create a route (listening on /strip-q), and we create a plugin on that route that removes our undesirable parameters:

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: strip-query
  service: mockbin
  paths:
    - /strip-q
plugins:
- name: request-transformer
  route: strip-query
  config:
    remove:
      querystring:
        - api_key
        - password

Let’s send a request to this path, adding on a few query parameters. We’ll include the ones we want to strip away, along with one that we would like to keep. Don’t forget to restart Kong!

~/project$ sudo kong restart
~/project$ curl -X GET \
'http://localhost:8000/strip-q?api_key=THISISMYAPIKEY&username=johndoe&password=THISISMYPASSWORD'

Taking a glance at the Mockbin log, we can see that this most recent GET request received the query string parameter we wanted to keep (username=johndoe) but didn’t receive the undesirable ones. That’s a good sign.

A closer look at the query string array in Request Details shows that the plugin worked as expected, removing the api_key and password parameters before forwarding the request to Mockbin.

Just a note: You could write kong.yml with multiple routes and multiple plugins. In this article, our file always shows just a single route and a single plugin. That’s just to keep it simple, but you’re not restricted to do it this way.

3. Move API key from query string to header

Previously, we copied an API key from the query parameter to a header value, but we left the query parameter in the request. In the following example, let’s do something similar, but we’ll clean up after ourselves by removing the query parameter too. Our configuration for the plugin this time around: 1) adds a header based on the query parameter value, and then 2) removes the query parameter.

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: move-query-to-header
  service: mockbin
  paths:
    - /move-q2h
plugins:
- name: request-transformer
  route: move-query-to-header
  config:
    add:
      headers:
        - api-key:$(query_params.api_key)
    remove:
      querystring:
        - api_key

At this point, the keen observer might be asking: “Wait, if you remove the query parameter, how would you copy its value over to the header? Does the configuration order matter, where you have to copy it before you remove it?” The answer is… no! Kong’s documentation on template values explains it this way:

Note: The plugin creates a non-mutable table of request headers, query strings, and captured URIs before the transformation. Therefore, any update or removal of params used in a template does not affect the rendered value of a template.

Let’s restart Kong and send our request.

~/project$ sudo kong restart
~/project$ curl -X GET 'http://localhost:8000/move-q2h?api_key=THISISMYAPIKEY'

An inspection of the Request Details at the Mockbin log shows the value in our headers:

4. Add version number to query string

Another common case is simply adding a query string parameter to a request. In this example, we’ll add a fixed API version number to all requests that come through our /add-q route:

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: add-to-query
  service: mockbin
  paths:
    - /add-q
plugins:
- name: request-transformer
  route: add-to-query
  config:
    add:
      querystring:
        - api_version:2.0

We will add the parameter called api_version, and its value will be 2.0. Let’s restart Kong and send our request.

~/project$ sudo kong restart
~/project$ curl -X GET 'http://localhost:8000/add-q?username=johndoe'

While we only set one query parameter (username=johndoe), we inspect our transformed request at the Mockbin log. There we see in the Request Details that Mockbin received two parameters with the request:

It looks like our query parameter add was successful.

5. Modify header token to Bearer Auth

Now, let’s experiment with our headers a bit. Imagine the scenario where your API users are attaching their authorization token as a header called token, but you wrote your API to use the Bearer Auth scheme. The token value should be in a header called Authorization, following the word “Bearer!” Collin, the junior dev who wrote that erroneous API documentation, will get a poor performance review. In the meantime, you’ll use request transformation to fix Collin’s mess.

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: modify-header
  service: mockbin
  paths:
    - /token-to-auth
plugins:
- name: request-transformer
  route: modify-header
  config:
    add:
      headers:
        - Authorization:Bearer $(headers["token"])
    remove:
      headers:
        - token

Our configuration again takes advantage of template values, copying the token value from the token header and structuring a proper Bearer Auth header value. And, of course, we remove the undesirable token header after we get what we need.

We restart Kong and send our request.

~/project$ sudo kong restart
~/project$ curl -X GET \
-H 'token:thisisanopaquestringthatrepresentsajwt' \
'http://localhost:8000/token-to-auth'

For this request, we sent our token value in the header called
token. Let’s inspect the Request Details at our Mockbin log:

We see the Bearer Auth scheme in our header now. A further inspection of the details shows that the offending token header is not present. On a side note, you might notice above that we had configured our plugin to add a header with the name “Authorization” (capitalized), but what shows up in Mockbin is “authorized” (lowercase). It looks like this might be an issue with Mockbin—which lowercases all of its header names—and not an issue with the Request Transformer plugin (which addressed this specific issue in a pull request).

6. Move JWT from header (Bearer Auth) to body

Let’s do some request transformations that modify the request body. In this example, we’ll take the token in Bearer Auth format, and we’ll write it to our request JSON body as a value. While some upstream services look in the header for an authorization token, others might look to the request body. Request transformations provide flexibility, especially when modifying upstream services isn’t an option.

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: jwt-to-body
  service: mockbin
  paths:
    - /jwt2b
plugins:
- name: request-transformer
  route: jwt-to-body
  config:
    add:
      body:
        - jwt:$(headers["Authorization"]:gsub("^Bearer ",""))

The syntax for the above plugin configuration is a bit more complicated, but you can probably discern what’s happening. We want to add a key-value pair to our request body. The key is jwt, and the value will be taken with the Authorization header but with the initial “Bearer” (plus space) removed.

The plugin documentation notes that the placeholder is evaluated as Lua expression. So, if you know Lua, you can probably do some pretty powerful transformations.

Let’s restart Kong and send our POST request, along with headers and body data:

~/project$ sudo kong restart
~/project$ curl -X POST \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer thisisanopaquestringthatrepresentsajwt' \
-d '{"username":"johndoe" }' \
'http://localhost:8000/jwt2b'

When we look at our Mockbin log for this request, we immediately see the Request Body. It contains the original body we sent ("username":"johndoe"), but it also contains
jwt, with the token value that was in our Bearer Auth header.

You might notice, from our plugin configuration, that we decided to leave our Bearer Auth header in the request:

7. Sanitize body

Perhaps for sensitive data that might come through the request body, you will need a request transformation that sanitizes it but doesn’t remove it. This helps your upstream service know that the data did come through, but it’s just no longer available. For this type of transformation, your configuration might look like this:

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: sanitize-body
  service: mockbin
  paths:
    - /san-b
plugins:
- name: request-transformer
  route: sanitize-body
  config:
    replace:
      body:
      - last4: ****
      - api_key: ********
      - password: ********

In this request transformation, we’re looking in our JSON body for last4, api_key or password. For any of those keys, we want to replace the provided value with a redacted one. After restarting Kong, we send our request:

~/project$ sudo kong restart
~/project$ curl -X POST \
-H 'Content-Type: application/json' \
-d '{"api_key":"THISISMYAPIKEY", "username":"johndoe", "password":"THISISMYPASSWORD", "last4":9876}' \
'http://localhost:8000/san-b'

A quick glance at the Mockbin log shows our transformed Request Body. Recall that Kong transformed this body before it was sent to the upstream service, so we can be sure that none of this sensitive data got past our gateway.

8. Change HTTP method

For our final example, let’s demonstrate a request method transformation. Let’s assume that PUT requests to a given route need to be converted to POST requests, and instead, we’d add an "action":"PUT" key-value pair into the JSON request body.

In this configuration, we configure our route only to listen for PUT requests. Then, we configure the plugin to transform the http_method to POST. Lastly, we add a key-value pair to our body.

# ~/project/kong.yml
_format_version: "2.1"

services:
- name: mockbin
  url: https://mockbin.org/bin/REPLACE-WITH-YOUR-BIN-IDENTIFIER
routes:
- name: put-to-post
  service: mockbin
  methods:
    - PUT
  paths:
    - /put-to-post
plugins:
- name: request-transformer
  route: put-to-post
  config:
    http_method: POST
    add:
      body:
      - action:PUT

Restart Kong and send the PUT request.

~/project$ sudo kong restart
~/project$ curl -X PUT \
-H 'Content-Type: application/json' \
-d '{"username":"johndoe" }' \
'http://localhost:8000/put-to-post'

We see the successful result in our Mockbin log. The most recent request was a POST (not PUT) request. And, we see the action key-value pair added to our Request Body.

Conclusion

We’ve covered a lot of examples in this walkthrough. We’ve transformed query string parameters, headers, bodies and even request methods. The Request Transformer plugin offers several ways to transform parts of the request. While we’ve covered many of them, combining multiple transformations can yield powerful results that uniquely shape a request to fit your API.

We didn’t cover URI transformations, which allow you to transform the upstream request URI based on the incoming request. You can imagine taking a long REST-compliant URI, chock-full of a chain of resources, ids, sub-resources and more ids. Using URI capturing and template values, the Request Transformer plugin can capture all of that data in the URI and write it into the request body instead, sending the request upstream to a general all-purpose endpoint.

Ready for Advanced Request Transformations?

The Kong Konnect enterprise-level Request Transformer Advanced plugin provides even more targeted transformations with regular expression matching, variables and substitutions.

All in all, the ability to transform your requests at the gateway level before they hit your upstream service is critical. Whether it’s because of deprecated specs, documentation-reality misalignment or data privacy and security concerns, having the flexibility to shape incoming requests—and to do so quickly and simply—might be exactly what your DevOps team needs to save the day in a pinch.

Once you’ve successfully set up API gateway request transformation policies, you may find these other tutorials helpful:

The post 8 Common API Gateway Request Transformation Policies appeared first on KongHQ.

20 Lines of JavaScript to Create a Kong Gateway Plugin

Michael Heap — Wed, 26 May 2021 13:00:48 +0000

We recently sat down to discuss the language for the next Kong Gateway Plugin Development Kit (PDK). Given the number of JavaScript developers in the world and the variety of libraries and debugging tools available, there was only one logical choice. I’m excited to share that with the Kong Gateway (OSS) 2.4 release, that functionality is now available to you all!

To show the power of the new JavaScript PDK, we’re going to implement a plugin that adds X-Clacks-Overhead, a non-standardized HTTP header based on the work of Terry Pratchett to all responses.

Bootstrapping Your Development Environment

The JavaScript plugin support in Kong Gateway works by running a Node.js server on the same machine as Kong Gateway and passing messages back and forth using msgpack.

This means that we need a development environment that can run both the Kong Gateway and a Node.js process. You can configure this on your local machine, but to make things easier, I’ve put together a docker-based environment for you to use.

It might take a minute or two to download the images and build our Node.js environment. I recommend running it now in the background as you keep reading:

$ git clone https://github.com/Kong/docker-kong-js-pdk
$ cd kong-js-pdk-dev
$ docker-compose build

Creating Your First Plugin

The configuration provided in the environment we created reads all plugins from the plugins directory. It’s currently empty as we have not created our first plugin yet.

The JavaScript PDK uses the name of the JS file as the name of the plugin. Let’s go ahead and create a file called clacks.js in the plugins directory with the following contents:

class ClacksPlugin {
  async access(kong) {
    await kong.response.setHeader(`X-Clacks-Overhead`, "GNU Terry Pratchett");
  }
}

module.exports = {
  Plugin: ClacksPlugin,
  Version: "0.1.0"
};

The kong object passed into the access method is an instance of the JavaScript PDK provided by the plugin server. This means that we do not need to require kong-pdk in our plugins, as it is automatically made available.

There are five phases available for HTTP requests in the life-cycle of a Kong Gateway request:

-certificate – Executed once per request when the connection is SSL/TLS enabled
-rewrite – Performed before the API gateway does any routing
-access – All routing is done, and the plugin knows which service the request is bound to. This is the last phase before the API gateway makes a request to upstream
-response – Allows you to manipulate the response from the upstream. Implementing this phase has a performance penalty as it enables request buffering
-log – Executed after the request has been completed

Enable the Plugin

The environment we’re running uses Kong’s declarative config capability. That means that we need to update the config file to enable our new plugin. Open up config/kong.yml, and you should see a service defined that proxies to mockbin.org:

services:
  - name: example-service
    url: https://mockbin.org

As our file name was clacks.js, our plugin will be called clacks. Let’s enable the plugin in the definition now:

services:
  - name: example-service
    url: https://mockbin.org
    plugins:
      - name: clacks

Kong Gateway only allows you to use plugins that are on an allowlist for security purposes, so we’ll also need to add clacks to that list. Open up docker-compose.yml and edit the value of KONG_PLUGINS so that it looks like the following:

KONG_PLUGINS: bundled,clacks

Making a Request

At this point the API gateway is ready to run our new plugin, so let’s go ahead and start it:

$ docker-compose up

The docker-compose.yml file forwards the API gateway port to our local machine. That means we can make requests to localhost:8000 to test our service.

$ curl -I localhost:8000

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Connection: keep-alive
X-Clacks-Overhead: GNU Terry Pratchett
...snip...

I can see the X-Clacks-Overhead header in the response, which means that our plugin works as intended!

Making It Configurable

The custom JavaScript plugin we built today is a simple plugin that does one thing and does it well. I want to take a moment to show you how you can make that behavior customizable using plugin configuration too.

There is an ongoing discussion based on RFC 6648 about if custom headers need an X- prefix. Let’s make our plugin configurable so that people can decide if they want to use the X- prefix.

Plugin configuration is controlled using the Schema property in module.exports at the end of clacks.js. Let’s add an entry to define a use_prefix option that’s a boolean with a default value of true:

module.exports = {
  Plugin: ClacksPlugin,
  Schema: [{ use_prefix: { type: "boolean", default: true } }],
  Version: "0.1.0"
};

Any configuration provided to the plugin is passed in using the constructor. Let’s go ahead and capture that in clacks.js so that we can use it in our access method and update access so that it only adds the X- prefix if use_prefix is true:

class ClacksPlugin {
  constructor(config) {
    this.config = config;
  }

  async access(kong) {
    const prefix = this.config.use_prefix ? "X-" : "";
    await kong.response.setHeader(
    `${prefix}Clacks-Overhead`,
    "GNU Terry Pratchett"
    );
  }
}

If we run our plugin now, it will behave the same way as it did with a hardcoded X- prefix. Let’s update our API gateway config in config/kong.yml to set use_prefix to false.

services:
  - name: example-service
    url: https://mockbin.org
    plugins:
      - name: clacks
        config:
          use_prefix: false

If we restart our API gateway by pressing Ctrl+C then running docker-compose up again, we should now be able to make a request to localhost:8000 and see Clacks-Overhead header without the X- prefix:

$ curl -I localhost:8000

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Clacks-Overhead: GNU Terry Pratchett
...snip...

Conclusion

Just 20 lines of Javascript, and we have a working Kong Gateway plugin, complete with configuration options!

What we’ve built together is a trivial plugin, but using the environment provided and what you’ve learned about Kong’s configuration, you can go ahead and build plugins to your heart’s content.

If you’re looking for more plugin examples, take a look at some demo plugins:

If you have any questions, post them on Kong Nation .

To stay in touch, join the Kong Community.

Once you’ve successfully set up a custom Kong plugin with JavaScript, you may find these other tutorials helpful:

The post Building a Kong Gateway Plugin with JavaScript appeared first on KongHQ.

Protecting Services With API Gateway Rate Limiting

Julia Salem — Tue, 18 May 2021 13:00:19 +0000

The Kong Gateway Rate Limiting plugin is one of our most popular traffic control add-ons. You can configure the plugin with a policy for what constitutes “similar requests” (requests coming from the same IP address, for example), and you can set your limits (limit to 10 requests per minute, for example). This tutorial will walk through how simple it is to enable rate limiting in your Kong Gateway.

Rate Limiting: Protecting Your Server 101

Let’s take a step back and go over the concept of rate limiting for those who aren’t familiar.

Rate limiting is remarkably effective and ridiculously simple. It’s also regularly forgotten. Rate limiting is a defensive measure you can use to prevent your server or application from being paralyzed. By restricting the number of similar requests that can hit your server within a window of time, you ensure your server won’t be overwhelmed and debilitated.

You’re not only guarding against malicious requests. Yes, you want to shut down a bot that’s trying to discover login credentials with a brute force attack. You want to stop scrapers from slurping up your content. You want to safeguard your server from DDOS attacks.

But it’s also vital to limit non-malicious requests. Sometimes, it’s somebody else’s buggy code that hits your API endpoint 10 times a second rather than one time every 10 minutes. Or, perhaps only your premium users get unlimited API requests, while your free-tier users only get a hundred requests an hour.

Mini-Project for Kong Gateway Rate Limiting

In our mini-project for this article, we’re going to walk through a basic use case: Kong Gateway with the Rate Limiting plugin protecting a simple API server. Here are our steps:

Create Node.js Express API server with a single “hello world” endpoint.
Install and set up Kong Gateway.
Configure Kong Gateway to sit in front of our API server.
Add and configure the Rate Limiting plugin.
Test our rate limiting policies.

After walking through these steps together, you’ll have what you need to tailor the Rate Limiting plugin for your unique business needs.

Want to set up rate limiting for your API gateway with clicks instead of code? Try Konnect for free >>

Create a Node.js Express API Server

To get started, we’ll create a simple API server with a single endpoint that listens for a GET request and responds with “hello world.” At the command line, create a project folder and initialize a Node.js application:

~/$ mkdir project
~/$ cd project
~/project$ yarn init
// Accept all defaults

Then, we’ll add Express, which is the only package we’ll need:

~/project$ yarn add express

Lastly, let’s create a simple server with our “hello world” endpoint. In your project folder, create an index.js file with these contents:

/* PATH: ~/project/index.js
*/
const express = require('express')
const server = express()
const port = 3000
server.get('/', (req, res) => {
 console.log(req.headers)
 res.status(200).send('Hello world!')
})
server.listen(port, () => {
 console.log(`Server is listening on http://localhost:${port}`)
})

Now, spin up your server:

~/project$ node index.js

In your browser, you can visit http://localhost:3000. Here’s what you should see:

Let’s also use Insomnia, a desktop client for API testing. In Insomnia, we send a GET request to http://localhost:3000.

Our Node.js API server with its single endpoint is up and running. We have our 200 OK response.

Keep that terminal window with the node running. We’ll do the rest of our work in a separate terminal window.

Install and Set Up Kong Gateway

Next, we’ll install Kong Gateway to sit in front of our API server. These steps will vary depending on your local environment.

Simple usage of the Rate Limiting plugin supports configuring Kong in DB-less mode with a declarative configuration. That means, instead of sending configurations to Kong Gateway one step at a time, with those configurations stored in a database, we can use a single declarative .yml file for specifying our entire Kong configuration.

After installing Kong, we generate a starter .yml file:

~/project$ kong config init
~/project$ tree -L 1
.
├── index.js
├── kong.yml
├── node_modules
├── package.json
└── yarn.lock
1 directory, 4 files

We’ll come back to that kong.yml file in a moment.

Now, let’s tell Kong where to look for that declarative configuration file upon startup. In /etc/kong, there is a
kong.conf.default file that we’ll need to copy as kong.conf and then edit:

~/project$ cd /etc/kong
/etc/kong$ sudo su
root:/etc/kong$ cp kong.conf.default kong.conf

Next, we edit the kong.conf file. You’ll likely need root privileges to do this. There are two edits that we need to make:

# PATH: /etc/kong/kong.conf

# line ~839: Uncomment this line and set to off
database = off

# line ~1023, Uncomment this line. Set to absolute path to kong.yml
declarative_config = /PATH/TO/YOUR/project/kong.yml

Configure Kong Gateway With Our Service and Routes

Before we start up Kong, let’s edit the declarative configuration file generated in our project folder. It should look like this:

# PATH: ~/project/kong.yml

_format_version: "2.1"

services:
- name: my-api-server
  url: http://localhost:3000/
  routes:
  - name: api-routes
    paths:
    - /api

Let’s walk through what this configuration does. After setting the syntax version (2.1), we configure a new upstream service for which Kong will serve as an API gateway. Our service, which we name my-api-server, listens for requests at the URL http://localhost:3000.

We associate a route (arbitrarily named
api-routes) for our service with the path /api. Kong Gateway will listen for requests to /api, and then route those requests to our API server at http://localhost:3000.

With our declarative configuration file in place, we start Kong:

~/project$ sudo kong start

Now, in Insomnia, we send a GET request through Kong Gateway, which listens at http://localhost:8000, to the /api path:

Note that, by sending our request to port 8000, we are going through Kong Gateway to get to our API server.

Add the Kong Gateway Rate Limiting Plugin

Now that we have Kong Gateway sitting in front of our API server, we’ll add in the Rate Limiting Plugin and test it out. We will need to add a few lines to our kong.yml declarative configuration file:

# PATH: ~/project/kong.yml

_format_version: "2.1"

services:
- name: my-api-server
  url: http://localhost:3000/
  routes:
  - name: api-routes
    paths:
    - /api
  plugins:
  - name: rate-limiting
    config:
      minute: 5
      hour: 12
      policy: local

We’ve added the entire plugins section underneath our my-api-server service. We specify the name of the plugin, rate-limiting. This name is not arbitrary but refers to the actual rate-limiting plugin in the Kong package.

In this first run, we’ve configured the plugin with minute: 5, which allows for up to five requests per minute. We’ve also added hour : 12, which limits the requests per hour to 12. We’re using the local policy, which has to do with how Kong will store and increment request counts. We’ll talk about this policy configuration more below.

Let’s restart Kong:

~/project$ sudo kong restart

Now, back in Insomnia, we test out sending some requests to our API endpoint. If you send a request every few seconds, you’ll see that the first five requests received a 200 OK response. However, if you exceed five requests within a minute:

The Rate Limiting plugin detects that requests have exceeded the five-per-minute rule. It doesn’t let the subsequent request through to the API server. Instead, we get a 429 Too Many Requests response.

Our plugin works!

If you wait a minute and then try your requests again, you’ll see that you can get another five successful requests until Kong blocks you again with another 429.

At this point, we’ve sent 10 requests over several minutes. But you’ll recall that we configured our plugin with hour: 12. That means our 11th and 12th requests will be successful. However, the system will reject our 13th request even though we haven’t exceeded the five-per-minute rule . That’s because we’ll have exceeded the 12-per-hour rule.

The Rate Limiting plugin allows you to limit the number of requests per second, minute, hour, day, month and year.

Rate Limiting “Similar” Requests

We’ve configured Kong to count (and limit) requests to our server in our simple use case so far. More likely, we’ll want to apply our rate limits to similar requests. By “similar,” we might mean “coming from the same IP address” or “using the same API key.”

For example, let’s say that all users of our API server need to send requests with a unique API key set in headers as x-api-key. We can configure Kong to apply its rate limits on a per-API-key basis as follows:

# PATH: ~/project/kong.yml

_format_version: "2.1"

services:
- name: my-api-server
  url: http://localhost:3000/
  routes:
  - name: api-routes
    paths:
    - /api
  plugins:
  - name: rate-limiting
    config:
      minute: 5
      hour: 12
      policy: local
      limit_by: header
      header_name: x-api-key

The Rate Limiting plugin will apply its limits by grouping requests according to a value found in the x-api-key in the header. Requests with the same x-api-key will be considered “similar” requests.

Let’s restart Kong and see this in action:

~/project$ sudo kong restart

Back in Insomnia, we’ll adjust our GET request slightly by adding a new x-api-key header value. Let’s first set the value to user1. We send the request multiple times. Our first five requests return a 200 OK:

When we exceed the five-per-minute rule, however, this is what we see:

If we change the x-api-key to a different value (for example, user2), we immediately get 200 OK responses:

Requests from user2 are counted separately from requests from user1. Each unique x-api-key value gets (according to our rate limiting rules) up to five requests per minute and up to 12 requests per hour.

You can configure the plugin to limit_by IP address, a header value, request path or even credentials (like OAuth2 or JWT).

Counter Storage “Policy”

In our example above, we set the policy of the Rate Limiting plugin to local. This policy configuration controls how Kong will store request counters to apply its rate limits. The local policy stores in-memory counters. It’s the simplest strategy to implement (there’s nothing else you need to do!).

However, request counters with this strategy are only mostly accurate. If you want basic protection for your server, and it’s acceptable if you miscount a request here and there, then the local policy is sufficient.

The documentation for the Rate Limiting plugin instructs those with needs where “every transaction counts” to use the cluster (writes to the database) or redis (writes to Redis key-value store) policy instead.

Advanced Rate Limiting

For many common business cases, the open source Rate Limiting plugin has enough configuration options to meet your needs. For those with more complex needs (multiple limits or time windows, integration with Redis Sentinel), Kong also offers their Rate Limiting Advanced plugin for Kong Konnect.

Let’s briefly recap what we did in our mini-project. We spun up a simple API server. Then, we installed and configured Kong Gateway to sit in front of our API server. Next, we added the Rate Limiting plugin to count and limit requests to our server, showing how Kong blocks requests once we exceed certain limits within a window of time.

Lastly, we demonstrated how to group (and count) requests as “similar” with the example of a header value. Doing so enabled our plugin to differentiate counts for requests coming from two different users so that you can apply that rate limit to each user separately.

That’s all there is to it. Kong’s Rate Limiting plugin makes protecting your server ridiculously simple. As a basic traffic control defense measure, rate limiting can bring incredible power and peace of mind. You can find more details in the rate limiting plugin doc.

If you have any additional questions, post them on Kong Nation. To stay in touch, join the Kong Community.

Now that you’re successfully protecting services with the Kong Gateway Rate Limiting plugin, you may find these other tutorials helpful:

The post Protecting Services With Kong Gateway Rate Limiting appeared first on KongHQ.

Using Kubernetes Ingress Controller as an API Gateway

Viktor Gamov — Tue, 23 Mar 2021 14:18:22 +0000

In this first section, I’ll provide a quick overview of the business case and the tools you can use to create a Kubernetes ingress API gateway. If you’re already familiar, you could skip ahead to the tutorial section.

Digital transformation has led to a high velocity of data moving through APIs to applications and devices. Companies with legacy infrastructures are experiencing inconsistencies, failures and increased costs. And most importantly, dissatisfied customers.

All this has led to significant restructuring and modernization, especially within IT. A primary strategy is to embrace Kubernetes and decouple monolithic systems. On top of that, IT leadership is tasking DevOps teams to find systems, like an API gateway or Kubernetes ingress controller, to support API traffic growth while minimizing costs.

API gateways are crucial components of microservice architectures. The API gateway acts as a single entry point into a distributed system, providing a unified interface for clients who don’t need to care (or know) that the system aggregates their API call response from multiple microservices.

Some everyday use cases for API gateways include:

• Routing inbound requests to the appropriate microservice
• Presenting a unified interface to a distributed architecture by aggregating responses from multiple backend services
• Transforming microservice responses into the format required by the caller
• Implementing non-functional/policy concerns such as authentication, logging, monitoring and observability, rate-limiting, IP filtering, and attack mitigation
• Facilitating deployment strategies such as blue/green or canary releases

API gateways can simplify the development and maintenance of a microservice architecture. As a result, freeing up development teams to focus on the business logic of individual components.

Like Papa John’s, many companies select a Kubernetes API gateway at the beginning or partway through their transition to multi-cloud. Doing so makes it necessary to choose a solution that can function with on-prem services and the cloud.

Kubernetes

Kubernetes is becoming the hosting platform of choice for distributed architectures. It offers auto-scaling, fault tolerance and zero-downtime deployments out of the box.

By providing a widely accepted, standard approach with a carefully designed API, Kubernetes has spawned a thriving ecosystem of products and tools that make it much easier to deploy and maintain complex systems.

Kubernetes Ingress Controller

As a native Kubernetes application, Kong is installed and managed precisely as any other Kubernetes resource. It integrates well with other CNCF projects and automatically updates itself with zero downtime in response to cluster events like pod deployments. There’s also a great plugin ecosystem and native gRPC support.

This article will walk through how easy it is to set up the open source Kong Ingress Controller as a Kubernetes API gateway on a cluster.

Use Case: Routing API Calls to Backend Services

To keep this article to a manageable size, I will only cover a single, straightforward use case.

I will create a Kubernetes cluster, deploy two dummy microservices, “foo” and “bar,” install and configure Kong to route inbound calls to /foo to the foo microservice and send calls to /bar to the bar microservice.

The information in this post barely scratches the surface of what you can do with Kong, but it’s a good starting point.

Prerequisites

There are a few things you’ll need to work through in this article.

I’m going to create a “real” Kubernetes cluster on DigitalOcean because it’s quick and easy, and I like to keep things as close to real-world scenarios as possible. If you want to work locally, you can use minikube or KinD. You will need to fake a load-balancer, though, either using the minikube tunnel or setting up a port forward to the API gateway.

For DigitalOcean, you will need:

• A DigitalOcean account
• A DigitalOcean API token with read and write scopes
• The doctl command-line tool

To build and push docker images representing our microservices, you will need:

• Docker
• An account on Docker Hub

Note: Optional because you can deploy the images I’ve already created.

You will also need kubectl to access the Kubernetes cluster.

Setting Up doctl

After installing doctl, you’ll need to authenticate using the DigitalOcean API token:

2
3
4
$ doctl auth init
...
Enter your access token:  <-- paste your API token, when prompted
Validating token... OK

Create Kubernetes Cluster

Now that you have authenticated doctl, you can create your Kubernetes cluster with this command:

$ doctl kubernetes cluster create mycluster --size s-1vcpu-2gb --count 1

Note: The command spins up a Kubernetes cluster on DigitalOcean. Doing so will incur charges (approximately $0.01/hour, at the time of writing) as long as it is running. Please remember to destroy any resources you create when you finish with them.

The command creates a cluster with a single worker node of the smallest viable size in the New York data center. It’s the smallest and simplest cluster (and also the cheapest to run). You can explore other options by running doctl kubernetes --help.

The command will take several minutes to complete, and you should see an output like this:


2
3
4
5
6
7
8
$ doctl kubernetes cluster create mycluster --size s-1vcpu-2gb --count 1
Notice: Cluster is provisioning, waiting for cluster to be running
....................................................
Notice: Cluster created, fetching credentials
Notice: Adding cluster credentials to kubeconfig file found in "/Users/david/.kube/config"
Notice: Setting current-context to do-nyc1-mycluster
ID                                      Name         Region    Version        Auto Upgrade    Status     Node Pools
4cf2159a-01c1-423c-907d-51f19c3f9a01    mycluster    nyc1      1.20.2-do.0    false           running    mycluster-default-pool

As you can see, the command automatically adds cluster credentials and a context to the ~/.kube/config file, so you should be able to access your cluster using kubectl:


2
3
4
5
6
$ kubectl get namespace
NAME              STATUS   AGE
default           Active   24m
kube-node-lease   Active   24m
kube-public       Active   24m
kube-system       Active   24m

Create Dummy Microservices

To represent backend microservices, I’m going to use a trivial Python Flask application that returns a JSON string:

foo.py


2
3
4
5
6
7
8
9
from flask import Flask
app = Flask(__name__)

@app.route('/foo')
def hello():
    return '{"msg":"Hello from the foo microservice"}'

if __name__ == "__main__":
    app.run(debug=True, host='0.0.0.0')

This Dockerfile builds a docker image you can deploy:

Dockerfile

FROM python:3-alpine

WORKDIR /app

RUN echo "Flask==1.1.1" > requirements.txt
RUN pip install -r requirements.txt
COPY foo.py .

EXPOSE 5000

CMD ["python", "foo.py"]

The files for our “foo” and “bar” services are almost identical, so I’m only going to show the “foo” files here.

This gist contains files and a script to build foo and bar microservices docker images and push them to Docker Hub as:

• digitalronin/foo-microservice:0.1
• digitalronin/bar-microservice:0.1

Note: You don’t have to build and push these images. You can just use the ones I’ve already created.

Deploy Dummy Microservices

You’ll need a manifest that defines a Deployment and a Service for each microservice, both for “foo” and “bar.” The manifest for “foo” (again, I’m only showing the “foo” example here, since the “bar” file is nearly identical) would look like this:

foo.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: foo-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: foo
  template:
    metadata:
      labels:
        app: foo
    spec:
      containers:
      - name: api
        image: digitalronin/foo-microservice:0.1
        ports:
        - containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:
  name: foo-service
  labels:
    app: foo-service
spec:
  ports:
  - port: 5000
    name: http
    targetPort: 5000
  selector:
    app: foo

This gist has manifests for both microservices, which you can download and deploy to your cluster like this:

$ kubectl apply -f foo.yaml
$ kubectl apply -f bar.yaml

Access the Services

You can check that the microservices are running correctly using a port forward:

$ kubectl port-forward service/foo-service 5000:5000

Then, in a different terminal:


2
$ curl http://localhost:5000/foo
{"msg":"Hello from the foo microservice"}

Ditto for bar, also using port 5000.

Install Kong for Kubernetes

Now that you have our two microservices running in our Kubernetes cluster, let’s install Kong.

There are several options for this, which you will find in the documentation. I’m going to apply the manifest directly, like this:

$ kubectl create -f https://bit.ly/k4k8s

The last few lines of output should look like this:

...
service/kong-proxy created
service/kong-validation-webhook created
deployment.apps/ingress-kong created

Note: You may receive several API deprecation warnings at this point, which you can ignore. Kong’s choice of API versions allows Kong Ingress Controller to support the broadest range of Kubernetes versions possible.

Installing Kong will create a DigitalOcean load balancer. It’s the internet-facing endpoint to which you will make API calls to access our microservices.

Note: DigitalOcean load balancers incur charges, so please remember to delete your load balancer along with your cluster when you are finished.

Creating the load balancer will take a minute or two. You can monitor its progress like this:

$ kubectl -n kong get service kong-proxy
NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
kong-proxy                LoadBalancer   10.245.14.22    <pending>     80:32073/TCP,443:30537/TCP   71s

Once the system creates the load balancer, the EXTERNAL-IP value will change from <pending> to a real IP address:

$ kubectl -n kong get service kong-proxy
NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                      AGE
kong-proxy                LoadBalancer   10.245.14.22    167.172.7.192   80:32073/TCP,443:30537/TCP   3m45s

For convenience, let’s export that IP number as an environment variable:

$ export PROXY_IP=167.172.7.192 # <--- use your own EXTERNAL-IP number here

Now you can check that Kong is working:

$ curl $PROXY_IP
{"message":"no Route matched with those values"}

Note: It’s the correct response because you haven’t yet told Kong what to do with any API calls it receives.

Configure Kong Gateway

You can use Ingress resources like this to configure Kong to route API calls to the microservices:

foo-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: foo
  namespace: default

spec:
  ingressClassName: kong
  rules:
  - http:
      paths:
      - path: /foo
        pathType: Prefix
        backend:
          service:
            name: foo-service
            port:
              number: 5000

This gist defines ingresses for both microservices. Download and apply them:

$ kubectl apply -f foo-ingress.yaml
$ kubectl apply -f bar-ingress.yaml

Now, Kong will route calls to /foo to the foo microservice and /bar to bar.

You can check this using curl:

$ curl $PROXY_IP/foo
{"msg":"Hello from the foo microservice"}

$ curl $PROXY_IP/bar
{"msg":"Hello from the bar microservice"}

What Else Can You Do?

In this article, I have:

• Deployed a Kubernetes cluster on DigitalOcean
• Created Docker images for two dummy microservices, “foo” and “bar”
• Deployed the microservices to the Kubernetes cluster
• Installed the Kong Ingress Controller
• Configured Kong to route API calls to the appropriate backend microservice

I’ve demonstrated one simple use of Kong, but it’s only a starting point. With Kong for Kubernetes, here are several examples of other things you can do:

Authentication

By adding an authentication plugin to Kong, you can require your API callers to provide a valid JSON Web Token (JWT) and check each call against an Access Control List (ACL) to ensure callers are entitled to perform the relevant operations.

Certificate management

You can enable integration with cert-manager to provision and auto-renew SSL certificates for your API endpoints so that all your API traffic is encrypted as it travels over the public internet.

gRPC support

Kong natively supports gRPC, so it’s easy to add gRPC support to your API.

You can do a lot more with Kong, and I’d encourage you to look at the documentation and start to explore some of the other features.

The API gateway is a crucial part of a microservices architecture, and the Kong Ingress Controller is well suited for this role in a Kubernetes cluster. You can manage it in the same way as any other Kubernetes resource.

Cleanup

Don’t forget to destroy your Kubernetes cluster when you are finished with it so that you don’t incur unnecessary charges:

1
2
3
4
5
6
$ kubectl delete -f https://bit.ly/k4k8s  # <-- this will destroy the load-balancer

$ doctl kubernetes cluster delete mycluster
Warning: Are you sure you want to delete this Kubernetes cluster? (y/N) ? y
Notice: Cluster deleted, removing credentials
...

Note: If you delete the cluster first, the load balancer will be left behind. You can delete any leftover resources via the DigitalOcean web interface.

Thanks for walking through this tutorial with us. It was originally published on our blog: https://bit.ly/316W3zw

Authentication with Kong's JWT Plugin

Julia Salem — Mon, 15 Mar 2021 22:38:31 +0000

As you build and maintain more applications, your authentication strategy becomes increasingly important. It may also be top of mind for your boss since technology leaders cited “improve application security” as one of their top priorities in this year’s Digital Innovation Benchmark.

The Kong Gateway JWT plugin is one strategy for API gateway authentication. JWT simplifies authentication setup, allowing you to focus more on coding and less on security.

Authentication Is Tough

You know you need a secure front door to your system. If requests don’t have the right credentials, the door should remain locked. If they do have the proper credentials, the entry should be smooth.

But how do you verify that credentials are authentic? And how do you make sure there aren’t other ways for those without the right credentials to get into your system?

Let’s walk through those scenarios as I demonstrate how to secure a service (in this case, an API server) with Kong Gateway and its JWT plugin. I’ll cover all the steps to set up, configure and test the service — giving you the foundational knowledge needed to implement these tools independently.

Core Concepts

First, let’s cover the core technologies. If you’re already familiar with these and just want to get started, feel free to skip ahead.

What Is Kong Gateway?

As more companies move from monolithic systems to microservices, a decoupled front-line API gateway to those services — providing authentication, traffic control, request and response transformation — becomes increasingly crucial. Kong Gateway, which is open source, serves as that thin layer between your users and your upstream microservices.

What Is JWT?

The JSON Web Token (JWT) format lets two parties exchange secure claims. It’s a way of saying, “I am so-and-so, which should give me access to that resource. Here is my access token to prove it.”

A JWT has a data payload signed by a trusted party to prevent spoofing. An authorizer verifies that the JWT token is authentic, allowing (or forbidding) access to that resource. Typically, a JWT payload is not encrypted; it’s open for the whole world to read. However, what’s critical is the authenticity of a token, which depends on a trusted party signing it.

What Does Kong's JWT API Gateway Plugin Do?

In this approach, the plugin serves as the JWT authorizer. It authenticates the JWT in the HTTP request by verifying that token’s claims and ensuring a trusted party signed it. Then, depending on whether these steps were successful, Kong Gateway routes the upstream service request.

Keep in mind that authentication in this context means validating the user’s credentials. That’s the job of the JWT plugin. There’s no way to know how a user got a valid JWT. The system just knows that the user has one and is presenting it for authentication. If the JWT is authentic, you can be confident that the user is who they say.

The Basic Use Case

In this basic use case, I have a login server that accepts login attempts with a user’s email and password. If the email/password checks out, the server generates and signs a JWT and hands it back to the user.

With JWT in hand, the user tries to access our microservice: a simple API server with a single endpoint. Kong Gateway sits in front of your API server, using the JWT plugin for authentication. The user presents his JWT with his request.

First, the plugin verifies the token’s authenticity. Next, it confirms the installation steps of the claims inside the payload. A common claim used is an expiration timestamp for the access token. It’s essentially saying, “This token is valid until this date and time.” So, the plugin will check the token’s expiration date.

If the JWT passes all the necessary checks, Kong Gateway grants access to the requested server endpoint. Otherwise, it responds with 401 Unauthorized.

The approach is quite simple:

Set up a basic Node.js Express server with a single endpoint.
Set up Kong Gateway as an API gateway to your server.
Enable the JWT plugin to protect your server endpoint with JWT authentication.

1. Set Up a Node.js Express Server and Endpoint

On your local machine, create a folder for your project. Then, initialize a new Node.js project. In the following examples, I’ll use yarn, but you could use npm too:


3
~$ mkdir project
~$ cd project
~/project$ yarn init  # Use all of the yarn defaults here.

Next, add Express to your project:

~/project$ yarn add express

In your project folder, create the entry point file, index.js, which will spin up an Express server with a single endpoint. Allow a GET request to /, which will respond with the string, “Hello world!”

/* PATH: ~/project/index.js
*/

const express = require('express')
const server = express()
const port = 3000

server.get('/', (req, res) => {
  console.log(req.headers)
  res.status(200).send('Hello world!')
})

server.listen(port, () => {
  console.log(`Server is listening on http://localhost:${port}`)
})

That was simple enough! Your single endpoint should log the request headers and then send “Hello world!” back to the client with a 200 status.

Start your server:

~/project$ node index.js

You can use your browser to test this new endpoint by visiting http://localhost:3000.

Your API server endpoint should be working now!

Next, use Insomnia to send the request and inspect the response. Because of its usability, you’re going to want to use Insomnia exclusively once you start sending requests with a JWT.

In Insomnia, create a GET request to http://localhost:3000.

In Insomnia, you should get a 200 OK with “Hello world!” in the response body.

It looks like the API server is up and running. Now, it’s time to put Kong Gateway in front of it.

2. Set Up Kong Gateway

I won’t cover the details here, but the Kong Gateway installation steps may look different depending on your system.

Once you’ve installed Kong, you’ll need to take a few additional steps.

DB-Less Declarative Configuration
There are two primary ways to configure Kong. Imperative configuration issues step-by-step configuration commands to Kong through its admin API. Meanwhile, declarative configuration stores the entire configuration in a single .yml file then loads it into Kong upon startup. Additionally, you can configure Kong to hook into your database, providing more control over the different nodes it manages.

For the simple setup example, I’ll use database-less declarative configuration. When you start up Kong, you’ll tell it where to find a .yml file with all of the configuration declared within.

In your project folder, run the following command, which generates an initial kong.yml declarative configuration file.


2
3
4
5
6
7
8
9
~/project$ kong config init
~/project$ tree -L 1
.
├── index.js
├── kong.yml
├── node_modules
├── package.json
└── yarn.lock
1 directory, 4 files

Next, you’ll need to configure the system’s kong.conf file before starting up Kong. If you’re working on Ubuntu, you’ll be working in /etc/kong. Here is a template to copy over and then edit.

~/project$ cd /etc/kong
/etc/kong$ sudo su

root:/etc/kong$ tree
.
├── kong.conf.default
└── kong.logrotate
0 directories, 2 files

root:/etc/kong$ cp kong.conf.default kong.conf

There are only two edits you need to make in your kong.conf file.

# PATH: /etc/kong/kong.conf

# Around line 839, uncomment and set to off
database = off

# Around line 1023, uncomment and set an absolute path to kong.yml
declarative_config = /PATH/TO/YOUR/project/kong.yml

When Kong starts up, it will be in DB-less mode, meaning it will look to your project’s kong.yml file for a configuration.

Finally, you’ll need to edit your kong.yml file to set up a gateway in front of your API server “hello world” endpoint.


2
3
4
5
6
7
8
9
10
11
12
13
/* PATH: ~/project/kong.yml
*/

_format_version: "2.1"

services:
- name: my-api-server
  url: http://localhost:3000/
routes:
- name: api-requests
  service: my-api-server
  paths:
    - /api

Let’s go over this.

The _format_version metadata specifies the version number of your declarative configuration format.

Next, you define your service, which Kong describes as “an entity representing an external upstream API or microservice.” You can name your service my-api-service and specify its URL — you’ll recall that the Express server listens for requests at http://localhost:3000.

Next, define routes, which “determine how (and if) requests are sent to their Services after they reach Kong Gateway.” The (local) URL for Kong is http://localhost:8000. You should declare your route so that Kong listens for requests at http://localhost:8000/api, then routes to your service.

Let’s see this in action. Make sure your Express server is running in a separate terminal. Then, start Kong.

~/project$ sudo kong start

In your browser, go to http://localhost:8000/api

Kong Gateway is up. Finally, add authentication.

3. Attach JWT Plugin to Kong Gateway

To add the JWT plugin, add a “plugins” definition to your kong.yml file:

/* PATH: ~/project/kong.yml
*/

_format_version: "2.1"

services:
- name: my-api-server
  url: http://localhost:3000/
routes:
- name: api-requests
  service: my-api-server
  paths:
    - /api
plugins:
- name: jwt
  service: my-api-server
  enabled: true
  config:
    key_claim_name: kid
    claims_to_verify:
    - exp

Here, you can add the plugin named jwt and attach it to your service called my-api-server. For its configuration options, tell the plugin to check the exp value to verify that the access token has not expired.

At this point, restart Kong and see what happens:

1
~/project$ sudo kong restart

The response is 401 Unauthorized. Excellent! Kong now requires a valid JWT for any requests to your API server. Next, you need to tell Kong what constitutes a valid JWT.

In kong.yml, you need to add a consumer and a credential. Kong describes consumers as being “associated with individuals using your Service, and can be used for tracking, access management, and more.” In a more elaborate setting, every one of your API users could be a consumer. That’s a use case you can read more about towards the end of this article. In this situation, your login server is your consumer. Your login server will be the entity generating JWTs and handing them out. Users who make a request to Kong will be holding a “login server” JWT.

Edit your kong.yml file by adding the “consumers” and “jwt_secrets” definitions:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
/* PATH: ~/project/kong.yml
*/

_format_version: "2.1"

services:
- name: my-api-server
  url: http://localhost:3000/
routes:
- name: api-requests
  service: my-api-server
  paths:
    - /api
plugins:
- name: jwt
  service: my-api-server
  enabled: true
  config:
    key_claim_name: kid
    claims_to_verify:
    - exp
consumers:
  - username: login_server_issuer
jwt_secrets:
  - consumer: login_server_issuer
    secret: "secret-hash-brown-bear-market-rate-limit"

You’ve added a new consumer, named login_server_issuer. Then, you added a JWT API gateway credential for that consumer, which contains the secret used to sign JWTs for this consumer. Authentication requires two parts:

The kid (key identifier) value in the JWT header, which is a unique identifier that lets the plugin determine which consumer allegedly issued this JWT
Verification of the consumer’s secret – Was this the secret used to sign this JWT API gateway? If so, then this JWT is authentic.

Before continuing, remember to restart Kong:

1
~/project$ sudo kong restart

If you want to generate a JWT for testing, you need the secret (which you have) and the key to use for the kid value. Kong gives us access to that value through its admin API at http://localhost:8001. You send a GET request to the admin API’s endpoint /consumers/CONSUMER-USERNAME/jwt. This gives us information about this consumer’s JWT credential.

As you inspect this credential’s information, you should see the JWT secret and signing algorithm. What you’re looking for, though, is the key. In the above example, that’s 1nzcMG9Xg7n1lLgmltHnkAmkt7yp4fjZ. This is what you use as the kid value in the JWT header. The Kong plugin will see this kid value, track down the associated consumer and secret, then make sure the JWT was signed with that secret.

To test this, let’s start with the happy path. You need a JWT with a header that includes the correct kid value, signed with the right secret. For simplicity, let’s do this at jwt.io. Here, you can craft your payload, set the signing secret and then copy/paste the resulting JWT.

In the payload, the kid must match the key value from above. Also, because you configured the plugin to check JWT token expiration, you should set the exp (Unix timestamp) far into the future. The name and email are inconsequential; they just demonstrate that you can put other helpful data in the JWT payload.

Lastly, include your JWT secret at the bottom for proper signing. The result is an encoded JWT.

Back in Insomnia, you have your original request that resulted in 401. You need to add “Authorization” to that request. Choose “Auth – Bearer Token,” then paste in your encoded JWT from above.

Now, with a valid JWT attached, resend the request.

Your JWT should have been validated, and Kong routed us to the API server!

If you look back at the terminal running the Express server, you’ll recall that you’re logging the request headers to the console. When the JWT plugin authenticates an access token, it writes some additional values to the upstream headers, namely the consumer id, username and credential identifier (the key value).

But what happens if your JWT is not valid? Let’s test and see.

First, sign the JWT with a different secret. Back at jwt.io, keep the payload, but change the signing secret. Copy the resulting JWT to Insomnia, and send your request again. You’ll get a 401 with “Invalid Signature.”

If your secret is correct, but the kid is incorrect, Kong won’t find an associated credential. Without that credential, there’s no way to find the secret for authenticating the JWT.

Lastly, if the exp value is in the past, then your JWT has expired. Just as you expected, you get the following response.

And that’s it! You should be up and running with Kong Gateway and the JWT Plugin acting as an authentication layer in front of an API server.

Set It and Forget It

Implementing authentication —and getting it right —is hard work. As microservices become the norm, delegating authentication makes more and more sense. “Rolling your own” implementation for JWT authentication can muddy a code base and still leave you wondering if you got it right. By using well-tested and community-adopted services that handle concerns like routing, logging or authentication, developers can shift their focus back to their projects’ unique needs.

With that, you now have a solid foundation for getting started with Kong Gateway and the JWT plugin. It’s time to get to work.

Thanks for walking through this tutorial with us. It was originally published on our blog: https://bit.ly/3eJGhCS