Skull Island


Skull Island is a declarative configuration management tool for
Kong API Gateway to backup and synchronize Kong
API Gateway configuration. We recommend using Kong Dashboard
to add routes through a UI and then using Skull Island to backup the
configuration changes and later synchronize it. Skull Island is inspired by Kongfig and Biplane.

Installation

This application is meant to be used as a command line tool.
You can install the latest version globally:
``bash
npm install -g skull-island
`

If you have an older version and need to upgrade to the latest version:
`bash
npm upgrade -g skull-island
`

Running the application

$3


In order to backup a running Kong API gateway configuration to disk, use
the backup command:
`bash
skull-island backup --url http://127.0.0.1:8001
`

Kong API Gateway is running on 127.0.0.1 and the administration port
is running on port 8001. You can find additional parameters using
skull-island backup -h. The default backup file is generated in the
current directory called kong-backup.json.

$3


In order to synchronize a configuration on disk to a running Kong API
Gateway, use the synchronize command:
`bash
skull-island synchronize --url http://127.0.0.1:8001
`

Kong API Gateway is running on 127.0.0.1 and the administration port
is running on port 8001. You can find additional parameters using
skull-island synchronize -h. The default backup file that is used for
the synchronization process must exist in the current directory and is
expected to be called kong-backup.json.

$3


In order to wipe a Kong API Gateway clean of entities (APIs, Plugins,
Consumers, and Consumer Credentials), use the teardown command:
`bash
skull-island teardown --url http://127.0.0.1:8001
`

Kong API Gateway is running on 127.0.0.1 and the administration port
is running on port 8001.

Building out a Kong API gateway environment to do local testing

Ensure you have the latest version of Docker running:

$3

To bring up Cassandra and the Kong API Gateway, use
`
docker-compose up
`

You can add a -d flag if you want to run it in the background and not
have the logs pollute your terminal. You can use Kong Dashboard via
http://localhost:8080 and use http://kong-api-gateway:8001 as the
Kong node URL since Kong-Dashboard is running inside a Docker
container so we make use of Container Name DNS resolution.

Once you are finished, and want to clean up, use
`
docker-compose down
`

$3

1. create a virtual Docker network that will host Kong and Cassandra so
they may communicate with each other

`
docker network create kong-network
`

2. create the Cassandra Docker container on the kong-network

`
docker run -d --name kong-database --network kong-network cassandra:3
`

3. create the Kong API Gateway Docker container on the kong-network and
expose ports over to the host network to access the admin and proxy APIs

`
docker run -d --name kong --network kong-network -e "KONG_DATABASE=cassandra"
-e "KONG_CASSANDRA_CONTACT_POINTS=kong-database"
-p 8000:8000
-p 8443:8443
-p 8001:8001
-p 7946:7946
-p 7946:7946/udp
kong:latest
`

Integration tests


In order to run tests, you need to start up Kong locally by following
the Docker Compose instructions and run: docker-compose up

Once the services are up, you can run integration tests by executing:
gulp test

Please note, you will need to install gulp globally to do this.

Caveats and Design Decisions


Kong's basic-auth consumer credentials currently cannot be backed up
and re-applied because the backup obtains encrypted credentials and
reapplying the encrypted credentials causes the credentials to be
encrypted once more which is incorrect and will cause your credentials
to stop working. As a result, we disable synchronization of basic
authentication consumer credentials by default. There is a flag (-b)
which allows you to synchronize basic authentication credentials
provided that you store plaintext credentials in the backup file. The
intended use case for Skull-Island was to be used in conjunction with
Kong-Dashboard in order to
have a better process workflow rather than coming up with your own JSON
configuration.

For the synchronization process, extra entities for APIs, Plugins and
Consumers that are present on the server and are not present on disk are
deleted. For now, all Consumer Credentials (except basic-authentication
unless specified) are removed from the server completely and then
synchronized from disk resulting in slight downtime.

$3

As part of the workflow, you might introduce
fill-in-the-blanks,
to minimize duplication for Skull-Island backups especially when you
need to deploy your changes to multiple environments. upstream_url
field is a Kong URL required field. if the kong-backup.json file has
some endpoints without proper upstream_url field then those specific
endpoints will be skipped and the script won't fail during synchronization
process. This decision was taken because some environments may have more
APIs defined when compared to other environments.

$3

Certificate backups are supported but follow a different convention. By
default, Kong places the public and private key data directly in the
JSON response. For example:
`json
{
"created_at": 1485521710000,
"cert": "public-key1",
"key": "private-key1",
"id": "0e06dee7-ddba-4398-87b9-134944b0aa91"
}
`

Our convention is to refer to file paths for certificates:
`json
{
"id": "0e06dee7-ddba-4398-87b9-134944b0aa91",
"created_at": 1485521710000,
"cert_path": "certificates/0e06dee7-ddba-4398-87b9-134944b0aa91.pub.pem",
"key_path": "certificates/0e06dee7-ddba-4398-87b9-134944b0aa91.pem"
}
`

In the example above, we create pem files and refer to them via
relative file paths (so they will materialize in your current directory
in the certificates folder). cert_path corresponds to your public key
and key_path corresponds to your private key. Both keys are PEM
encoded.

Note: If your certificate paths do not contain the certificates, the
synchronization of these certificates will be skipped and a warning will
be printed.

$3


The release process uses the release-me NPM module

$3

To my understanding, the dependency graph can be visualized like this:

- Consumers -depend on-> Plugins -depend on-> APIs
- SNIs -depend-on-> Certificates


There are global Plugins which have no dependency on APIs like Syslog.
So in terms of deleting or adding, you need to start with the Consumers,
followed by Plugins and finally API`s.

In order to perform a synchronization, you need to pull down the current
state of the API gateway and have the backup file on hand. You need to
perform the following steps for Consumers, Plugins and APIs

- Check what is present in the backup file and compare the entries to
the entries pulled from the API gateway,
- if we have less entries in the file then we need to remove entries
from the API gateway.
- If we have more entries in the file then we need to add entries
into the API gateway

- See if existing entries need to be updated (you could always blindly
update the server with the data from the file to avoid complications)

Note: you cannot rely on blind updates to existing records
(for Consumer Credentials), you must look at each one for differences
and if you find differences, then delete the one on the server and
upload the new one. You could actually do this process blindly
(delete then add).