lotan

Lotan
=====
####Lotan -- the multi-headed Docker container runner

Wouldn't it be nice to be able to automatically deploy Docker containers?

* without any setup or teardown
* without dedicated hosts, or a virtual servers
* without any commands to run
* without any AWS or other cloud configuration changes
* without any changes to DNS configuration
* without having to change elastic load balancers
* without any extra security rules

For example -- imagine that every GitHub pull request for your project could be
viewed in a browser by simply clicking on its name.

Lotan allows you to automatically run many containers of a Docker repository
concurrently on a single port on a single host. Each container appears under
a subdomain named after the tag for its image.

Setup is easy -- the source code in this GIT repo is pre-configured to run tagged images from the markriggins/todowrangler repository, which are just color variations of
jbdely's angular example application

Once you've finished the setup, you can click on these links to automatically pull,
run, and access various colorful versions todowrangler -- each running in its own container.

* red.todowrangler.lotan.com todo-wrangler
* green.todowrangler.lotan.com todo-wrangler
* purple.todowrangler.lotan.com todo-wrangler

Pull Request Previewer

Lotan was created to allow developers, managers and quality-assurance teams to easily
view containers running for GitHub pull requests.

Suppose we have an web-application named todowangler that expects to run on
port 80. Jenkins has been configured to build the todowrangler project, and
successful builds are pushed to a Docker registry and tagged with the pull request
number.

The following images have been built and pushed to DockerHub

|Images|Pull Request Number|Description|
|:---------|----|-------|
|markriggins/todowrangler:pr-3001|3001|Change the banner to Green|
|markriggins/todowrangler:pr-3002|3002|Change the banner to Purple|

Once everything has been setup, the todowrangler application can be automatically
viewed at any available tag by simply using the tag as a sub-domain prefix
to the URL like this:

|Tag|URL|
|:---------|:----|
|pr-3001| http://pr-3001.todowrangler.lotan.info|
|pr-3002| http://pr-3002.todowrangler.lotan.info|

Notice that once Lotan has been configured, new images can be run with **no additional
setup or intervention**, no AWS Task Definitions, no Docker compose commands, yaml
editing etc, no linux shell stuff. It just happens.

Other use cases

Lotan can be used to automatically run any tagged image from a Docker repository
that is compatible with HTTP and nginx proxy_pass.

How it works

1. Tags -- Each container runs on its own separate sub-domain, that is
named after the images's tag
2. Proxy -- All requests are handled by a special nginx proxy that routes
requests to running containers based on the sub-domain names.
3. Containers -- If no container exists for the tag, then the tagged image
will be pulled from the Docker registry and a new container will be run to
handle requests on that subdomain
4. Ports -- Each container runs on a randomly-assigned port, so you can
run as many simultaneous versions of the image as desired with no conflicting
ports
5. Images -- Lotan keeps a record of containers and images, and a queue
of actions to perform (such as starting a new container)
6. lotan-queue-processor -- A lotan-queue-processor process does all of
the image pulling and container running by scanning its queue for actions.
To start a new container, the lotan-queue-processor pulls the desired
image at the specified tag from the registry and runs a new container on a
random port. If the container successfully starts, then the random port is
recorded for future use by the above proxy; otherwise, the errors are saved
for future display via the admin interface.
7. lotan-reaper -- The lotan-reaper process periodically removes stale containers
that have not been recently accessed and restarts containers that have somehow
stopped but should be running. Stale images are periodically refreshed by pulling
them from the registry.

Setting It all Up

#### Configuring Lotan
1. Install Lotan on a host and setup DNS to route the the top level domain
and a wild-card for subdomains:

tagwrangler.lotan.info
*.tagwrangler.lotan.info

The wild-card DNS name that will match any sub-domain.
See Wild-Card DNS Names for details

Note: you can use any DNS name you wish

For demonstration purposes, you can add a short list of known-in-advance
image names to your /etc/hosts file, using the IP address of your Docker engine,
but the beauty of using wild-card DNS is that you do not have to know the tags
in advance

192.168.99.100 todowrangler.lotan.info
192.168.99.100 red.todowrangler.lotan.info
192.168.99.100 green.todowrangler.lotan.info
192.168.99.100 purple.todowrangler.lotan.info
192.168.99.100 bad.todowrangler.lotan.info

2. Configure the host with routing rules and privileges to access any resources
needed by the images you wish to run.

3. Run Lotan, using the following command

docker run --rm -it \
-p 80:80 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $(which docker):/bin/docker \
--name lotan \
markriggins/lotan

By default, Lotan will run images from the markriggins/todowrangler repository,
which contains images for the tags: red, green, blue, purple and pr-3002. But
you can alter Lotans behavior by setting the following environment variables.

4. Environment Variables

* TERM -- the terminal type. Default: xterm
* REPOSITORY -- the docker repository of images. Note that Lotan will only run
images from a single repository. Default: markriggins/todowrangler.
* TL_DOMAIN -- the top-level DNS domain which must route to the host where
Lotan is running. Default: todowrangler.lotan.info
* INACTIVITY_STOP_TIMEOUT_MINUTES -- Stop containers that are inactive for more
than $INACTIVITY_STOP_TIMEOUT_MINUTES minutes. Default: 10
* INACTIVITY_RMI_TIMEOUT_MINUTES -- Remove images that are inactive for more
than $INACTIVITY_RMI_TIMEOUT_MINUTES minutes. Default: 60
* IMAGE_REFRESH_INTERVAL_MINUTES -- Refresh images from the registry every
$IMAGE_REFRESH_INTERVAL_MINUTES minutes. Default: 5
* ENV_ARGS -- a list of env args and values to pass to containers
* LOTAN_CONTAINER_PORT -- Lotan proxies to a single port. If a container exposes multiple ports, then you must choose one and set export its value thru this environment variable.
* TRANSLITERATE_SEMVER_UNDERSCORES_TO_DOTS -- Set to 0 to stop flattening tags like 1.9.5 to 1_9_5. [ The default behavior makes the tags fit into a single subdomain level, which works better for SSL wild-card certificates]
* TAG_FILTER -- a Lua regex to filter tag names listed in the drop-down list. NOTE: Lua has its own
rules for regexes; for example, % is the escape character!!
* ADMIN_WELCOME -- HTML to present on the ADMIN page (home page) as a welcome message.
* ADMIN_INFO -- HTML to present on the ADMIN beneath the Containers table.

#### Passing Environment Variable to the Containers

Environment variables that begin with LOTAN_CONTAINER_ENV_ will be passed thru to the container without the prefix.
In other words, if you define LOTAN_CONTAINER_ENV_PLANET="earth" for Lotan, then Lotan will pass the environment
variable PLANET="earth" to each container.

#### AWS ECR Setup
The AWS ECS Registry requires a special login step before images can be pulled from the registry. Lotan will handle this for you, but you must define some extra environment variables:

-e AWS_DEFAULT_REGION=us-east-1 \
-e AWS_SECRET_ACCESS_KEY=my-secret-aws-key \
-e AWS_ACCESS_KEY_ID=my-access-key-id \

If the user-account that starts Lotan has an AWS credentials file (such as ~/.aws/credentials), then you can mount the credentials file as a volume and keep the AWS_SECRET_ACCESS_KEY off the command-line where is may be viewed by a ps -ef

docker run -d -v $HOME/.aws/credentials:/root/.aws/credentials \
-e AWS_DEFAULT_REGION=us-east-1 ...

In general, secrets should not be passed through the environment in AWS as part of a TaskDefinition because aws ecs describe-task-definition will include the environment variables, and thus leak the secrets to any user account with permission to describe task definitions

Testing it

1. Auto-Run
Do all the setup and then try accessing subdomains for known-good tags. If the
tag is good, on the first attempt you will be given a spinner page that will
wait for the image to pull and the container to start running. If you wait long enough, then the spinner will be replaced with the output from the new container.

Containers usually start quickly -- but you don't have to wait -- just try the
url again in a few minutes.

2. Admin
A very simple admin panel is available at the top level domain, which will
list the running containers an allow you to start, stop or remove them, as
well as add new tags.

* Click on the tag to access the container
* Click Stop to stop the container
* Click X to remove it.
* Enter a tag and click Add to start a new container

The admin interface runs on your TL_DOMAIN http://green.todowrangler.lotan.info.
See Setting It all Up for details on customizing your TL_DOMAIN.

3. Common errors and their meanings

|Errors|What it means|
|-------|---|
|no container tag| Lotab could not determine the tag by the DNS name that you used to access the site. Your dns subdomain name must be a valid Docker tag without dashes, and you must have wild-card DNS working
|no such container tag| Lotan tried to pull the image with the given tag but no such image exists
|Error messages | When Lotan tried to start the image, it failed with the given error messages (Displayed in your browser when you try to access the subdomain)

Security Concerns

Lotan is a simple router -- your application is responsible for its own security.

Lotan minimizes security risks by:

* limiting its user inteface to simple tags and actions
* restricting itself to running images from a single repository
* simply proxying requests to your application

But Lotan is alpha-quality -- DO NOT use Lotan for sensitive or personally identifiable
information

Lotan intentionally limits itself to work with a single repository.
Although it might seem to be a good idea to be able to run just any image,
you must remember that docker containers from arbitrary sources pose significant
security risks. So for security reasons, run a separate Lotan host
for each repository that you wish to support.

The error messages that Lotan displays to browsers are limited to those that
occur while trying to start the image, typically Docker error messages, but
there is some risk of leaking information if your application writes sensitive
information to stdout or stderr during startup.

Performance and Stability

Lotan is alpha quality for use at your own risk. Since it only runs
a single container per image, Lotan should only be used for testing
and previewing, never for production.

Wild-Card DNS Names

If you are using AWS Route 53, then you can add A records for *.your-domain,
but life is more difficult for OS X and even Linux

1. dnsproxy -- One solution is to use a dns proxy that does understand wildcards -- such
as dnsproxy.py

2. Dnsmask -- On OS X, you can brew install Dnsmasq

3. Linux -- On linux systems, if you are running a local DNS server
then checkout http://rivenlinux.info/wildcard-dns/

4. /etc/hosts -- If you know the possible subdomain names in advance, you
can simply add them to your /etc/hosts file

Future Features

1. On-demand builds. If no image exists on the Docker registry with the tag,
but the tag exists in the GIT repository, then:
a. clone the GIT repository
b. checkout the tag
c. build the image
d. run it locally.
e. support an option to also push the image to the registry

But I think this is better left to Jenkins or Travis

2. Multiple Repositories per Registry. We could determine the registry from the
domain name like we do now for the tag.

* < tag > . < registry > . lotan.info
* registry white list -- we would have to lock down the registry to a
white list
* The Admin UI would have to support multiple repositories.