Integra & Docker: A Practical How-To
What is green and has the shape of a container? Why Integra, of course -- nicely packaged in Docker containers.
As you might have seen in our Integra 1.1 announcement, one of the coolest things we did was to push every Integra provider and the Integra Reactor to DockerHub. The container revolution is coming just like the VM revolution came many years ago; if you haven't had the chance to learn and familiarize yourself with Docker and containers in general, this is as good a time as any.
Having the ability to execute Integra in Docker containers gives you tremendous power in terms of scalability: run them in EC2 Container Services with an Elastic Load Balancer, let Kubernetes in the Google Container Engine do the same thing for you, or choose from any of the other container options available out there. Either way, before you get to that point this post will help you get off the ground and try out Integra in containers on your own. We will go down to the Linux CLI, upgrade the Kernel and do other things by hand. Sounds scarier than it actually is but we'll cover some interesting concepts that will be helpful to know. Other platforms such as Project Atomic take care of those things for you, but more importantly provide a production-ready environment where versions have been tested for compatibility, etc. Please treat this blog post as a POC / learning experiment.
This blog will produce a CentOS 6.6 minimal image with a 3.10 kernel, running the Integra Reactor and SSH Provider in Docker 1.7. We will also cover how to install the Integra vCenter plugin so you have an end-to-end Integra setup.
Things We Need
We don't have many moving parts, but we have some nonetheless. Please download:
Installing CentOS and the vCSA are outside of the scope of this blog, so we will assume you have those ready to go. Installing Linux nowadays has become a matter of clicking Next, Next, Next, so I don't have a shadow of a doubt that you will have any trouble installing that. Along the same lines, installing and configuring vCSA is also relatively straightforward; there is plenty of documentation at VMware to help you get going.
All of the CLI commands below are available in a PasteBin for your reference, in the order they appear.
With CentOS 6.6 up and running, the task at hand is upgrading the kernel to version 3.10. Containers have been around for a while but are just now gaining popularity. If they had souls they would live in the Linux kernel, so getting the engine upgraded to a Docker-required level is essential. Let's drop to a shell as root and upgrade the kernel.
First, let's do an overall update to the system.
Hit Y when prompted to proceed and relax for a few minutes while the update takes place. Once this is finished, we will download v3.10 of the kernel and install it (source reference).
Modify /etc/grub.conf and set the default from 1 to 0. This indicates the bootloader to boot our newly installed 3.10 kernel.
At this point you are golden: verify you have kernel version 3.10.
Since Integra will be running as containers in our CentOS host, there is one last step we need to take care of so we can expose the Integra reactor to the outside world. With a simple change to enable IP forwarding, we will be good to go.
Docker makes some solid instructions available for installing the software. It literally takes 5 steps to get Docker up and running.
For the purposes of this blog we didn't go the extra step of creating a Docker group nor pulling down the hello-world image as mentioned in the Docker documentation.
Let's check that Docker is installed and running correctly.
Ready For Integra
With all the foundation pieces in place we can go ahead and pull Integra images and run them. In this example we will run the Reactor and the SSH provider; running other providers is equally simple, so there is no need to run them all for the purposes of this blog.
If you recall, we will also install the Integra vCenter plugin. The plugin needs direct access to the Integra Reactor, so if you are behind a firewall you will need to open port 8443. That is the only port that needs access because all other communication will be between the Reactor and the SSH provider. Given that both are running in containers under the same host, both share the same virtual network and can communicate between each other.
At this point we can verify no containers are currently running.
Let's go ahead an pull down the Reactor from DockerHub.
Once the image is fully downloaded we can kick it off. We get back the Container ID.
Notice the use of the --iptables=true and --icc=true flags. The first one tells Docker to go ahead an modify iptables on our behalf so the port on the host is open and forwarded to the container, the second is shown for illustration purposes (the default value is true) but it tells Docker that we want to allow container communication. If you look closely at the -p 8443:8443 option, this means we want to map port 8443 on the host to port 8443 on the container. Running the ps command reveals that in the PORTS column.
Last but not least, let's check that the Reactor is actually responding to requests on port 8443. We find the IP address by inspecting the container.
We can clearly see the Reactor's version, running pretty at 1.1.0.
Pulling and running the SSH provider is almost identical to running the Reactor as seen above. The only differences lie in not providing the --iptables flag as there is no need to export provider ports to the outside world, and also omitting the -p port mapping option. Again, this is because the Reactor and the SSH provider are running in the same host. If you were to run the provider somewhere else behind a different firewall, you will most definitely have to open up the provider's port in the firewall to allow the Reactor to reach the provider.
Let's go ahead and run the SSH provider.
We issue the ps command and can see both containers running.
All set! Let's now turn our attention to installing the Integra vCenter plugin.
Integra vCenter Plugin
For the plugin we will rely on running Integra's judo installer from the CentOS host. Let's install Java, Git and the other tools that the installer relies on. Having seen the beauty of running things in containers, I know you are thinking "why not package the vCenter deployer in a container?" And we just might... let us know what you think.
After the prerequisite installation finishes successfully, invoke the Integra installer.
Type in 1.1.0 and hit return. After you accept the license agreement, select option 1 to install the vCenter plugin. Option 2 would install the Reactor and all the providers as Linux services, but since we have what we need running in containers this is an option we can safely skip.
Hit 3 to exit out of this menu and let's head over to vCenter to check the Integra plugin. Upon logging in and clicking the Integra icon, you will be prompted for credentials (admin / integra), the Reactor's URL and a license. The URL is the IP address of your CentOS host, and the port is 8443 as you may suspect from our earlier configuration. If you have the host behind a firewall, you will have to do port forwarding from your public IP to the host's IP address. The forwarding from the host to the container was taken care of when we used the --iptables flag during container execution.
If you do not have an Integra license, please contact us and request one. We are happy to provide you with an Integra trial license.
Last but not least, let's add the SSH provider. First we need to grab the IP address of the container where the SSH provider is running so we can add it to the Reactor.
We have to give props to the Docker folks for such creative container names. Those given to your containers will differ from those shown in this post, but you can easily retrieve them with the ps command as seen above. The port where the SSH provider is running can also be found in the output of the ps command. With that information in hand, we addd the provider in Integra.
Containers are here to stay, and as you have seen in this post Integra is ready to go with all of its Docker images in DockerHub. Having all the components nicely packed in Docker images makes for deployment of Integra components a very simple operation. As you experienced first hand in this blog, the most complex parts are setting up the host, installing Docker and getting all the versions correctly aligned. Once that hurdle is over, doing a pull and a run are trivial operations.
You may be wondering what happens to all your workflows when you pull and run a new version of the Reactor. Also new in 1.1.0 is the ability to store workflows in GitHub; simply export your workflows to GitHub, tear down the container, and when you are ready with the new container image, simply import your workflows from GitHub.
There are many other interesting things that we can do, such as configuring your containers to start piping logs to a centralized server, or exploring local logs by going into the containers themselves. We will save these for future posts, so stay tuned to this space for that. As always, if you have any questions or suggestions, drop us a line or reach out in any of our social media outlets.