There's been a lot of interest on how to migrate from a Rails to a Docker stack. While we still continue to support the native Rack stack, this blogpost is for anyone who's starting to experiment with Docker and Cloud 66. The aim is to help you take your first steps towards polyglot development, so you can start to mix and match technology in support of microservice architectures.
For the purpose of this help guide, we've made the assumption the reader has no previous knowledge of Docker, and therefore will walk you through the simplest possible way to dockerize your Rails/Rack-based application in 3 easy steps:
- Use 'Starter' for generating the files needed for Docker and Cloud 66 to deploy your application
- Make sure your image builds and runs on your machine using Docker
- Deploy your application on Cloud 66
We'll use a simple (and not particularly inventive!) Ruby on Rails project, using MySQL found here.
Dockerizing your app with Starter
We'll use Cloud 66 Starter which is an open-source command line tool, which generates a Dockerfile, docker-compose.yml and a service.yml file from arbitrary source code.
You can download the latest release executable and run it on your development machine. On MacOS make sure the downloaded release is executable:
$ chmod a+x starter
After you install Starter, you can run it in your repository:
cd /my/project/directory
starter
Cloud 66 Starter ~ (c) 2015 Cloud 66
Checking templates in /Users/andreas/.starter
----> Downloading from https://raw.githubusercontent.com/cloud66/starter/master/templates/templates.json
Local templates are up to date
Detecting framework for the project at /Users/andreas/code/agalanom/rails-mysql
Found ruby application
Enter ruby version: [latest] 2.1
----> Found config/database.yml
Found mysql, confirm? [Y/n] y
Add any other databases? [y/N] n
----> Analyzing dependencies
----> Parsing Procfile
----> Found Procfile item worker
----> Found Procfile item scheduler
This command will be run after each build: '/bin/sh -c "RAILS_ENV=_env:RAILS_ENV bundle exec rake db:schema:load"', confirm? [Y/n] y
This command will be run after each deployment: '/bin/sh -c "RAILS_ENV=_env:RAILS_ENV bundle exec rake db:migrate"', confirm? [Y/n] y
----> Writing Dockerfile...
----> Writing service.yml...
----> Writing docker-compose.yml...
Warnings:
* database.yml: Make sure you are using environment variables. -> http://help.cloud66.com/deployment/environment-variables
* No command was defined for 'web' service so 'bundle exec rails server -e _env:RAILS_ENV' was assumed. Please make sure this is using a production server.
Done
Starter will attempt to detect your Ruby version and what database you're using. If those are not detected properly, you have the option of entering the information in the command line when prompted. You are strongly recommended to specify the Ruby version of your project, unless you want to always use the latest version.
The detected databases will be run as containers for development purposes. We generally advise against running your database inside a container in production, as by nature of containers, the data will be lost if your container dies or re-starts. Containers are great for stateless parts of your architecture, but this means you shouldn't rely on the data in the database container.
Running Starter will generate the following files needed for your Docker stack:
- Dockerfile: a Docker specification text document that contains all the commands a user could call on the command line to assemble an image
- docker-compose.yml: a Docker specification file for making it easy to run your dockerized application on your machine and mimic the Docker infrastructure on Cloud 66, without of course all the extra ops stuff you get when running docker in production with Cloud 66.
- service.yml: a Cloud 66 service definition file, which is used to define the service configurations on a stack.
Run your Dockerised application on your machine
Install Docker
First, you need to install Docker on your machine. If you're running Linux, you can install the official Docker packages.
If you’re on OS X or Windows, the easiest way is to install Docker via Docker Toolbox.
You should note that if you're using Docker Toolbox, Docker is not running natively on your system but in a VirtualBox, so you’ll need to run eval $(docker-machine env default) in order to run commands from your terminal. You can also use the Docker QuickStart Terminal that's installed with the Toolbox, which will open a new terminal window to do this for you.
You can very easily run your application using docker-compose. With this tool you don't need to know all the Docker commands such as mounting volumes and binding your service ports. Instead, you can start it with one simple command.
Build your Docker image
First we need to build the image. In your code directory run
docker-compose build
You should get an output like the below:
mysql uses an image, skipping
Building web
Step 1 : FROM ruby:2.1
---> c6ebc3270d1c
Step 2 : MAINTAINER andreas@cloud66.com
---> Running in d2fa44f872b4
---> 2ad4a2b99618
Removing intermediate container d2fa44f872b4
Step 3 : RUN apt-get update -qq && apt-get install -y build-essential nodejs
---> Running in 20adeb6fe6e1
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
[...]
0 upgraded, 13 newly installed, 0 to remove and 33 not upgraded.
Need to get 4903 kB of archives.
After this operation, 12.5 MB of additional disk space will be used.
[...]
---> b05ea1482583
Removing intermediate container 20adeb6fe6e1
Step 4 : RUN bundle config build.nokogiri --use-system-libraries
---> Running in 95a89273c1df
---> 205488384cf0
Removing intermediate container 95a89273c1df
Step 5 : WORKDIR /tmp
---> Running in 27cd1be49d94
---> 6d8601d77d3e
Removing intermediate container 27cd1be49d94
Step 6 : ADD Gemfile /tmp/Gemfile
---> 3715bb7b2505
Removing intermediate container d54d60dc7b65
Step 7 : ADD Gemfile.lock /tmp/Gemfile.lock
---> 7ab2459dd6ea
Removing intermediate container 5bccf7a558b2
Step 8 : RUN bundle install
---> Running in 686734da3529
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
[...]
Bundle complete! 11 Gemfile dependencies, 53 gems now installed.
Bundled gems are installed into /usr/local/bundle.
---> 34ed88cfc33c
Removing intermediate container 686734da3529
Step 9 : ENV APP_HOME /app
---> Running in c25023c553f8
---> 9450495019d7
Removing intermediate container c25023c553f8
Step 10 : RUN mkdir $APP_HOME
---> Running in 80f2718a3c2a
---> c92f8d10adb0
Removing intermediate container 80f2718a3c2a
Step 11 : WORKDIR $APP_HOME
---> Running in 2f2743f46a1d
---> 42259c41ef45
Removing intermediate container 2f2743f46a1d
Step 12 : EXPOSE 3000
---> Running in 6472c8327eb2
---> 8bcd20b4f0d0
Removing intermediate container 6472c8327eb2
Step 13 : ADD . $APP_HOME
---> e805b07a1818
Removing intermediate container d7b1eca730ca
Successfully built e805b07a1818
Useful tip
Add an alias for docker-compose so that you can use it as dc to save some typing! You can do that withalias dc="docker-compose".
Common issues
If the Docker build does not complete successfully, make sure you're using the correct version of the Ruby image. Open your Dockerfile with a text editor and at the top, you should see
FROM ruby:xxx. Avoid usinglatest(which is the default if the version of your project is not detected), as this is likely to break support from some gems you might be using. Instead, as an example, for the latest stable 2.1 release (i.e. 2.1.8 as of now) you can useFROM ruby:2.1. You can see the available tags you can use at the official Ruby Docker Hub repository.As Docker runs Linux, you need to avoid using any Windows-specific code such as using a Gemfile.lock that is generated from a Windows machine.
You are almost guaranteed to use the Nokogiri gem for your XML parsing needs. Unfortunately, because of a current Docker/Nokogiri issue, the build will fail when deploying on Cloud 66. To solve this issue, you need to add the following in your Dockerfile before the
RUN bundle installstep to apply a Bundler config update:RUN bundle config build.nokogiri --use-system-libraries
Run your Docker application
To run your application, you can do
docker-compose up
which starts all the services required - in this case, Rails (web) and MySQL.
To see if the services have started successfully you can run
docker-compose ps
You should see something similar to
Name Command State Ports
------------------------------------------------------------------------------------
railsmysql_mysql_1 /entrypoint.sh mysqld Up 3306/tcp
railsmysql_web_1 bundle exec rails s -b 0.0.0.0 Up 0.0.0.0:3000->3000/tcp
Docker-compose starts all the services at the same time, and some services might be quicker to start than others. In some cases, the web service might initialize first and then fail because it cannot connect to the database.
The way to avoid this is to make your services robust to check if the DB exists before connecting and have a retry script. It's not needed in this example, since we're not running rake db:setup inside the up sequence. Read more about how to do the database setup in your container below in the Database setup note.
If this happens, docker-compose ps will report the state of the service as Exit followed by some exit code. In this case, you can start the services one by one, using
docker-compose up [-d] <service>
where the -d flag will start the service in the background (in daemon mode).
In our case, we can do
docker-compose up -d mysql
docker-compose up web
If you run your service in daemon mode, you can see your application logs by running
docker-compose logs <service>
It's also possible to view the output from all containers interpolated by using the command with no service name argument provided.
For things like running a migration, the docker-compose file generated also hooks up your computer's volume to the running container, so you also can save your migrations on local disk and commit them to your repository if desired.
If you want to run a one-off command you can use:
docker-compose run [--service-ports] <service> <command>
where the optional --service-ports argument runs the command with the service's ports enabled and mapped to the host.
For example, if you want to run a migration, you can run docker-compose run web rake db:migrate. As a bit of a special case, to look inside a container, you can run docker-compose run <service> /bin/bash or /bin/sh if the former is not available.
Database setup
The first time you run your application, make sure mysql is up and running, and run the following command once to setup the database, which runs db:create, db:schema:load and db:seed.
docker-compose run web bundle exec rake db:setup
Common issue
If you find that for any reason the logs of your container are not updating, you can use
docker logs [-f] <container_name>
where the the -f flag will keep following the log and the container name can be found from the output of the docker-compose ps command. For example:docker logs -f railsmysql_web_1 will follow the log output of the web service.
Accessing your application
If you're using Docker Toolbox, to access your application, you can find your Docker Machine IP by runningdocker-machine ip ${DOCKER_MACHINE_NAME}
By default, your application should be accessible at 192.168.99.100:3000
However, it is much more convenient and makes more sense to map this address on your localhost so you can access it directly. You can do that by opening VirtualBox and going to Settings > Network > Adapter 1 tab and click on Port Forwarding. There, you can add a new port forwarding rule for the port you're using (e.g. Host IP: 127.0.0.1, Host Port: 3000, Guest Port: 3000). Note that this is not needed if you are running on Linux.
Common issue
If your server is not accessible, make sure the start command is bundle exec rails server -b 0.0.0.0 to bind to all interfaces.
In the same way as with the up command, you can stop all the containers running with
docker-compose stop
or a specific service with
docker-compose stop <service>
There are also kill and restart commands available, which are self explanatory and work in the same fashion.
Now that we made sure the application is running in Docker, we can create a new Docker stack in Cloud 66.
Deploy a new Docker Stack on Cloud 66.
Cloud 66 expects your Dockerfile to be with your source code, so you need to commit that with the code into your repository.
Now you can use the web interface to create a new stack for your app in Cloud 66. Switch to the advanced tab, give your stack a name and select an environment. Then simply paste the contents of the service.yml generated using Starter and go to the next step. Select your deployment target and cloud provider as normal, and choose if you want to deploy your databases locally, on a dedicated server or to use an external server. That's it, you're good to go!
We'll build your image for you and deploy it on your server. On your stack overview you can see the details of your build by navigating to Build and Deployment to see your deployment history. There you can see the whole build output and identify errors if any have occurred.
We're here to help developers focus on app development, so leave the ops part to Cloud 66. We more than welcome any contributions in Starter - especially templates for Ruby or any other languages you might be developing on! Comments are welcome.