Running your CI tests for opensource project on Travis.Ci seems like a no-brainer. Travis offers quite robust integration into Github and boasts with impressive feature set, that’s flexible and very convenient.
But their offering for private projects is kinda steep, especially when you’re just starting out, like Steve Domin recently noticed.
There are some great hosted alternatives that are cheaper and might work for you, like
- https://drone.io/pricing -> less expensive than jenkins+ec2
Then there is always your trusted Jenkins for old-style control-freaks, who like to host everything themselves.
Is there another opensource solution?
Indeed, there is. Drone.io (the last one in the alternatives list) also offers an opensource version that you can host yourself. It was just recently endowed with a really cool matrix build feature.
The information about it is scattered around and is sometimes a bit outdated, so here are some links to get you started:
Today we’re going to setup a CI for a Phoenix project hosted on Github (or Bitbucket, both at same time is not possible) with a ridiculous fast turn-around time and total cost of 10$/m + some initial time investment.
It’s gonna be a fun and rewarding experience, so let’s get started!
- DigitalOcean account –> Get 10$ credit on signing up (referral link)
- Github account / Bitbucket account
- A Phoenix application, that you want to run CI for
Setup Digital Ocean
We could cheat here and just install one of the provides App-Images on Digital Ocean:
If you want to have more details, it’s not that hard to install the real thing on a plain Ubuntu VM (> 14.04).
Install Docker following the instructions here: https://docs.docker.com/engine/installation/ubuntulinux/
## we assume Ubuntu Trusty 14.04 # update APT sources $ sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D $ sudo touch /etc/apt/sources.list.d/docker.list $ sudo echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" > /etc/apt/sources.list.d/docker.list ## update sources $ apt-get update ## install docker $ sudo apt-get install docker-engine ## start docker daemon $ sudo service docker start ## test that it was installed $ sudo docker run hello-world
Install Drone.io following the instructions here: http://readme.drone.io/setup/overview/
$ sudo docker pull drone/drone:0.4
Configure Github Application
Github Setting -> Applications -> Developer Applications (https://github.com/settings/developers)
- Application Name: Drone CI
- Homepage URL: IP from you DO Droplet, like: http://18.104.22.168/
- Authorization callback URL: http://22.214.171.124/authorize
Contunue Drone.io Configuration
$ sudo mkdir -p /etc/drone/ $ sudo vim /etc/drone/dronerc
# now start your Drone.io container sudo docker run \ --volume /var/lib/drone:/var/lib/drone \ --volume /var/run/docker.sock:/var/run/docker.sock \ --env-file /etc/drone/dronerc \ --restart=always \ --publish=80:8000 \ --detach=true \ --name=drone \ drone/drone:0.4
Open in the browser the IP of your Drone DO Droplet and after a short authorization you will be greated with a list of your projects on Github.
List of projects
Configure a Phoenix project to work with Drone
Now we are ready to setup our project to run on Drone.io. In similar fashion to Travis.ci it reads its configuration from a YAML file named
.drone.yml in the root of the project.
The simplest config for an Elixir project would look like that:
$ cat .drone.yml build: image: bitwalker/alpine-elixir-phoenix:2.0 commands: - mix deps.get - mix test
This would be enough to run a tests for a simple library.
To run tests for a Phoenix project we need to specify a little bit more.
- a Postgres database server
- caching for dependencies (so we don’t have to compile them everytime our build runs)
Also in the current version of Ecto (1.1.3 ) it relies on psql binary to be present to run the mix commands for DB creation + DB migration.
This dependency will be removed in the upcoming version 2.0, see pull request: https://github.com/elixir-lang/ecto/pull/1163 - Switch pg storage management away from psql
To make it even more interesting we’re gonna use an umbrella project with two Phoenix apps with different DB configurations. Somehow there is a bug here, so that the ecto.migrate for one of the databases is not executed during a normal mix test run. As a workaround I run the commands explicitly before tests.
So, to recap:
- install postgresql-client (so psql is present)
- run ecto.migrate for both test DBs
- use PosgreSQL Docker image in compose block
- cache the dependencies + the _build folder (maybe too fragile, needs real life experience over longer time…)
$ cat .drone.yml cache: mount: - _build - deps build: image: bitwalker/alpine-elixir-phoenix:2.0 commands: - apk update - apk add postgresql-client - mix deps.get - MIX_ENV=test mix ecto.create -r Back.Repo - MIX_ENV=test mix ecto.migrate -r Back.Repo - MIX_ENV=test mix ecto.create ecto.migrate -r Front.Repo - mix test compose: database: image: postgres:9.5 environment: - POSTGRES_USER=postgres - POSTGRES_PASSWORD=postgres - POSTGRES_DB=postgres
Here is the example application, that we’re going to test - https://github.com/ruby2elixir/phoenix_crud_benchmark_60_controllers.
It executes full real CRUD controller tests for 60 controllers in one application.
An uncached initial CI test takes something around 2 minutes.
After we have some “warm” caches, it takes less than 1m 15sec to get feedback. For 60 real controllers with DB tests ;)
Those initial results make me smile. A broken build in the evening is much faster to fix if it takes only a few minutes to get the results back.
Also you don’t have to break the bank to foster best practices for running Continious Integration on a tight budget.
Please leave any feedback whether you’ve found that article helpful or if there are some improvements / mistakes.
More links about Drone.io: