Setting up Jenkins with Mesos


In this blog, I am going to explain the setups to configure Mesos with Jenkins. Jenkins connect with Mesos using mesos-jenkins plugin which allows Jenkins to dynamically launch Jenkins slaves on a Mesos cluster depending on the workload! Put simply, whenever the Jenkins Build Queue starts getting bigger, this plugin automatically spins up additional Jenkins slave(s) on Mesos so that jobs can be immediately scheduled! Similarly, when a Jenkins slave is idle for a long time it is automatically shut down.

1. Prerequisites

I assume, you have

1. A Jenkins instance with administrator privilege.
2. Working Mesos Cluster with at-least one master and one slave. For instructions on setting up a Mesos cluster, please refer to my blog Setup Standalone Mesos on Ubuntu.

I have installed Jenkins on my OpenStack environment having IP address is 10.1.0.8 and Mesos cluster working at IP address 10.1.0.17 with one master and one slaves.

Mesos-master

2. Installing the plugin

Go to Manage Jenkins > Manage Plugins > There is a tab called ‘available’, once you will click “Available” tab, then choose the ‘mesos’ plugins to install. Scroll all the way down, then you’ll see the “Install without restart” button as well as the “Download new and install after restart” button. The former is the result of this work, allowing you to start using the new plugins right away. The latter is the traditional behaviour, where new plugins take effect after the next restart.

select-jenkins-plugin

Click the button Install without restart on the left, and the plugin gets downloaded, installed, and activated:

Installing-jenkins-plugin-pravin

3. Configuring the plugin

Now go to ‘Configure’ page in Jenkins. If the plugin is successfully installed you should see an option to ‘Add a new cloud’ at the bottom of the page. Add the ‘Mesos Cloud’ and give the path to the Mesos native library (e.g., libmesos.so on Linux or libmesos.dylib on OSX) and the address (HOST:PORT) of a running Mesos master.

configure-mesos-cloud

If you want to test immediately connectivity to Mesos, you can set ‘On-demand framework registration’ to ‘no’ and the framework will appear in Mesos as soon as you save. Otherwise it will register and unregister automatically when a build is scheduled on Mesos.
Note: Ensure Mesos slaves have a jenkins user or the user the Jenkins master is running as. jenkins user should have JAVA_HOME environment variable setup.

4. Set up and try out a build job

Now – set up a new job in Jenkins. On the configure screen, check the box that says “restrict where this build can run”

configure-job

Put in “mesos” (that was in the label in plugin configuration).

At this point you are good to go. If you check the Mesos console, you should see that Jenkins Scheduler is now setup as a framework – which means it is able to accept jobs:

mesos-slave

You have done with all configuration, now you can run the job – and it will run on Mesos. You will see an executor magically appear – and then pause for a little bit (while the slave.jar is setup etc) – and then run the job. Slave will become idle once job complete and after few second, slave will shutdown.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s