Installing Graphite on CentOS 6.2

Having installed RabbitMQ and Sensu in the last post we now need a way to visualize all the data we have gathered. For this purpose we start with Graphite to plot continuous data.

Doing so was quite an adventure since it was hard to find an adequate installation procedure. There are no official RPMs out there, so I had to built my own.

Again, with the hint of a colleague, I was able to get the whole thing done with the following three part series blog post

First, I had to get all the dependent software, for building the RPMs and then installing Graphite.

{% highlight none %} yum install -y gcc zlib-devel curl curl-devel openssl rpm-build gcc-c++ rpm-build python python-ldap python-memcached python-sqlite2 pycairo python-twisted Django django-tagging bitmap bitmap-fonts python-devel glibc-devel gcc-c++ openssl-devel python-zope-interface httpd memcached mod_wsgi {% endhighlight %}

On my virtual machine, I also had to get the wget.

{% highlight none %} yum install -y wget {% endhighlight %}

An Experiment #

Although the blog post used version 0.9.9, I have tried my luck with version 0.9.10. You might want to skip these steps, since it did not work out and I have reverted to 0.9.9 anyways.

As normal user do the following:

{% highlight none %} cd mkdir rpmbuild mkdir rpmbuild/SOURCES {% endhighlight %}

{% highlight none %} cd rpmbuild/SOURCES wget http://launchpad.net/graphite/0.9/0.9.10/+download/whisper-0.9.10.tar.gz wget http://launchpad.net/graphite/0.9/0.9.10/+download/carbon-0.9.10.tar.gz wget http://launchpad.net/graphite/0.9/0.9.10/+download/graphite-web-0.9.10.tar.gz cp carbon-0.9.10.tar.gz carbon-0.9.10.tar.gz.orig cp graphite-web-0.9.10.tar.gz graphite-web-0.9.10.tar.gz.orig cp whisper-0.9.10.tar.gz whisper-0.9.10.tar.gz.orig {% endhighlight %}

{% highlight none %} tar -zxvf whisper-0.9.10.tar.gz cd whisper-0.9.10 python setup.py bdist_rpm rpm -ivh dist/whisper-0.9.10-1.src.rpm cd .. cp whisper-0.9.10.tar.gz.orig whisper-0.9.10.tar.gz cd ../SPECS/ {% endhighlight %}

Edit the spec if you want to add a suffix to the end of the release information and to add a description

{% highlight none %} vi whisper.spec rpmbuild -ba whisper.spec sudo yum –nogpgcheck localinstall -y ../RPMS/noarch/whisper-0.9.10-1.noarch.rpm {% endhighlight %}

Now, let’s do the same for carbon and graphite

{% highlight none %} cd ~/rpmbuild/SOURCES/ tar -zxvf graphite-web-0.9.10.tar.gz cd graphite-web-0.9.10 python setup.py bdist_rpm {% endhighlight %}

Here, I got an error saying:

{% highlight none %} error: error in ‘post_install’ option: ‘distro/redhat/misc/postinstall’ does not exist or is not a file {% endhighlight %}

So, I had to find all files containing “distro” by grep -iR 'distro' *. The result was setup.cfg and MANIFEST.in where I changed “distro” to “dist”. Ran the command again and it worked.

But then I have realized, that the directory “dist” did not exist at all. Clever me! What did I do then? Removed everything and continued with the 0.9.9 versions of whisper, carbon and graphite.

Okay, that was the point where I have removed everything and continued with version 0.9.9.

Building the RPMs and installing Graphite #

{% highlight none %} cd ~/rpmbuild/SOURCES wget http://launchpad.net/graphite/0.9/0.9.9/+download/whisper-0.9.9.tar.gz wget http://launchpad.net/graphite/0.9/0.9.9/+download/carbon-0.9.9.tar.gz wget http://launchpad.net/graphite/0.9/0.9.9/+download/graphite-web-0.9.9.tar.gz cp carbon-0.9.9.tar.gz carbon-0.9.9.tar.gz.orig cp graphite-web-0.9.9.tar.gz graphite-web-0.9.9.tar.gz.orig cp whisper-0.9.9.tar.gz whisper-0.9.9.tar.gz.orig {% endhighlight %}

{% highlight none %} cd ~/rpmbuild/SOURCES/ tar -zxvf whisper-0.9.9.tar.gz cd whisper-0.9.9 python setup.py bdist_rpm rpm -ivh dist/whisper-0.9.9-1.src.rpm cd .. cp whisper-0.9.9.tar.gz.orig whisper-0.9.9.tar.gz cd ../SPECS/ rpmbuild -ba whisper.spec sudo yum –nogpgcheck localinstall -y ../RPMS/noarch/whisper-0.9.9-1.noarch.rpm {% endhighlight %}

{% highlight none %} cd ~/rpmbuild/SOURCES/ tar -zxvf graphite-web-0.9.9.tar.gz cd graphite-web-0.9.9 python setup.py bdist_rpm rpm -ivh dist/graphite-web-0.9.9-1.src.rpm cd ../ cp graphite-web-0.9.9.tar.gz.orig graphite-web-0.9.9.tar.gz cd ../SPECS/ rpmbuild -ba graphite-web.spec sudo yum –nogpgcheck localinstall -y ../RPMS/noarch/graphite-web-0.9.9-1.noarch.rpm {% endhighlight %}

{% highlight none %} cd ~/rpmbuild/SOURCES/ tar -zxvf carbon-0.9.9.tar.gz cd carbon-0.9.9 python setup.py bdist_rpm rpm -ivh dist/carbon-0.9.9-1.src.rpm cd .. cp carbon-0.9.9.tar.gz.orig carbon-0.9.9.tar.gz cd ../SPECS/ rpmbuild -ba carbon.spec sudo yum –nogpgcheck localinstall -y ../RPMS/noarch/carbon-0.9.9-1.noarch.rpm {% endhighlight %}

Let’s continue with part two of the blog post series where we setup graphite.

Configuration #

It starts by installing some dependencies, but I have skipped that, because they are a partial repetition of what we have already installed. Then all RPMS will be installed, but we have already done that as well.

We continue with setting up carbon and graphite-web configuration files.

{% highlight none %} cd /opt/graphite/conf/ sudo cp graphite.wsgi.example graphite.wsgi sudo cp storage-schemas.conf.example storage-schemas.conf sudo cp carbon.conf.example carbon.conf cd ../webapp/graphite sudo cp local_settings.py.example local_settings.py {% endhighlight %}

{% highlight none %} sudo vim local_settings.py {% endhighlight %}

Where I have added/uncommented/edited the following lines:

{% highlight none %} TIME_ZONE = ‘Europe/Berlin’ MEMCACHE_HOSTS = [‘127.0.0.1:11211’] {% endhighlight %}

Let’s create the Django database with:

{% highlight none %} sudo python /opt/graphite/webapp/graphite/manage.py syncdb {% endhighlight %}

You will be asked to create a superuser. I have answered that with a ’no’.

Apache and wsgi is next. I already had those two, but if you want to install it do `sudo yum install -y httpd wsgi.

Next create the file /etc/httpd/conf.d/graphite.conf as descibed in the blog post.

Make /etc/httpd/conf.d/wsgi.conf look like this:

{% highlight none %} LoadModule wsgi_module modules/mod_wsgi.so WSGISocketPrefix /var/run/wsgi {% endhighlight %}

Now, set some rights:

{% highlight none %} chown -R apache:apache /opt/graphite/storage/ {% endhighlight %}

Almost done. Create a start/stop-script /etc/init.d/carbon-cache as described in the post and make it executable:

{% highlight none %} sudo chmod +x /etc/init.d/carbon-cache {% endhighlight %}

Start everything up as root:

{% highlight none %} service memcached start service carbon-cache start service httpd start {% endhighlight %}

And finally: Test if the webapp works by pointing your browser to http://localhost:9080. My ports are a bit different since I am using vagrant with port-forwarding.

Nope, did not work. In /var/log/httpd/graphite_error_log, I got the following:

{% highlight none %} IOError: [Errno 13] Permission denied: ‘/opt/graphite/storage/log/webapp/info.log’ {% endhighlight %}

The file was not there at all … Anyways, that had something to do with permissions. After some googling I ran into

{% highlight none %} chown -R apache:apache /opt/graphite/storage {% endhighlight %}

That did it! It worked! Strange, because I was sure I have already did that.

Feeding Data into Graphite #

Let’s get some data and visualize it. How this is done is explained in the last and third part of the excellent blog post.

Graphite is listening on ports 2003 and 2004 for incoming data. You can check that by running nmap -v -sT localhost. The format of the data is object value timestamp.

Am not repeating the example from the blog post here, but showing another example from the sensu community plugin:

{% highlight none %} $ gem install sensu-plugin –no-rdoc –no-ri $ cd /etc/sensu/plugins $ wget https://raw.github.com/sensu/sensu-community-plugins/master/plugins/system/cpu-metrics.rb $ chmod +x cpu-metrics.rb {% endhighlight %}

Plus, I had to add the following section to /etc/sensu/conf.d/client.json:

{% highlight none %} { … “checks”: { … “cpu_metrics”: { “type”: “metric”, “handlers”: [“graphite”], “command”: “/etc/sensu/plugins/cpu-metrics.rb –scheme stats.:::name:::”, “interval”: 60, “subscribers”: [ “webservers” ] } } } {% endhighlight %}

In Graphite, I was then selecting a few of those measurements to let them be visualized in the dashboard:

And there we finish this article, whereas the original blog post still goes on with more ways on how to gather data and feed it into graphite. Check it out!

Done for today.