Sunday, June 1, 2014

Graphite on Nginx/uwsgi - Debian Wheezy

Graphite it an opensource application used to track metrics on a time series database. You can send metrics using almost any programming language and its way easier than other good options that may result a little more difficult to understand tools like OpenTSDB and RRDtool.

There are lots of tutorials and scripts that show how to install it,  most of them use apache to deploy the web interface. The graphite web app is written in Django and as such it's really easy to deploy using nginx.

Enough talking, let's get to work:

Install the required packages:
# Install the required packages for graphite
aptitude install -y python2.6 python-pip python-cairo python-django \
  python-django-tagging python-twisted python-memcache python-pysqlite2 \
  python-simplejson

# Install nginx and the required cgi modules
aptitude install -y nginx uwsgi-plugin-python uwsgi

# install graphite via PIP
pip install whisper carbon graphite-web
Create the configuration files:
cd /opt/graphite/conf/
# Create a directory with all configurations 
# It will allow us to keep this dir clean and use the original
# files as templates
mkdir examples; mv *.example examples

# create a copy of the required files
cp examples/storage-schemas.conf.example storage-schemas.conf
cp examples/storage-aggregation.conf.example storage-aggregation.conf
cp examples/carbon.conf.example carbon.conf
cp examples/graphite.wsgi.example wsgi.py
Create webapp configuration file
cp /opt/graphite/webapp/graphite/{local_settings.py.example,local_settings.py}
Uncomment the parameter "TIME_ZONE" on "/opt/graphite/webapp/graphite/local_settings.py" and replace with the result of "cat /etc/timezone". In my case the resulting edit looks like:
TIME_ZONE = 'America/Caracas'
Otherwise the webapp will assume the default time zone for Django apps: 'America/Chicago'.
Create nginx configuration file:
#/etc/nginx/sites-available/graphite
server {
  listen 8080;
  charset utf-8;
  access_log /var/log/nginx/graphite.access.log;
  error_log /var/log/nginx/graphite.error.log;

  location / {
  include uwsgi_params;
  uwsgi_pass 127.0.0.1:3031;
  }
}
Enable the virtualhost:
ln -s /etc/nginx/sites-available/graphite /etc/nginx/sites-enabled/
Create the uwsgi configuration file:
#/etc/uwsgi/apps-available/graphite.ini
[uwsgi]
processes = 2
socket = 127.0.0.1:3031
gid = www-data
uid = www-data
chdir = /opt/graphite/conf
module = wsgi:application
Final uwsgi application configurations:
# enable the uwsgi app 
ln -s /etc/uwsgi/apps-available/graphite.ini /etc/uwsgi/apps-enabled/

# create the local database for the webapp
python /opt/graphite/webapp/graphite/manage.py syncdb

# Allow access to  www-data to the webapp and storage  directories
chown -R www-data:www-data /opt/graphite/webapp/  /opt/graphite/storage/
Start carbon, the uwsgi and nginx daemons:
/opt/graphite/bin/carbon-cache.py start
/etc/init.d/uwsgi restart
/etc/init.d/nginx restart
Check http://yourIPAddress:8080 and you should see the graphite web interface.

graphite



If you have any issues, remember to check the following log files for possible errors:
/var/log/nginx/graphite.error.log
/var/log/uwsgi/app/graphite.log
As a last resource you can set the Django app into debug mode by setting "Debug = True" on "/opt/graphite/webapp/graphite/local_settings.py".It should provide enough information to give you a hint on what may be causing the problem.

10 comments:

  1. Thanks for the tutorial, however when I run this on a new machine, I get Python application not found, any thoughts.

    Thanks

    ReplyDelete
    Replies
    1. Hi Geoff,

      I'm glad that you are using the tutorial. Remember to verify that uwsgi started properly, so check the log file and verify if it's listening ("netstat -tulpn | grep 3031").

      Did you see anything on the log files? Please send me more information to help you out.

      Delete
  2. Got the same error here, resolved using the following graphite.ini:

    [uwsgi]
    processes = 2
    socket = 127.0.0.1:3031
    gid = www-data
    uid = www-data
    chdir = /opt/graphite/webapp/graphite
    pythonpath = /opt/graphite/webapp
    env = DJANGO_SETTINGS_MODULE=graphite.settings
    module = django.core.handlers.wsgi:WSGIHandler()


    Also you forgot to mention the database deployment step:

    cd /opt/graphite/webapp/graphite
    ./manage.py syncdb

    Thanks for the tutorial!

    ReplyDelete
    Replies
    1. Hello and thanks for the feedback. You were right, I completely forgot to mention the creation of the django database. I had just updated with your suggestion.

      I followed the updated steps on a new vm and I did not require the last three parameters that you suggested for "graphite.ini".

      Delete
  3. Great post, thanks.

    There's a bug with the JS on this blog. Whenever I copy-paste the example configuration, it tries to append " - See more at: http://alfredocambera.blogspot.co.uk/2014/06/graphite-on-nginxuwsgi-debian-wheezy.html#sthash.bwSCpYOf.dpuf". However, it strips all the newlines! It makes it very hard to use your configuration examples.

    ReplyDelete
    Replies
    1. Hello Wilfred,

      I'm really sorry about the issue. Currently I'm using a very old JS library (https://code.google.com/p/google-code-prettify/) to do the syntax highlight. During the next week I'll be updating all my blog posts to make use of http://prismjs.com/. It looks pretty cool and has support for modern browsers.

      Thanks for your suggestion.

      Delete
  4. Also, you've written apps-enable rather than apps-enabled for the uwsgi symlink.

    ReplyDelete
    Replies
    1. Thanks for noticing and reporting the mistake. I appreciate your feedback.

      Delete
  5. "mkdir examples; move *.example examples"
    could be :
    "mkdir examples; mv *.example examples"

    And i've got the same error too : Python application not found

    ReplyDelete
    Replies
    1. Hello Nicolas,

      You are completely right, blog post updated :D. I think all you need to do is create the webapp database using "python /opt/graphite/webapp/graphite/manage.py syncdb" @cafe1 suggested. And please make sure to do a second check on all the "chown".

      Please, let me know if that solves your problem.

      Delete