Syslog-ng to graph

I just had this idea yesterday and I thought I'd share in case anybody finds it useful. Let's say you have a legacy application and you want to graph something about it but it has no snmp capabilities. If it uses syslog then there's another way...

Syslog-ng

Syslog-ng has built-in parsing and can also output to mysql instead of (or as well as) text files.

So let's say you've got an application that sends sms messages via an sms gateway and logs something like this:

Mar 11 14:33:46 pigeon sms_send[22023]: Sent SMS from 'some@account.com' to 'xxxx' (101 bytes, 2.294 seconds) 

and you'd like to produce a graph of sms messages sent per time unit.

First you define your filter...

filter f_sms { host("^pigeon") and program("sms_send"); }; 

your parser...

parser p_sms { csv-parser(columns("sms.1", "sms.2", "sms.3", "sms.user", "sms.5", "sms.phone", "sms.size", "sms.duration") delimiters(" ") flags(escape-none,strip-whitespace,drop-invalid,greedy) template("${MSG}")); }; 

your destination...

destination d_sql_sms { sql(type(mysql) host("127.0.0.1") username("$myuser") password("$mypass") database("syslog") table("sms") columns("id int UNSIGNED auto_increment primary key","username varchar(100)", "datetime timestamp") values('','${sms.user}', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC') indexes("datetime","username")); }; 

and finally put it all together

log { source(s_network); filter(f_sms); parser(p_sms); destination(d_sql_sms); }; 

Graph

Now you've got a table in mysql (syslog-ng will create the table if it doesn't exist and the user has enough rights) that will contain all the info you want. Since the id of the table is an unsigned int, in effect it behaves as a 32-bit counter. You can then simply write a simple script that will fetch that number and feed it to your graphing program of choice (I'm using pnp4nagios along with -obviously- nagios)

root # cat check_sms_send.sh #!/bin/bash sms=$(mysql --defaults-file=/usr/local/nagios3/libexec/sms_send_mysql_opts -h syslog-host -N -B -e "select max(id) from sms" syslog) echo "OK|sms=${sms}c;;;;" exit 0 

And here you go! graph

Of course there are other ways to do it but for me it was the easiest since all the software was already in place and there was no need to install anything.