I installed syslog-ng and set it to log to a txt file locally. I then ran loggen to test how many messages per second it could process. I was reading about performance tweaks I could do to increase performance, but I wanted to get a base-line number first to see if I was actually benefiting from these adjustments.
Now initially, I hit a limit of about 8600msg/sec which was pretty low I thought. None of the tweaks made this number any higher either. So I decided to write the logs locally, instead of sending them over the network and I got the same 8600/sec. So I started sending logs to /dev/null thinking maybe the hard drive was limiting me, but still 8600/sec. Loggen says my bandwidth is only 2250.58 kB/sec, so I feel like I have plenty of overhead, and my cpu barely hits 30% load.
I thought maybe the laptop was too old so I moved to my i5 desktop, and just barely hit 9000/sec. So to make sure I wasn't crazy I logged onto the mail server, and set loggen to test with a rate of 1,000,000/sec. The test spiked at 212,000msg/sec before I cancelled it.
So what I don't understand is how from a clean install of syslog-ng my machine logging locally (or /dev/null) can't do more than 9,000/sec but the server sending data over the network is more than 20x as fast.
I don't know much about syslog-ng so it's entirely possible I've just done something wrong but here's my syslog-ng.conf settings:
source s_test { udp(ip(127.0.0.1) port(5002)); };
destination d_test { file("/home/test.txt"); };
log { source(s_test); destination(d_test); };
and my loggen command:
/usr/bin/loggen -iD -r 20000 192.168.1.34 5002
results:
average rate = 9031.86 msg/sec, count=90320, time=10.001, (last) msg size=256, bandwidth=2257.97 kB/sec
[link] [23 comments]