BiggiWiki

I wanted to see how much traffic my webserver got because I suddenly got a spike in web hits. I found a couple of scripts, but I generally disliked them, as they all used “cat” or “wc”, which is quite heavy when you have a couple of million lines. That would mean spending a lot of time looping the same log file. Instead we can tail it.

Here's some example data:

LOOPING! - Count: 61 - Last loop second was 1 seconds ago
LOOPING! - Count: 40 - Last loop second was 1 seconds ago
LOOPING! - Count: 44 - Last loop second was 1 seconds ago
LOOPING! - Count: 45 - Last loop second was 1 seconds ago
LOOPING! - Count: 45 - Last loop second was 1 seconds ago
LOOPING! - Count: 42 - Last loop second was 1 seconds ago
LOOPING! - Count: 51 - Last loop second was 1 seconds ago
LOOPING! - Count: 44 - Last loop second was 1 seconds ago
LOOPING! - Count: 44 - Last loop second was 1 seconds ago

The script used 1% CPU on my server. It's not a CPU or disk hogger, as I do not continuously read the entire file, I just tail it and count the changes. It's pretty basic for now, and as it does what I need, I'm not planning to expand it. If you have any changes you'd like me to write in here, feel free to mail me.

Usage example: ./readlinespersecond.sh /var/log/nginx/access.log

readlinespersecond.sh

#!/bin/bash
 
# TODO:
# Follow symlinks
 
# No filename specified?
if [ "$1" == "" ]; then
	echo "ERROR: Specify the file to read from.";
	exit 1
fi
 
# Does the file exist?
if [ ! -f $1 ]; then
	echo "ERROR: File not found";
	exit 1
fi
 
# Got ASCII?
if ! [[ "`file -b $1`" =~ ^ASCII\ text* ]]; then
	echo "ERROR: File is not ASCII text.";
	exit 1
fi
 
# Finally.. Let's get started.
CS=""
CSO=`date +%s`
# Second Counter
SC=0
tail -n0 -f $1 | while read line; do
# Get current timestamp
	CS=`date +%s`
# CS != CSO, show stats
	if [ "$CS" != "$CSO" ]; then
		LS=$((CS-CSO))
		echo "LOOPING! - Count: $SC - Last loop second was $LS seconds ago"
		SC=0
		CSO=$CS
	else
		SC=$(( SC + 1 ))
	fi
done