Throttling verbose output
Most developers know how to follow a log file while it is being written:
tail -f log/debug.log
However I’m currently working on some code to process millions of records, and my log output is quite verbose. So during testing, it is useful for me to only see every Nth line of log output, just to get a sense of the progress:
tail -f log/debug.log | awk '!(NR % 100)'
This shows me every 100th line. Sample output:
... D, [2014-12-02T16:04:18.977359 #25053] DEBUG -- : Processing record 1611045 D, [2014-12-02T16:04:20.235950 #25053] DEBUG -- : Processing record 1611303 D, [2014-12-02T16:04:21.587648 #25053] DEBUG -- : Processing record 1611581 D, [2014-12-02T16:04:22.822479 #25053] DEBUG -- : Processing record 1611777 ...
Typing out that
awk command is annoying, and I’ll probably forget it anyway.
So it makes sense to define an alias:
alias throttle="awk '!(NR % 100)'"
So now throttling my output is as simple as:
tail -f log/debug.log | throttle