/proc
A long time ago in another job far far away I was responsible for managing system administrators, and for ensuring the eternal availability of all network services. As is the wont of people trying to make sys admin jobs more efficient and tolerable, I wrote a number of automation scripts to collect information from servers and report it centrally. The actual sys admins in the department at the time had their hands full with their day to day work, so I dedicated some of my time building automation tools for them (at the expense of other things on my to do list, I’m sure). Now there are lots of projects you can borrow other people’s hard work to accomplish this, but back then there was less and I did it myself.
During that time I became way too familiar with the /proc directory on Redhat servers. I have since forgotten most of what I knew at the time, but this nice article by Federico Kereki on Linux.com brought quite a bit back for me. Everything you ever wanted to know and more about a host you can get from /proc.
More than the actual results, I remember going through source code figuring out what the information in the /proc files actually meant, and which information could be reliably used to provide which results. Its easy to misuse statistics, as much so on understanding system health as in any other arena.
Anyway, nothing productive to say here, other than the article is worth reading.