The Daily WTF · ITAPPMONROBOTAt the turn of the 21st century, Initrode Global's server infrastructure began showing cracks. Anyone that had been in the server room could immediately tell that its growth had been organic. Rackmounted servers sat next to recommissioned workstations, with cables barely secured by cable ties. Clearly there had been some effort to clean things up a bit, but whoever put forth that effort gave up halfway through.
It wasn't pretty, but it worked for years. As time passed, though, a proprietary gateway server to communicate with credit processing agencies would crash more and more frequently. And these were bad crashes, too — the kind of crashes where the server wouldn't respond to ping and would have to be restarted manually. It wasn't really a big deal for the admin, Erik, to hit the restart button on the server when he was there, but that was only 40 hours a week. The credit union needed it to be active 24/7, but was unwilling to hire 24 hour staff in the datacenter. The problem kept getting worse and worse, so the IT manager called up a meeting.