Do you talk in your sleep?
It’s been a busy week and about 5pm tonight, we were finishing off bouncing some ideas round and contemplating going to the welcome party for new students. Then the new fileserver dropped offline. Completely.
These new file-servers are whizzy, expensive boxes with lots of storage. They’ve been rock solid through all the testing we’ve been able to throw at them. To discover that the live server had become un-contactable was disturbing to say the least. We investigated. And investigated. We would restart one, walk to the other server room to restart the other and find the first had gone off-line before we got to the second. We googled. We read manual pages. Everything pointed to something on the network causing these machines to lose track of what was happening.
We monitored the network – lots of data on the network which didn’t conform to any known standard. It came in bursts of about fifteen or twenty seconds of junk then a pause for a while. We started disconnecting parts of the network to track down the source, one of us frantically typing commands with the other monitoring the flow of junk.
After a couple of false starts, we found the right area of the building and went in search of this errant machine. Perhaps something had been taken over by a particularly nasty virus? Or some experiment control system had been infected by a trojan even now attempting to disrupt nuclear centrifuges. Or an old machine was emitting rubbish as the dying gasp of an aged network card.
What we weren’t expecting was a nice, new, shiny PC in sleep mode. Sending a stream of utter junk across the network. It’s unplugged now and the network is much quieter. Our fileservers are happy once more.
So if you were at the welcome party, sorry for not joining you. We’ll catch up soon, I hope. And in the meantime, don’t encourage sleep-talking.