Clean it up, clean it up

It’s been a crazy last few weeks but the server move has gone off fairly well. I managed to move all of the uclick servers in one weekend from our old facility to the new one. In the mix I managed to change internal IPs of all of our servers which at current count sits at about 50. I had a lot of help from Samuel and Doug in the recoding department and from what I count in subversion we’ve got a bit over 200,000 lines of code in perl modules and a little more than 40,000 in command line programs, my best guess is that’s about half the code we actually have on the servers. To get all that fixed up in less than a week is an accomplishment I think we should be proud of and I’m happy with myself as I’ve been the “official” Sysadmin for a couple of months now. The week before Memorial Day, Memorial Day weekend and a couple days after I didn’t really exist for anything other than work so it was extremely nice of Kate to put up with my work induced absence during those days. I’ve got a couple of photos of the old facility, the first is from the days I went down and pulled all the boxes of items that had been stashed inside of cabinets. I’d say about ninety percent of it was trash that went directly into the dumpster.

There’s various reasons it got to the point that it is and my goal though is to keep the new facility in showable condition. In order to do that I can’t have the network closet look like this

You can’t see it but that’s an Extreme 8806 behind the network wiring and it’s a shame to have it that way, it’s a very nice piece of hardware. These racks were crap really and there wasn’t room mount things to the side, not much additional depth beyond what was needed to get the server into the rack, the mounting posts were fixed and C channels just to add an extra piece of complexity. Our new server racks aren’t the ultimate design but they are definitely a step up from what we had before. Next week I’ll get some pictures of the new racks and the layout I’ve got going there. My networking cable isn’t exactly where I want it to be at in that process yet but I’ve got some items I’ll be getting from Panduit to help resolve that situation. I’ve learned more about Pix firewalls and VPNs in the last week than I will probably ever use in my life again and while I had a fairly good set of cisco configs to work from I had to figure out how two rework two new switches, upgrade and downgrade the OS on a Pix 525 as well as learning how to establish and troubleshoot a VPN tunnel on a 515. Pretty good for a weekends work. Now we’ll be in four racks instead of six but by going from DL380 G1 (3Us) to DL360 G5s the amount of space being saved is tremendous. The new facility still has a few of those old G1 (and 1850) beasts running but they should be on the short timers list and migrated over to newer hardware soon. I would say the new facility is overbuilt, but it’s cheaper to over build a little bit now and take the hit to buy products that allow for expansion at a slightly higher cost than have to worry about how to do it later at higher cost and the possible addition of downtime when we start to notice problems cropping up, that’s my theory anyway.