For those of us running virtual infrastructures, this is a nice way to train on that Celerra platform and evaluate it for your needs. He has some HOWTO docs in there to help you get started. I saw the Celerra interface at CLARiiON days in North Carolina about a month ago and it’s really easy to use, reminding me of my ONTAP management days.
Our storage vendor EMC holds a very popular quarterly event called CLARiiON Days. Understandably, this event fills up very fast. I’m not sure what other locations host the event, but I was able to attend this event last week at EMC’s RTP North Carolina engineering facility, a sizable building of about 400 or so employees. None of them sales people!
The event began with a limo pickup at the airport to take you to the APEX manufacturing facility where the CLARiiON product is built. I missed all that fun when the stewardess did not show up for my 6:30am flight out of LaGuardia Airport in NY, causing Delta to search for another stewardess for 2 and a half hours. I finally got into Raleigh around 11:15 and made it to the RTP just before noon. I ended up missing the lunch at the APEX facility, so one of the engineers helping at the event was kind enough to pick up lunch for me at their really nice cafeteria – we should all be lucky to eat like them! I was also able to spend some time talking to a few of the engineers, something that you rarely get to do without a sales person around to throw buzz words.
Once everyone else arrived, the speaking sessions began. We had a little tour inside a lab showing us their CX and NS products in the flesh, demonstrating the redundant technologies their products support – the very reason why you pay so much for a CX4-960. There were a couple of discussions about drive technologies where they pushed the flash drives as their Tier 1 storage technology. We got some quick briefings on a few things that their new Unified Storage platform was pushing: thin LUN provisioning , dedupe, and policy-based file archiving (similar to DiskXtender). The evening was capped off with a limo ride back to the hotels to freshen up (my first chance to check in) followed by a nice dinner with the engineers.
On the second day we got into the replication technologies supported by their CLARiiON systems fronted by a Celerra head. The Celerra looks more like what I was used to managing in my NetApp days as it sports the same bilingual filesystem sharing to CIFS and NFS clients. Snapview, Snapsure, Mirrorview and Recoverpoint were all touched on as solutions to backing up LUNs and providing flexible LUN management. The product used to manage all of these replication systems is Replication Manager, an agent based system which is aware of various applications and knows how to quiesce them while allowing sessions to remain undisturbed. RM gives you fully synchronized LUN snapshots, which keeps in mind those times you have to restore from more than one LUN for the same moment – a key issue when dealing with databases.
The discussions eventually all came back around to VMWare, that glorious technology that lets us consolidate hardware, IO, and networking. Paired with EMC’s storage consolidation, the two are a great fit. Specifically, they hit the points on VMWare View (previously called Virtual Desktop Infrastructure) and Site Recovery Manager. After having run a DR setup project with BMR Restores synchronized with DoubleTake, I consider SRM to be one of the greatest technologies I’ve seen. Although there is the requirement that your Production and DR sites must both be virtualized, your replication and storage technologies are not limited to EMC. SRM will coordinate the entire failover and failback between Production and DR sites, including making sure VM snapshots take place and replication journaling continues during DR tests. SRM has a number of plugins for the Virtual Infrastructure Client to work with various systems like Lefthand networks. They were able to monitor LUN snapshotting and journaling using Replication Manager while they tested a demo DR site. This alone was worth the trip! For those of you who want to run a demo of the system, there is a great tutorial at http://viops.vmware.com/home/docs/DOC-1235 on how to do it on your desktop with VMWare Workstation. As I run VMWare Server 2.0 at home, I’m going to take a crack at it with that.
At the end, every one decided to leave the event right after lunch, as most of the attendees were from the North Carolina area. Since the event went for a couple more hours, I had the engineers to myself and got a great demo of the Celerra interface. They have some great management technologies surrounding that storage platform. I haven’t used one yet, but my opinion right now is that I like it better than the NetApp ONTAP interface I used for abour 8 years at previous companies. The Celerra system seems to do more than the NetApp systems do, and combining them with a CLARiiON makes it an even more powerful solution. Once 2pm hit, the limos took the rest of us back to the airport and our flights back. My return home was met with yet another delay when our plane had a “small hydraulic leak” but they quickly decided to switch us to another plane and we were quickly on our way.
If you ever have the opportunity, ask your sales rep for details on the next CLARiiON Days event and beg your management to let you make the trip. You really will be glad you did and it’s possible you will have some great new ideas to take back with you. There are some coming technologies like vCloud that will likely be able to take advantage of the special capabilities of EMC’s storage offerings. I hope to make it back next year.
macmini:~$ zdump -v /etc/localtime | grep 2009
/etc/localtime Sun Mar 8 06:59:59 2009 UTC = Sun Mar 8 01:59:59 2009 EST isdst=0
/etc/localtime Sun Mar 8 07:00:00 2009 UTC = Sun Mar 8 03:00:00 2009 EDT isdst=1
/etc/localtime Sun Nov 1 05:59:59 2009 UTC = Sun Nov 1 01:59:59 2009 EDT isdst=1
/etc/localtime Sun Nov 1 06:00:00 2009 UTC = Sun Nov 1 01:00:00 2009 EST isdst=0
For Windows users, this is a little more complicated. Windows’ most recent DST update came in December 2008, so it is critical that you are up to date on your systems’ patches. http://support.microsoft.com/gp/cp_dst
Once you’ve patched the OS, then you realize that the applications that ride on top of that OS must also be patched. Microsoft has a list of their applications on that page as well that require updates. Products like Oracle must be aware of the time changes or you risk potential data corruption. Third party software may also need to be DST aware and not freak out when 3:00:00 follows 1:59:59.
By now this is pretty old news. You have all your systems patched, right? Not always so. Due to regulations and paperwork, patching in some establishments is not as easy as in others. This means patch schedules run far behind the patch releases, or are never applied at all. This in turn makes for an exciting Sunday of adjusting clocks or an even more exciting Monday of restoring data. Plan accordingly!
For those returning to work this Monday after the holidays, after a lengthy time away from the office, please give a little extra time to thinking about those things that are normally routine jobs. Routine tasks are generally not given much thought and when you’ve been away from the office, you forget the little things. It’s easy to skip a step that you normally would have taken care of, leading to disaster if you’re not careful. Take an extra second before you hit ENTER to check your CLI arguments, or before you click OK to the dialog box in front of you. Verify that tape you pull out of the library, be sure that system is the one you were supposed to take a full back of, and make sure that VM is really the one you wanted to delete. It’s a little insurance that might pay off this week.
Sunday afternoon I began the process of adding a couple of 1TB drives to the Dell 2950 server I’m using as a VMWare Server at home. This system is pretty beefy at 2×3GHz dual core 2×2 Xeons with 16GB memory, but the pair of 70GB drives it sported were just not enough. I kept running out of room for VMs and had to delete VMs that I could potentially have been interoperating with others.
To accommodate the drives we had to purchase a pair of carriers to replace the dummy covers on the drive slots. First thing I noticed was that the carriers listed SAS and SATAII screw holes in the wrong positions. After a momentary chill down my spine, I screwed the drives in and shoved the drives into the server with a little bit of force and everything fit.
Booting the system up, I noticed that the firmware was at 1.1.0 and gave a date of 2005. This seemed odd for something that would need to work with these newer drives, but I gave it no thought. Until I saw that the drives were not being recognized. Bummer. After reading a few forums I saw that some people had seen this and updating the firmware would fix it. Luckily this server has another year of warranty left on it, so help was available.
I downloaded a BIOS update and the BMC update just in case. The BMC was also pretty far behind, so it was worth doing that as well. The server has no floppy drive. A fully decked out server, and no one got the $20 floppy drive. Well, in cases like this, I’ve used a BartPE disk with network support to mount a samba share off my file server. The first disk I found was a couple of years old and didn’t have the necessary drivers. So next I burned my own BartPE disk with the necessary Dell drivers and everything looked peachy. Until I tried to run the 16-bit BIOS and BMC updaters that PE doesn’t support. So I reached for another blank disk to make a generic DOS boot disk with the necessary updaters on it, when I noticed that I only had DVD-Rs left. The server has a CD drive, not a DVD drive. OK, so lets go get some CD-Rs. At 7pm on a Sunday, Staples has been closed for over an hour.
Things were conspiring against me, so I needed to take a new tack. I discovered a spare USB keyfob that I had bought as a 3-pack for my two older kids on a back-to-school sale. I found the HP USB Disk Storage utility and a Win98 boot files zip and quickly made a bootable flash drive loaded with the necessary files. The server sees the flash drive as a hard drive and hitting F11 allows you to select it from the Boot menu. In just a few minutes, I updated the server and could see the drives in the RAID controller. I got a hold of the MegaRAID Manager software for Linux and used it to configure the drives from my CentOS 5 installation. Right before bed, I started the fsck on the 1TB logical volume and when morning rolled around, I was greeted with a shiny new filesystem.
I think the lesson here is that if you are going to get a nice expensive system, spring for some of the cheap extras like the floppy or DVD drive. No, you might not use it very often, but at the time you critically need it, it will save you some precious time. At this moment I’m copying the VMs to the new partition so that I can free up that space on the main drive and start running more VMs. I’m pretty lucky to have this system on hand, as it helps me train on systems that I might not be able to work with in the office. It’s a great way to supplement one’s training, and if you think about it, not so expensive in the long run when you consider that you will be a better (and possibly better paid?) admin for it.
Recently Symantec made the announcement that due to some issues with Symantec Endpoint Protection, it would not support the VMotion of virtual machines. Shortly after a flurry of blogging activity surrounding this incredulous news, Symantec retracted this statement. While Symantec’s case is just the most recent and visible, they are not alone in misunderstanding how to handle virtualization.
Let me say that “no company today can afford to misunderstand system virtualization and not support it.” Virtualization is here. It is a reality of the datacenter. There are many who are behind the curve or resisting out of some misplaced fear, but have no doubt that it is here to stay and will only be more prevalent. Resistance is futile, especially in today’s economic climate where virtualization saves on hardware and licensing costs. In this case, Symantec comes across as confused, and we won’t soon forget how they stumbled over this issue.
The fact that Microsoft is only now getting around to considering the concepts of VMotion to the applause of the throngs of people that have just heard about Windows Server 2008, goes to prove that there are too many who have not yet taken advantage of virtualization and are only now getting around to learning about the first generation concepts. I consider VMWare to be 3 or 4 generations ahead of Microsoft with all the various layers that now ride on top of the basic VMotion concepts. When they say they will have some similar capability in 2010, they are admitting that they will be behind another 2 generations of virtualization whenever this finally comes around. No vCloud or DRS or VDI capability will yet be available unless some third party vendor is waiting in the wings to save them. In typical Microsoft fashion, they will most certainly encourage gullible third parties to fill this niche for them while Microsoft works to eventually roll it into their own product and drive those poor saps out of business. But the reality is that the better products don’t always win, and if Microsoft gets smart and pours real resources into their virtualization, they can really put the heat on VMWare. However, Microsoft’s beheamoth attitude is exactly what keeps them from being nimble enough to compete with Apple or VMWare.
VMWare has set up a new site with documented and proven best practices from both their internal use and their customers. The well organized site is designed to lead you directly to the information you need. Coupling this with VMTN community forums and VMWare’s whitepapers section, you have all the info you need to support your virtual infrastructure.
This of us who work in datacenters know that things change. Because racks may stay the same, the room may not look different, but inside the racks there are always things going on. Old systems are always being swapped out for new and smaller systems, cabling will change, a new system is added, or something will be pulled out for repair. That’s when the maddening hunt for wire cutters begins. One wrong move and you’ve cut through a USB cable, or worse, a power cable. And sometimes you may not even notice a little nick into the cable. Sure, some zip ties have a nice release clip, but often the ends are trimmed off to make the bundle neater, making them useless for expanding a wire bundle, or they are left in place for years and the plastic has warped to the point where the tie will not easily release. Yeeach!
Today I cleaned up a number of cables in a rack that had grown quite ratty. Zip ties promote the ratnests of cabling because they are a pain to remove and end up being a permanent fixture. Cables will grow around them, being zip tied to the old bundle and other things around them. As you can see above, a single cable produced numerous ex-zip tie clippings. Like some terribly fungal toenails, these gross pieces flick around while being trimmed.
Instead, please do yourself and the other admins around you a favor and velcro cables together. Some racks have nice velcro solutions that include a small clip to help support fragile ones like fibre optic cables or will simply keep the cables out of the way so that the machines can vent in the rack. It also makes for a more temporary solution that can be easily removed when cables need to be added or removed from the bundle. We’ll all thank you when we have to come work in your room one day. Thank you.
