mainesysadmin.com » de-duplication

Very Small shootout: DeDuplication (NetApp) vs. Compression (vRanger Pro)

Tom — Thu, 31 Jul 2008 13:50:56 +0000

The purpose of this shootout was to analyze already backed up images of VM guests sitting idle on disk. We like to keep 7 days online, so I was curious as to how the two compared in our environment.

Compressed vs Deduplicated

COMPRESSED: (backed up via vRanger Pro over VCB, size on win NTFS volume)
citrix_server = 4.29gb
win_sql_server_64bit = 9.76gb
citrix_server2 = 6.14gb
win2k3_member = 4.97gb

compressed vm total = 25.16gb

UNCOMPRESSED:

citrix_server = 10gb
win_sql_server_64bit = 20gb
citrix_server2 = 10gb
win2k3_member = 35gb

uncompressed total = 75gb

DEDUPED VS COMPRESSED:

deduped = 54.29% savings (NetApp cmd output below)
compressed = 66.45% savings

Here’s the actual output from the NetApp find_space exe:
A-SIS Deduplication Space Savings estimate.

Name of Fingerprint File: d:\esx_deduped.txt Total Number of Directories: 5 Total Number of Files: 58 Total Number of 4K Blocks: 19662390 Total Number of Duplicate Blocks: 10673928 Percentage of Duplicate Data: 54.29 Scan Start Time: Wed, Jul 30 2008 3:40:16 PM Scan End Time: Wed, Jul 30 2008 4:22:25 PM

I’ll be honest, I was expecting deduplication to blow compression out of the water even with only 4 vm’s. I’ll go out on a limb and venture that deduplication would provide more results as you toss more vm’s into the picture.

NetApp and Data Deduplication (artist formally known as A-SIS)

Tom — Wed, 30 Jul 2008 18:49:07 +0000

I’ve been doing a little testing with NetApp Deduplication lately but only in a limited test environment. I know one thing for certain, deduplication and compression do not mix at all! When you start to look at this technology make sure you’re dealing with uncompressed data. This posting is more or less a bookmark for:

“Scott’s Quick Guide to Setting up Netapp Deduplication”

NetApp does provide a useful windows exe called “find_space” which you can run in report mode against any windows data volume.

vmworld 2007, lots cpu cores, de-dupe what? and skinny ubuntu

Tom — Wed, 12 Sep 2007 15:58:08 +0000

Lots is going on this week at VMworld 2007. I watched a video at the AMD booth of a server being upgraded from dual proc dual core to dual proc quad cores featuring their new “Barcelona” F series cpu. The only change required is a bios upgrade which they actually replaced with a new physical bios chip. They stated in the video that no additional power, cooling or hardware is required for the upgrade. The upgrade made a significant change in system performance. FMI for quad core opterons…

I’m seeing a lot more companies adapting a de-duplication technology to their product. Netapp and Data Domain are a couple of vendors that I spoke to about this. Data Domain was interesting to me because they supply you with an entry level HW appliance with redundant disk storage at RAID 6. The cost is about $17k for their entry level appliance. The appliance comes with 2.5 raw tb of space which is the equivalent to 25tb of actual vm files because of the capabilities of de-duplication. Pretty incredible.

Ubuntu is visiting VMworld again. They’ve released a version of ubuntu that’s slimmed down for usage with virtual appliances and virtual machines. The product is called Jeos (pronounced juice).

Tonight is the VMworld 2007 Party…