While much of the hype around dedupe has been focused on data deduplication for secondary disk storage, the technology is rapidly evolving and being applied at earlier points in the data path—upstream from backup storage systems.  Several backup software vendors, including EMC Avamar, have been differentiating their solutions with source-side deduplication.  Deduplication and compression are being applied to primary storage by EMC , NetApp, StorWize and Ocarina to reduce the footprint of primary data. Now Permabit has launched data reduction software that primary storage systems vendors can embed—potentially rivaling other vendors’ position here.

It’s definitely been interesting to watch capacity optimization technology (deduplication and compression) evolve.  The most notable trend is the move upstream.  Just last month, EMC announced its “Boost” option for Data Domain, which enabled Data Domain to move a portion of its deduplication processing up to the media server in NetBackup OST environments—thereby distributing the workload and gaining some network and performance efficiency in the process.  In a similar move, Symantec extended its deduplication processing to client systems (from the media server) to gain end-to-end efficiency in the backup process.

These shifts shouldn’t come as a surprise.  Vendors (and end-users) realize that there are big benefits to “stopping the avalanche at the top of the mountain.”  They’re also probably drooling over NetApp’s success with its storage efficiency efforts—headlined with NetApp deduplication in FAS arrays.  Just last month, NetApp posted seriously good financial results; its focus on server virtualization initiatives, where redundancy and proliferation create an optimal use case for deduplication, is probably a contributor to these results.

Permabit, long a provider of deduplication technology through its archive solution, has packaged its technology up for wide consumption.  Offering its Albireo High Performance Data Optimization Software via an SDK allows the primary dedupe “have nots” to embed Permabit’s time-tested deduplication technology into their primary storage offerings.

The objection every vendor moving the process upstream has to manage will be performance vs. cost.  Implementing deduplication to save bandwidth and capacity makes sense … but not if the added processing impacts application performance.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Tags: Avamar, backup, Data Domain, deduplication, EMC, NetApp, Ocarina, Permabit, primary storage, secondary storage, StorWize, Symantec, virtualization