Here are 5 data archiving best practices for better data storage management…

1. Leverage data storage management tools to improve budgeting and long-term planning. Establishing a baseline of your current data storage management environment is a good first step. By identifying and classifying various data pools inside your organization, you’ll have a better understanding of your archiving requirements. Storage resource management (SRM) tools will help you get a more accurate snapshot of your infrastructure and establish trends and statistics to identify data usage patterns.

2. Implement an email archiving strategy to save on backup costs and avoid legal trouble. Your company can save as much as 50 percent on backup costs with an archiving strategy in place. Although you may face legal requirements for email storage, storing email on your corporate server affects its performance over time and creates hassles in the event of a server disaster.

3. Implement Information Lifecycle Management (ILM) to help prioritize and store data according to its value. I could write an entire blog on this (and I probably will), but ILM will help you determine what data is most important, which will help you design a roadmap for where, and on what media, your data should be stored.

4. Ensure your data has the proper privacy controls to protect both the data and your privacy. We’ve all seen the headlines of one data breach after another. Most of them are avoidable, mainly because data breaches caused by human error. Laying out a process for handling data and encryption keys—including the creation, distribution, deployment storage, transmission and destruction of the keys—is an important best practice.

5. Establish a data retention policy that spells out how to categorize data and the retention period for each category. Set minimum and maximum retention periods for each category and address the disposal process. Educate employees on your policy and enforce it.

These data archiving tips and best practices can help you establish a data storage management program that can help you effectively trim down your essential data and the costs associated with the archiving of that data.

Even if you didn’t know anything about your data, would you know the best way to classify it?

Your best bet? Build an archive: This will quickly identify your “working set of data.” By using a file virtualization solution across the storage infrastructure, we’ve seen up to 50% cost reductions in disk and maintenance of a backup infrastructure AND a reduction in backup windows. (In one case we saw it decrease from 14 hours to 3 hours.) That means disaster recovery is vastly simpler and a great deal faster!

We’ve deployed other solutions, like Synthetic Fulls, to accomplish similar results. Backups have greatly reduced I/O loads on clients and improved the full server recovery process. Synthetic Fulls creates new backup sets offline without touching the source data or communicating with the original host server. This improves the ability to create new full backup tapes for other uses such as vaulting or setting up a new site or test system. Replicate and replay.

We’ve been working on this concept with a lot of our clients lately. Why not just move files that are candidates for being backed up to a separate tier of storage, keeping them as files in their native format, and organizing them in time coherent views? Originally it was because of cost, but not anymore. Solutions now cost the same as LTO4 media!

Users can restore files themselves from any point in time using a search engine. You don’t need backup software to do this, which means it’s simpler to deduplicate and compress, apply compliance and regulatory rules to it as policies, and use it as an archive. Indexing is easier, plus data mining and replication or other data moving requirements are more simply met. Copy-based backup is now a default paradigm for consumers, i.e. Apple’s Time Machine and EMC’s Mozy.

If you’re looking into this or other data protection or backup solutions, check out the Active Archive resources from our partner, Spectra Logic.

In the simplest terms, data created by humans—documents, transactions, and email for example—dedupes well in most dedupe systems. Photos, audio, video, imaging, or data created by computers generally don’t dedupe well, so you should store these sets of data on non-deduped storage. Learn what data does not dedupe well in your particular environment, and consider not deduping it. For some situations, you might consider a deduplication solution that can selectively avoid certain sets of data.

The length of time that data is retained affects data deduplication ratios in two ways: If more data is examined when deduplicating new data, you’re more likely to find duplicate data and increase space savings.

While you should closely examine this number when you’re comparing multiple products, try not to overanalyze this number once your system is up and running. Rather than performing more frequent full backups just to get a better data deduplication ratio, consider increasing your backup retention period for your on-disk data store. Once you have your first set of backups on disk, adding additional backups to that same deduped system will take up less space than sending them to tape.

Three down, two more to go… come back soon to see our final two data deduplication best practices. And as always, we value your feedback and ideas on how you deal with data deduplication.

Data retention policies are a critical component of data archiving. Once you have a data retention policy, enforce it for all the information you have on your network. You should also record the retention periods, both for distribution to users and as part of the legal defensibility record. The retention schedule does not need to be elaborate, but it should include specifics about the various categories and associated retention periods. When creating a written schedule, organizations should focus on two main criteria: the maximum retention periods and data distribution.

As part of a data retention policy, you should set minimum retention periods and maximum retention periods, in order to avoid over-retention. You can either include specific maximum retention periods for each category, or you can include a general clause that when the minimum retention period expires, the company no longer retains the data. This maximum retention period should also address the disposal of any currently retained data, including information archived under an interim infinite retention period.

Once you establish your archival retention periods, distribute the information to all users, both to communicate policy and to address the storage habits of individual users. In cases where individual employees use a variety of different email programs or in cases where it is common for users to archive e-mail messages, the retention schedule should make clear that this is not “retention” for purposes of the organization.

Organizations can meet legal preservation obligations with interim archiving, regardless of whether a complete retention policy is in place. Litigation requires that a company be immediately able to preserve and retain information (also known as instituting a legal hold) as soon as it reasonably anticipates litigation or governmental action.

Simply moving information into an archive does not constitute effective information management. In order to gain the highest benefit from archiving, companies should categorize all archived information, setting retention periods for each category. Well established retention periods with automated, effective, and easy-to-use archiving systems create streamlined, customized, and uncomplicated data preservation systems.

We hope you’ve enjoyed our series on data archiving and have found it informative. Let’s recap below the 5 data archiving best practices we’ve discussed over the last 5 posts.

Data protection and privacy continue to be a tremendous focus and risk for IT communities. While companies make great strides protecting data privacy in production application environments, they often overlook implementing similar strategies in non-production environments such as testing, development, and training.

Reliable, safe, and effective mechanisms for securing data at rest require the adoption and rigorous execution of well-defined processes for handling keys used to encrypt data and keys used to safeguard the data encryption keys. Key management is a comprehensive term that covers these controls—including the creation, distribution, deployment storage, transmission, and destruction of keys used to encipher data.

We’d love to hear what your doing to ensure the security and privacy of your data, in production or non-production environments. Leave a comment or send me an email!

In our next, and last, blog in this 5 part series on data archiving we’ll discuss establishing a data retention policy.

When you implement information lifecycle management (ILM), you effectively store your information in a manner consistent with the value of the data. All data maintained on storage networks has a defined lifecycle. This lifecycle identifies the way information travels through an organization from its creation to its archival and removal. The exact steps in data lifecycles largely depend on organizational policy, though data generally travels through three stages:

1. Stage 1: Creation/Acquisition of Data. During the creation of data, both data availability and data value are extremely high.
2. Stage 2: Publication. The value and availability requirements of published data, whether printed or accessed through other means, often depend on the content of that data.
3. Stage 3: Retention and Data Disposal. The length of time an organization archives and retains information depends on the nature of the data. However, increasing federal regulations, standards, and compliance measures often govern how long organizations must keep certain types of data.

The changing importance of data and the requirement for data availability create problems—it is costly to store all information on expensive, high-availability Tier 1storage systems. At some point, organizations must shift at least portions of corporate data to less expensive storage media. However, that merely raises the questions, “What data do we move to cheaper media? And when do we move it?”

ILM provides a strategy for data management throughout the information lifecycle. It identifies the processes and technologies that determine how data flows through an environment. Information path management is another consideration with ILM organizations are unlikely to offload all rarely used data to cheaper storage unless they can still access the data reliably, if needed.

With an effective email archiving strategy, companies can often save as much as 50% on backup costs. The average employee sends and receives more than 140 emails each day, according to a recent Osterman Research report, and according to courts and regulators, emails are official business documents. Motivated in part by the Federal Rules of Civil Procedures, as well as case law emerging around email discovery, companies now have new and increased responsibilities to control email in response to potential litigation.

In order to address this issue, organizations often impose strict mailbox quotas, limiting the amount of email any single user can save. Unfortunately, limiting email storage only encourages users to store email messages on their own desktop or laptop in PST files. This strategy presents complications if companies need to provide litigation holds on this data. Additionally, storing email messages on the corporate server negatively affects its performance over time, so it takes longer to recover in the event of a disaster, and is more likely to need additional capacity and other upgrades to accommodate the increased load.

Be sure to stay tuned for the next installment in this 5 part series on Data Archiving. Our next post will discuss Information Lifecycle Management.

Archiving is largely about control—knowing precisely what data you have, where you have it and having it at the right place, at the right time. Over the next few weeks we’ll explore 5 best practices and tips to trim down your essentials and archive your rarely used content.

When you’re designing and implementing a data archiving solution, a good first step is to establish a data management baseline of your current environment. By identifying and classifying the various data pools within your organization, you can better understand your archiving requirements. Storage resource management (SRM) tools present the best approach for capturing this baseline. SRM tools can greatly improve budgeting and long-term planning, which in turn can offer significant ROI.

By implementing and using a storage resource management (SRM) solution, you’ll have a more accurate perspective of your storage infrastructure. SRM gives you accurate proactive capacity planning and more accurate growth forecasting. And with the storage performance trending and near real-time statistics, you have much better insight into your day-to-day data usage patterns. This provides early warning signs to potential performance and capacity-related problems and downtime.

In our next data archiving post, 2 of 5, we’ll be exploring a data archiving strategy for email. Be sure to chime in with your thoughts and come back soon for the next post.