Organizations that measure their Personally Identifiable Information (PII) by the petabyte have come to accept that seemingly extreme measures to protect storage media are simply a necessary cost of doing business.
The more forward thinking among them, however, understand that there are ways not only to mitigate those costs but to enhance accountability at the same time.
Take the large banking and financial institution with more than 20 petabytes of data in disk storage spread across its data centers. The data centers are classified into three tiers, with tier 1 and tier 2 having the highest security and resiliency requirements. Tier 1 manages the most mission-critical applications (core applications software running the entire bank).
Tier 2 hosts the less mission-critical operations, including testing and development. Tier 3 manages legacy data and applications. The data centers use SAN storage technology (with a variety of Linux and Wintel servers) and multiple hardware vendors, including Hitachi, IBM and EMC.
Having endured the theft of backup tapes containing 1.2 million customer records and the investigations, lawsuits and brand damage that ensued, they decided to reevaluate their data security practices. That led to a fundamental decision that failed disk drives and other media could not leave their secure premises with accessible data on them, no matter what. A team was formed to study the possible options.
Sometimes painful experience is the best teacher
One such possibility was to implement disk encryption. Such an approach imposes server, storage, application performance, and key management overheads. Plus, there is tremendous added complexity associated with managing encryption keys. The firm evaluated current disk encryption solutions and concluded that it was unable to find an efficient cost model that also addressed its performance requirements. Moreover, encryption is not an accepted standard or methodology for data sanitization.
Another, simpler, option would be to keep and physically destroy the failed disks instead of returning them under warranty. The replacement cost of disk drives (hard disk drives, or HDDs, and solid state disks, or SSDs) can cost between $300 and $7,000. When a drive fails, the option would be to retain it and receive a large bill for its replacement cost. These potential costs continue to escalate as higher capacity drives, solid state drives and flash module drives come into wider use. The replacement cost of one Hitachi Accelerated Flash (HAF) Flash Module Drive (FMD) can easily exceed $10,000, depending on generation and capacity.
Cost of physical disk destruction
The bank recognized that the continued explosion of data would result in higher incidences of failure in its spinning disks, and newer media, and that keeping failed disks locked up in a secure vault within its data centers was not a financially sustainable strategy. In fact, it would cost in the millions of dollars per year.
The bank also wanted to ensure that the disk drives are destroyed in compliance with standards such as the U.S. Department of Defense (DoD) National Industrial Security Program Operating Manual (NISPOM), NISPOM 8-306, DoD 5220.22-M, and DoD 5220.22-M ECE.
They considered degaussing as an approach that would enable them to return the failed disk drives to the vendor. They concluded that this option destroyed the header record and made it very difficult, if not impossible, for the manufacturer to fix the drive, thus voiding the warranty agreement.
As a result of the tape data breach and the adoption of the various state data breach notification laws, the bank’s security and compliance teams concluded that they did not want media leaving the control of the bank via courier. Initially, this policy applied to tape media, but the definition was soon expanded to include all types of media, including disk drives. The internal audit teams identified failed disk drives shipped through disk vendors’ courier shipping solutions as outside the bank’s controls. As a result, a business case was developed and funded to come up with a strategy to mitigate that risk.
Building a business case
The evaluation team concluded that the process of destroying data in failed disks was too meticulous, involving multiple passes and requiring very distinct technical skills. The evaluation team decided that an on-staff/on-premise investment would not be a cost-effective use of the bank’s IT operations dollars and chose to go with Revert’s disk eradication service.
Together, the bank and Revert instituted the following actions:
In the bank’s opinion, the service enabled it to mitigate the risk from the failed media in a cost-effective manner with minimal disruption to its application performance and resiliency requirements. With both mainframe and SAN environments, it recognized that owning the equipment and staffing the meticulous process internally would not optimize its IT operations budget.
- Formalized the business process and workflow for the alert, documentation, report, and remediation of a failed disk event.
- Developed a secure Microsoft SharePoint application to document the process and communications between the bank and Revert. The corporation is also using the Microsoft SharePoint application to centralize the supporting data (standards-compliant data destruction certificates) for publishing to internal and external audit.
- Provided workspaces within the bank’s tier 1 and 2 data centers for the disk eradication process.
Turning to an expert
Revert’s on-site disk eradication service supports mainframe or open systems; RAID or JBOD; enterprise, midrange, and desktop and laptop storage from multiple vendors; and Fibre Channel, SCSI, FATA, SATA, and ATA/IDE disk drives of any capacity. The service’s ability to meet applicable standards, including the U.S. Department of Defense Data Sanitization Standards (DoD 5220.22-M, DoD 5220.22-M ECE), also appealed to the bank.
In time, the relationship broadened to encompass eradication of storage systems on the raised floor, systems that were being replaced by newer technology and/or being repurposed. In such cases, Revert is able to directly connect to storage shelves – bypassing the control unit – to eradicate hundreds of drives at a time.
The bank’s experience yielded the following lessons:
Corporations with international operations are advised to adopt data privacy practices that meet the local requirements. Data privacy should be viewed as a critical component of a firm’s information management practice. This integrated approach looks at the disposition, retention, sharing, and use of business records throughout their life cycle, from their creation and capture to events requiring their destruction.
Key lessons
- Identify the relationships across your firm’s data retention policies, the technical process and underlying technologies, and how they map to the firm’s IT security practice and IT infrastructure.
- Understand “what” and “where” your critical assets are located, taking into account that specific content can reside in multiple applications and across different forms of storage media. Audit the storage media, applications, and computing systems that may contain personal information.
- Understand your firm’s information, storage, and security architectures. Identify the potential process and technical gaps. In the bank’s case, what started out as a plan to encrypt tapes quickly pointed to the risks arising from sending out failed disks for exchange or repair.
- Identify the options available. Outside of the acquisition cost of the solution/technology, be clear on your firm’s application performance, resiliency and manpower investment requirements. Also, be cognizant of the trade-offs across these “needs” as your firm makes the purchasing decision.
- Formalize and document your firm’s practices for handling the disposition of data in failed and decommissioned media. If possible, automate the workflow from the time a trouble ticket is issued for the storage media, the quarantine and secure physical storage of the storage media, the alerts sent to the disk eradication service provider, and the certification and sign-offs involved in the actual destruction of the data. Designate an individual to centrally manage the documentation of the workflows, as well as the communications and relationships with service providers such as Revert.
- Providing the reports on and documenting the certificates of eradication by disk drive serial number are valuable during audits. Data privacy is a critical component of IT operational risk, which in turn is a key lever in measuring a corporation’s enterprise risk management rating.
- Adopting consistent standard practices for enforcing data security policies in failed and decommissioned media across data centers and across vendors also provides a legally defensible argument during legal disputes.