big data – big archives big protection challenges€¦ · an information explosion ... implements...
TRANSCRIPT
Linear Tape-Open, LTO, LTO Logo, Ultrium and Ultrium Logo are registered trademarks of HP, IBM and Quantum in the US and other countries. Other symbols may be trademarks of
other companies. Linear Tape File System is a trademark of the IBM Corp.
Big Data – Big ArchivesBig Protection Challenges
LTO Ultrium 5 Technologyand
Linear Tape File System
Bruce MasterSr. Program ManagerIBM Corporation
2
Agenda
� Storage Issues
� Best Practices – Unveiling the Facts– Addressing performance, data protection and
retention objectives
� Overview - LTO-5 Technology
� Introducing Linear Tape File System (LTFS)
3
Storage Issues……O bjectives
� Data is growing faster than I can manage
� Not all data is alike
� Costs are out of control
� Data is vulnerable to corruption threats
� My backup repository is growing
� Must protect and preserve data assets now and for the future
4
� 2.5 billion RFID tags sold in 2009
� 900 million GPS devices sold annually by 2013
� 76 million smart electric meters in 2009. 200M by 2014
� Text messages generate 400TB of data daily in the U.S.
� MRIs will generate a petabyte of data globally this year
Information is beaming in from everywhere!
BIG DATA !
High Volume !
5
2000 2005 2010 2015
Terabytes
Petabytes
Exabytes
Zettabytes
Gigabytes
� Storage budgets up 1%-5% in 2010
The information explosionmeets budget reality
� Storage requirements growing 20-40% per year
� Backup and Archive requirements growing 40-50% per year
An Information Explosion - Meets Budget Reality
6
Storage Hierarchy Balanced Diet
DRAM CacheSolid-State Drives (SSD)
Phase Change Memory
FC and SAS disksSAN
Tape
SATA disksVirtual Tape
NAS/iSCSI
Build out automated tiered storage
architecture to optimize
performance, protect data and reduce
costs
7
Rec
over
y T
ime
Obj
ectiv
e (g
uide
lines
onl
y)
Sec
onds
-M
inut
esM
inut
es -
Hou
rs.
Hou
rs -
Day
s
Mission CriticalDynamic Data
Active Online Data
Nearline-Arcchive Data
Not All Data Should Be Treated the Same Way
Disk and DB Mirroring
Electronic Vaulting / Replication
Tape Storage
Value of Data / Financial Investment
Time Value of Data Determines RTOAnd Storage Hierarchy Tiers
8
Most Network Data Sits Untouched
90% Never Accessed
Accessed <5 TimesAccessed >5 Times
Accessed Once
90% Never AccessedAccessed OnceAccessed <5 TimesAccessed >5 Times
Source: Government Computer News, July 1, 2008. “Most network data sits untouched” by Joab Jackson
�Three month study of a businesses 22TB disk data access
�Conducted by University of California, Santa Cruz
�90% of the data was COLD - never accessed after being stored on disk
�Another 6.5% of the data was COOL - accessed only once
�U of C recommendation: move data to less expensive and less energy consuming storage units ….. Use Tape!
Access Patterns of Data Stored on Disk.
9
Numerous Threats Can Corrupt and Destroy Data
ACCIDENTAL•Natural Disaster•System Error•Operation Error
INTENTIONAL•Virus •Theft•Hacker •Sabotage•Disgruntled employee
Risks of downtimeLost revenue and market shareLost productivityNon-compliance Loss of reputation and customer trustLoss of the business
• More than a quarter of the companies in a Forrester Research study declared a disaster in the past five years 1
• 76% of companies have experienced a disaster or major business disruptions 1
1 “Building the Business Case for Disaster Recovery Spending,” Forrester Research, April 2008
10
May 2009: Hackers 'destroy' both
production servers and there was no offline backup
Hard Lessons Learned
Jan 2010: University computer infected with a
virus may have exposed the personal information of 3,500
Feb 2009: Hackers broke into government admin
computers accessing 48K names and social security numbers
Mar 2009: Large pharmacy chain pays
$2.25M in fines for improper disposal of
patient data
Feb 2008: Bank lost unencrypted back-
up tapes with sensitive data about 12
million customers
2011: Software error corrupts many records
on primary and replicated disk.
Offline tape storage recovers records.
May 2009: Hackers 'destroy' both production servers and there was no offline backup
2011: Software error corrupts
many records on primary and
replicated storage. Offline tape
storage recovers records.
11
Tape Saves the Day…Provides Offline Protection
Ben Treynor, VP Engineering and Site Reliability Czar for Google Gmail, used the official Gmail blog to explain the situation where some users lost access to their email accounts during a software update that was buggy.
“I know what some of you are thinking: how could this happen if we have multiple copies of your data, in multiple data centers...well, in some rare instancessoftware bugs can affect several copies of the data.That’s what happened here. Some copies of mail were deleted...To protect your information from these unusual bugs, we also back it up to tape . Since the tapes are offline, they’re protected from such software bugs”.
12
Flood – Brisbane, Australia - January 2011
March 2011: Japan is Devastated
Protect Data - Out of Region
13
Agenda
� Storage Issues
� Best Practices – Unveiling the Facts– Addressing performance, data protection and
retention objectives
� Overview - LTO-5 Technology
� Introducing Linear Tape File System (LTFS)
14
Best Practices in Data Protection
� Have multiple copies or layers of protection: depending upon value of data, keep at least 3 copies, keep in different locations; one out of region – Use disk and tape
� Isolate one copy: at least one copy offline for logical system isolation to avoid intentional or unintentional corruption that can occur with online storage - Use tape; Keep offline
� Have technology diversification: copies on different forms of media to avoid a media or system process disaster - Use disk and tape
� Protect access to data: at rest and in transit – Use encryption & WORM
� Manage backup differently than archive:– Backup multiple, point in time consistent copies for operational and
DR recovery consistent with application specific RTO/RPO - Use disk and tape
– Archive single instanced data for long term retention: combination of disk & tape
*Best Practices Source: Debbie Beech, Sylvatica Consulting / David Hill, Mesabi Group
15
Large Truck Express Line Survives Hurricane Flood
Business Challenge:
• Hurricane Gastone flooded Data Center with 5 ft. of water
• Total loss of hardware, networks, phone systems,
generator, and utility power
• Good news: a tape backup of 100% of the data was
made the night before – stored off site!
Solution:
• Protect assets and business resilience with comprehensive best practice strategy
• Create nightly disk flash copy for fast retrieval and window-less backup to tape
• Backup 100% production data to LTO tape library nightly: tapes moved offsite
• Global Mirror DR site and backup to LTO tape library – Lights out!
• Creates 5 copies of data (2 offline on tape in different remote locations)
Benefits:
•Able to control TCO, access data and protect data with tiered storage strategy
•No production system interruption
•No save window-Set it & forget it
•No production cycles, no operators, lights out operations
•Logical data protection and out of region protection
Dick CrosbySystems Manager, Estes
"You are out of your mind if you think you can live without tape"
Implements Best Practice Data Protection and Retention Systems
16
“The reports of tapes demise are grossly exaggerated, ”
(borrowing a phrase from Mark Twain)
“Tape is cheap, safe and reliable and there is no substitute….the archive data backstop.
It is the data centre's lifebelt and lifeboat. Without it, when the data loss/data corruption storm strikes, you are sunk. It's that simple.”
Chris Mellor “Tales from the Storage Frontier” May 2011
17
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000P
etab
ytes
LTO 5
DAT 320
DAT 160
VS 160
SDLT S4
SDLT II
SDLT I
LTO 4
LTO 3
LTO 2
LTO 1
DLT IV
DDS-4
DDS-3
Dat 72
17
Tape Storage Continues Phenomenal GrowthTape Storage Continues Phenomenal Growth
Source: Santa Clara Consulting Group (SCCG) – Open System Tape Cartridge Shipments
•A total of 6.6M cartridges shipped in Q4 2011, nearly 90% LTO
•8% YtY Tape Growth
•LTO-5 tape continues to be the rising star of the tape market
-Q4 2011 Shipments up 19% quarter on quarter
-LTO-5 shipments up over 93% YtY Quarterly
82.6% of respondents are still using tape as their final destination for backupsPer 2011 Backup Central End User Survey by TruthInIT, Curtis Prestin, CEO
18
� It’s reliable high speed and capacity– Streams very fast and stores high capacity– Read after write verification for reliable writes– Servo tracking to help ensure precision tracking– Better bit error rate than disk! 1x10E17 bits vs. 1x10E15 bits
� It’s Cost-effective and Green– Lowest storage cost for the foreseeable future– Most energy efficient method of storing digital data – Cartridges on a shelf consume no energy
� It’s scalable – Easy to add additional storage (i.e. add cartridges)– Tape provides “infinite capacity” on demand
� It’s removable / transportable / shareable– Off-line and off-sight storage for data protection-archive– Cartridges are easy to ship (XX PetaBytes / Day)
�But tape is difficult to use!– Tape automation has simplified the process– LTFS makes tape easier to use than ever before
Why is LTO Tape on the Storage Hot List?
And easy
19
Tape Reliability Soars
Both disk and tape have made significant reliability improvements in recent years. For tape, reliability progress has been even better than disk comparing the BER (Bit Error Rate), which is quickly becoming a more popular means of measuring reliability.
Source: Tape Storage Future Directions and the Data ExplosionFred Moore, President, Horison, Inc. 2011
BER is the percentage of bits that have errors relative to the total number of bits received in a data transfer
2020
Costs Comparison Studies
ESG Backup/DR TCO Study: Dedupe VTL vs. LTO-5 Library System1
The Clipper Group Archive TCO Study: SATA Disk System vs. LTO-5 Tape Library System2
•12 Year TCO Archiving Study: costs covering hardware, maintenance, floor space and energy
•Disk storage was 15x Tape TCO
•The cost of energy alone for the average disk-based solution exceeded the entire TCO for the average tape-based solution
1A Comparative TCO Study: VTLs and Physical Tape, By Mark Peters, ESG, Feb. 2011.
2Clipper Notes report “In Search of the Long-Term Archiving Solution -Tape Delivers Significant TCO Advantage over Disk”, The Clipper Group, Dec.23, 2010. This was a general TCO study and did not specifically focus on LTFS or video storage.
•5 Year TCO Backup/DR Study; VTL with 15:1 Deduplication reduction ratio vs. LTO-5 Library
•Costs included hardware, maintenance, floor space, software, people and energy
•Scenarios included various DR methods (i.e. replication, PTAM)
•Dedupe VTL was from about 2-4 times more costly than tape system
3Cartridge price as of internet search Feb 2012.
0
2
4
6
8
10
12
14
16
TCO
DiskTape
15X
LTO-5 Cartridge is about 3 cents per GB uncompressed! 3
21
Archive Capability
Tape Disk
Source: Tape The Digital Curator of the Information Age. By Fred Moore, President, Horison, Inc.
22
Tape and Disk are Complementary for Optimal Performa nce, Archive, Data Protection and TCO
Virtual Tape LibraryTape Library
Application Servers
Backup Server
VTLTape Library
*Source: Fleishman-Hillard Research
Blended Tiered Storage Example - Layers of Protection
“There is no other medium that offers what tape does for archiving.”LTO tape technology continues to evolve with LTO-5, Curtis Preston,
techtarget.com Feb 18, 2010
Storage Manager Survey Results *• 61% of current disk-only users plan to start using tape
ReplicationDR
23
Agenda
� Storage Issues
� Best Practices – Unveiling the Facts– Addressing performance, data protection and
retention objectives
� Overview - LTO-5 Technology
� Introducing Linear Tape File System (LTFS)
24
LTO-5 Tape Can Preserve Large Backups and Archives
• LTO-5 Tape is Huge and Reliable
– 1.5 TB / cartridge native: 3 TB / cartridge (2:1)
– That’s about 30 DVD movies per cartridge
– Automation offerings from 20-1,000s of cartridge slots
– Highly Reliable: Servo Tracking, Read after Write
Verification, 250K MTBF Hours, up to 30 year shelf life
• LTO-5 Drives are Fast
– Up to 140 MB / sec. native
– Up to 280 MB / sec. (2:1 compressed)
– That’s > 1TB of saved data per drive / hr
(2:1 compressed)
LTOTAPE
25
LTO Data Security
� WORM (Write Once Read Many)– LTO 3, 4 and 5 drives and WORM cartridges – Unalterable tape data storage– Can append data to cartridge
� Tape Drive Encryption– LTO 4 and 5 Tape Drive Hardware Encryption– AES 256 bit encryption data key provided to tape drive– Data is compressed then written to tape cartridge in
encrypted form to maximize capacity and protect sensitive information
– Virtually no impact to drive performance– Helps eliminate need for encryption SW or appliance– Get encryption key management software from tape
vendors – Straight forward implementation process
26
Providence Health & Services Encrypts with LTO Tape
� Six data centers in five states all encrypting off-site media
� Daily backups are between 1 – 8 TB per site� Centralized, automated data protection system
eliminated manual management of backups� Effectively established a disk to disk to tape
strategy � Assured data is protected – LTO-4 addresses
security and compliance requirements
*See white paper: Securing Sensitive Information -- with LTO tape drive encryption. by Silverton Consulting at www.ultrium.com/whitepaper
“…it took only 1 to 2 days to implement encryption.” Mack Kigada, Data Storage Manager, Providence Health and Services
27
Agenda
� Storage Issues
� Best Practices – Unveiling the Facts– Addressing performance, data protection and
retention objectives
� Overview - LTO-5 Technology
� Introducing Linear Tape File System (LTFS)
“ Tape not only provides the best option for protecting most data today,
it is finding new roles to play.”Jon Toigo Mar 2012
28
■ A open software specification that allows simple and new ways of accessing data on tape (LTFS spec doc available at: www.ultrium.com/ltfs )
■ Self-describing tape format to address data archive requirements
■ Implemented on dual-partition LTO-5 tape
■ First partition holds the tape index / metadata
■ Second partition holds the content
■ It presents a tape as an extension of the operating system: appears as another drive letter, icon or folder like a disk or memory stick
What is the Linear Tape File System?
LTO Tape Joins the Ranks of Easy to Use Portability
29
LTFS: What are the potential benefits?LTFS: What are the potential benefits?
•Improved archive storage-“Memory stick like” self describing file system
-Tape can tell you what’s on it now and in the future
-Up to 30 year shelf life
•Easy to use-View contents in OS browser directory tree
-Simple “Drag & Drop” movement of data
•Increased data mobility-portability-Compatibility across OS environments
-No backup/archive software needed to view content
-A single storage media standard
-File, HW, SW and camera agnostic
•Reduced costs and energy consumption-LTO tape is less expensive than other storage formats
-Tape is “green”…a cartridge draws no power!
"I am shocked! This is exactly what we need!"
"LTO-5 technology gives tape-less work-
flow....with tape!"
"Now I can offer an LTO-5 archiving
service to my movie clients."
LTFS: How does it work?
• LTFS utilizes media partitioning (new to LTO Gen 5)
• Tape is logically divided “lengthwise” into two partitions
- Index partition : File system info, index, metadata (37.5 GB)
- Content partition : Contains the files / content bodies (1425 GB)
� When mounting the tape, the Index is copied to the workstation/server memory for fast access and updates
� Periodically the index is backed up to the content partition
BOT
EOT
Content Partition
File File File File
Index PartitionGuard Wraps
Index/Metadata
31
Tape browsing on Linux
Files can be accessed on tape directly from any application
Device Directory
Tape Contents
"We think that LTFS could be one of the most significant developments in
the tape drive space since the introduction of LTO itself.“
George Crump, Analyst, Storage Switzerland, May 2010
How it Works: LTFS in Action with File Browser
See LTFS in Action -1 Min. Movie Clip at: www.ultrium.com/ltfs
Oscar Winners
32
LTFS – Easily Exchange, Archive and Share Files
Easy to Use - Archive - Share
LTO-5 Drive
Share
Share
Share
33
Thought Equity Motion: Video Archiving in the Cloud
•Business Need
• Needed a low cost delivery platform for enterprise scale Video Supply Chain as a Service
• Information growth of ~100 TB per month
• Easy self-serve access required by clients
•Solution
• Linear Tape File System at several global locations, including some client facilities
• Tape Libraries and LTO-5 tape drives
•Benefits
• Opened up new business opportunities
• Enabled more predictable and transparent pricing for clients
• Portable, interoperable, scalable, cost-effective data protection and long-term storage
“LTO 5 and LTFS significantly reduce the ancillary costs around storage. This is a real game-changer!”
Mark Lemmons
CTO, Thought Equity Motion
3434
LTO Ultrium Roadmap to the Future
�Over 4M LTO Tape Drive Shipments�Over 200M LTO Cartridge Shipments