intelligent archiving strategies: toward ilm
Post on 14-Feb-2016
16 Views
Preview:
DESCRIPTION
TRANSCRIPT
Intelligent Archiving Strategies: Toward ILMArun Taneja, Founder and Consulting Analyst, Taneja GroupAlex Gorbansky, Senior Analyst, Taneja Group
Agenda A Bit of Historical Perspective Why Archive? What to Archive? The ILM Panacea Developing an Operational Archival Strategy Key Considerations Representative Vendors and Solutions Conclusions
Archival ≠ BackupBACKUP
Copying production data to an alternative medium for restorability in the event of data loss, corruption, or unavailability.
ARCHIVAL
Retention of historical data for future access for business reasons such as audits, customer issues, or litigation.
Some History On Archiving
3000 BCE
Ancient Egypt:• Library of Alexandria• Engravings
Middle Ages 1600s 1789 1884
Shift from Feudalism To Nation State:• Records• Property rights
American colonists• Births • Marriages• Businesses
French Revolution• Property records
American Historical Association• Archival standards • Marriages• Businesses
Archival Business Drivers TodayREGULATORY COMPLIANCE
REQUIREMENTS
EXPLOSIVE DATA GROWTH
APPLICATION PERFORMANCE DEGRADATION
RISING COSTS
What to Archive? Structured Data:
• ERP/CRM DB tiers• Business transactions
Unstructured Data: • Documents• X-Rays• Check Images• Voice recording
Semi-structured Data:• Email • Instant Messaging
ILM…ShmILM
“ILM” is an abstract framework for describing the processes and technology used to manage information throughout its life according to its business value.
“ILM” is NOT the panacea for your storage management challenges.
Archival is a key component of what vendors are calling “ILM
Applications: ERP, CRM, Email, Call Recording, Image Access
Application Data: Structured, Unstructured, Semi-Structured
Policies and RulesBusiness ContextReferential Integrity Regulatory Compliance
Snapshots HSMData Movement Technologies
Replication Backup Archival
PrimaryStorage Infrastructure Tiers
Secondary Tertiary
Developing an Archival StrategyPLAN When/How Data Classification Requirements
2. DESIGN
3. IMPLEMENT
4. REPORT &
TEST
Why Plan and When to Start Upfront Planning will Result in Significant Benefits in
Future Phases. Develop an Archival Strategy as part of your
application design and development process. Engage Key Stakeholders:
• Application Owners• Business Decision Makers: Compliance Officers, Legal
Identify Key Archival Business Drivers:• Regulatory Compliance• Other: Data Growth, Increasing Costs, Poor Performance
The Data Classification PuzzleAssess the application data in your shop
according to the following categories:• Structured: database• Unstructured: files, videos, images• Semi-structured: email
Identify specific data sets impacted by regulatory compliance:• Examples: Email, Medical Records, Call Recordings
Requirements DefinitionEngage Application OwnersCompliance not the ONLY archival driver Separate requirements processes for
applications impacted by compliance.Compliance-specific:
• Retention period• Media characteristics• Data restorability rates• Access control policies• Data availability/DR
General archival:• Data Access Patterns• Restore time requirements• Application performance• Cost structure• Access control policies• Data availability/DR
Taming the Compliance Monster1. Understand the Regulations: Significant Variance
by Industry
2. Assess/Communicate Requirements to Key Business Stakeholders
3. Judge Products for Yourself – Just because a vendor says a solution is “Compliant” doesn’t make it so.
4. Stay abreast of changes in regulatory mandates.
Defining Key Archival Metrics
Archive Distribution Percentages Across:• Online: Disk, Object-based storage• Near-line: Optical, Tape (local)• Off-line: Off-site vaults
Number of data copies• Local• Remote
Designing an Archival SolutionRequires an application specific
assessment – look for commonality in application requirements
Wholly enterprise-wide strategies will be difficult to build and sustain
Evaluate alternative solutions based on application requirements and metrics
Don’t Ignore the Organizational Dynamics
Archival Touches Multiple Organizations:• IT – Applications• IT – Infrastructure• Legal• Users
Consequences of mistakes are enormous:• Fines• Litigation
Consider organizing a cross-functional team led by an archival champion with a combination of technical and business expertise
Comprehensive Application Assessment
Data Classification Exercise Data Set Size and Historical and Predicted Data Growth Rates based
on business drivers Is Regulatory Compliance an Issue? Data Valuation over Time:
• Access patterns of data of 90 days old and beyond.• Cost of data loss
Going it alone can be difficult Available resources:
• Services organizations: GlassHouse, Accenture, EDS, Storage Vendor• Application Management Tools: File-Level SRM, Precise
Budgetary Requirements
Components of the Archival Stack
Application Specific ModuleDiscovery and analysis of data assets
Business rules and policies definitions
Identification and movement of specific data to
appropriate storage medium
Management, indexing of data and metadata
Access control mechanism
Application Data
Storage InfrastructurePhysical archive repository
Data Preservation and Protection
Indexing Technologies for Retrieval
Management
& Control
Physical
Repository
Data Flow
Structured Data Archival Challenges to Investigate
ERP deployments are still very nascent Preventing application downtime during archival Preserving referential data integrity:
• Archival of core data and associated data in other tables
Enforcing single read-only state across related data Delivering transparent access to archived/combined data via
native app UI• Maintaining performance of remote queries and union views.
Update process:• Restate vs. entire reload
Unstructured Data ConsiderationsScalability Sustained performance with data growth• Hierarchical file-systems limited at large scales
Content Access and VisibilityMeta data use to intelligently manage and
maintain archive addresses traditional file system limitations
Scalability of Index (Content addresses)
Email Archival Challenges
Stringent regulations: SEC Rule 17A-4• Non-rewriteable, non-reusable media• Verification of writes• Serialize units of media
Solution Requirements• Server-based capture• Support for multiple distributed Email Servers
Meta Data Holds Real Value
• Digital asset tied to specific infrastructure• No value outside of infrastructure context
• Self-describing attributes for digital asset• Enables powerful policy-based data
movement applications
Traditional File Systems
Object-based systems
Meta Data is data about data
Object Age and creation date Object Change History Associated application/users Access control Priority/Criticality Data Access/Frequency
Choosing the Right Storage Medium
Amount of D
ata
Probability of Reuse
D2D Systems
Libraries
Drives
< Seconds Minutes Hours to Days
1 WeekLife Expectancy
1 Month 3 Months 18 – 30 Years1 Year
Disk Systems
Recovery Time
Object Storage
Key Considerations for Storage Media
Cost Access time Application access method:
• NFS/CIFS• Application-specific API
Reliability/Availability Data Preservation Capability Scalability Archival solution integration
Storage Media ConsiderationsPros Cons
Primary Storage No risk of data lossInstantaneous access
Exorbitant costsPerformance degradation
Secondary Storage (SATA)
Cost effective Solid access time
IntegrationEnforcing preservationManagement
Object Storage Fit for large unstructured files Elimination of data redundancy WORM-like preservation
Price premiumPerformance scalability with index growth
Tape Most cost effective Removable Integrated WORM
Access timeReliability
Shifting towards an On-line Model
Primary
ObjectStorage
Tape
SATA
Representative VendorsStructured Data Email Unstructured
Archival Solutions
OuterBay, Princeton Softech, Applimation, Ixos,
Legato, KVS, Assentor Documentum, FileNet, NICE
SATA Object TapeStorage Platforms
CLARiiON, STK, IBM, Nexsan COPAN, Centera, Archivas, Permabit, DCT
STK, Quantum, ADIC, IBM
Start with your application vendor
Trust But Verify
Develop processes to periodically access historical data to test:• Data integrity• Access time
Manage capacity growth using vendor-supplied reporting tools
SummaryArchival is not backup and is not just about
complianceSuccessful strategy requires application-centric
approachEngage with key corporate stakeholders to
define requirements and select solutionsLook for automated and interoperable software
and hardware modules.Be Paranoid!
Thank you!
Arun Taneja
arunt@tanejagroup.comAlex Gorbansky
alex@tanejagroup.com
top related