sharepoint storage: virtualize and optimize
DESCRIPTION
Ryan Tacy AvePoint. SharePoint Storage: Virtualize and Optimize. Agenda. Native SharePoint Storage BLOBs and Databases Storage Optimization Methods Summary. The Basics of. SharePoint Storage. Planning for SharePoint Storage. Recycle bin Versioning Search and index information - PowerPoint PPT PresentationTRANSCRIPT
© AvePoint, Inc.
SharePoint Storage: Virtualize and Optimize
Ryan TacyAvePoint
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Agenda
Native SharePoint Storage
BLOBs and Databases
Storage Optimization Methods
Summary
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
SharePoint StorageThe Basics of
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Planning for SharePoint Storage
• Recycle bin• Versioning• Search and index information• Growth
• Good rule of thumb for initial planning is: 3.5 x file system
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Basics - Managing SharePoint storage
• Set sight quotas and alerts!– 10 GB quota, 8 GB alert is my favorite
• Monitor growth trends– Sites: slow over time or large jump in size?– Overall content DB size
• Split Content DBs if they get “too big”
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
How SharePoint “chooses” a Content DB for a site
• Highest remaining allotment rule– Content DB 1: 100 sites max– Content DB 2: 100 sites max
– Content DB 1: 100 sites max– Content DB 2: 200 sites max
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Optimal Content DB Sizing
• Backup & Recovery operations(<50-100 GB)• Performance (<500 GB… nervous at 300 GB)
– # of objects– size of objects– Hardware (servers and storage)
• Storage Cost (as small as possible!)
So what is too big?
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
BLOBsSize of Objects and Performance
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
BLOBs-- what’s the issue?
• BLOBs = Binary Large OBjects • SharePoint Content = BLOB + Metadata• Content DB = Database of… BLOBs + Metadata• SQL DB storage needs high IOPS / low latency• High IOPS + low latency storage = $$$$• BLOBs do not participate in query operations, so no
real reason to have BLOBs in a DB• DB full of BLOBs = wasted $$$
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
SharePoint WFE
SharePoint Object Model
SQL ServerB
LOB
s &
Met
adat
a
Content DB Config DB
Default SharePoint Storage
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Database Size ImplicationsBLOBs increase DB size, creating issues with:
• Backup & Recovery operations• Performance• Storage Costs
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
0 1 2 3 4
Active DataTotal Data
Issues with BLOBs get much worse over time
Time
Dat
a
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
SharePoint Storage Optimization Methods
• Move the BLOBs out of the database
• Archive content
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
ExtendingMoving the BLOBs out of the database
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Available APIs for Extending
SQL Remote BLOB Storage (RBS)
SharePoint External BLOB Storage (EBS)
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
EBS
• API available in SharePoint 2007 SP1+• EBS provider can take ownership of the BLOB• Provider generates a SharePoint token or stub• SharePoint knows how to retrieve the object • Transparent to end-users
SharePoint WFE
EBS Provider
BLOB
Metadata
SharePoint Object Model
SQL Server
Content DB Config DB
BLOB Store
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
EBS
• Implemented by SharePoint• Only 1 EBS Provider per SharePoint farm
• Orphaned BLOBs- no direct method to compare BLOB store and Content DB
• Compliance- what if I don’t want to allow SharePoint to delete the object?
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Provider Library Y
RBS
• API available in SQL 2008R2 Feature Pack
• Not unique to SharePoint, available to any application
• A Provider Library can be associated with each database
SharePoint WFE
SharePoint Object Model
Content DBX
Content DBY
Relational Access
BLOB Store
RBS Client Library
BLOB Store
BLOB Metadata
BLOB & Metadata
SQL Server
Provider Library X
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
RBS• Implemented by SQL• Only 1 RBS Provider per Content DB• Storage trigger on BLOB size
• Orphaned BLOBs much less of an issue• Can lock down operations, from a unified
storage perspective• Can be managed via Powershell
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
EBS
Tighter integration
with application,
allows for more rules and settings
EBS versus RBS, which is better?
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
EBS
Tighter integration
with application,
allows for more rules and settings
RBS
Simpler, allows unified storage
architecture across
applications
EBS versus RBS, which is better?
http://www.codeplex.com/sqlrbs
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
It looks like RBS has won…
SQL Remote BLOB Storage (RBS)
SharePoint External BLOB Storage (EBS)
SharePoint 2007
SharePoint 2010
Future SharePoint Release (SPS 5?)
SQL Server 2005
Future SQL Releases
SQL Server 2008
Microsoft will provide a powershell solution to migrate from EBS to RBS
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
2 Approaches, Same Result• Backup & Recovery operations
– Databases are 60-80% smaller– Need a method to backup BLOBs synchronously
• Performance– Databases are 60-80% smaller– Performance improvement increase as the file/BLOB size
increases. Microsoft research indicates:• <256kb, SQL better• 256kb to 1mb, SQL and file system comparable• >1mb, file system better
• Storage Cost– “Not as expensive” storage– Archiving still needed for true savings
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Web Services
Search / App Services
Cloud Storage
Tier 1Tier 2Tier 3
SQL Database
Extend Content
out of SQL
Extended SharePointStorage
Backup
Not involved in SharePoint
Storage
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Cloud Storage Use CaseSharePoint “Overdraft Protection”
DB alert set at 80 GB, limit at 100 GB0
80
100
Alert sent to admin
No action takenCloud
Storage
• Could be any storage
• Cloud is ideal “insurance”--cheap to setup, expensive to use
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
ConnectingAn approach to not creating the BLOB
problem in the first place
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Should you migrate data to SharePoint?• Business case
– Value add of legacy system– Maintenance costs
• Hardware• Licensing and support• Knowledge
– Migration costs• Migration process• Tools• Training
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Migrate Connect• Data is available through
SharePoint• Data is left in source (legacy)
system• Give legacy system second life
by increasing its value• Burden of storage is on legacy
system• Changes propagate to source• Connect and forget
• Data is available in SharePoint• Data is moved into SharePoint• SharePoint replaced legacy
system• Burden of storage is on
SharePoint• Changes saved in SharePoint• Migrate and decommission
Migrating vs. Connecting
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
ArchivingAdding Data Lifecycle Management to
the picture
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Data Lifecycle Management
• Records Center– Another SharePoint site– Higher % inactive content– Consider separate Content DB, with an RBS
provider implemented for this DB• Archiving
– Backup and delete– Partner solutions
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Web Services
Search / App Services
Cloud Storage
Tier 1Tier 2Tier 3
SQL Database
1. Analyze data2. Apply business
rules3. Fully utilize
storage
Extending andArchiving
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
1
2
3
4
Summary
BLOBs aren’t required for queries, only the metadata is, hindering query performance
RBS offers advantages over EBSRBS allows for a unified storage architecture across applications, not just SharePoint storage, but only available with SQL 2008 R2
Third Party solutions are required to leverage RBS/EBSCheck out AvePoint’s solutions for Storage Optimization, includes tools for extending, connecting, and archiving
SQL is not optimized for BLOBs
Leverage BLOB Services Technology to Optimize SharePoint StorageBLOB API’s can be leveraged to store BLOBs outside of SQL with no negative end-user impacts, to save $$ and optimize storage
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
?Questions?
© 2009 AvePoint, Inc. All rights reserved. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written consent of AvePoint , Inc.
Resources - www.AvePoint.com
34
Ryan [email protected](978) 409-2347
Download a FREE, fully-enabled 30 Day trial of DocAve at www.avepoint.com/download