[ic manage] workspace acceleration & network storage reduction
DESCRIPTION
Learn about IC Manage Views, a FUSE-based user space file system that allows near Zero Time Sync (ZTS) for workspaces of any size and complexity on both Windows and Linux.TRANSCRIPT
1
Workspace Acceleration & Network Storage Reduction
Shiv Sikand VP of Engineering
Roger March Chief Technology
Officer
2
Driven by Semiconductor Design, Video game software, Enterprise software…
Perforce depots: multi-TB+, 100,000,000s+ files
Single workspaces: multi-GB, 10,000s+ files
Users: 1000’s across globe
Digital Assets Growing at Rapid Rate
3
Source: 2012 LaunchM Survey, 524 respondents
Impact of Increasing Digital Assets: Survey cites Top File System Problems
44% 36% 36% 33%
25% 18%
5%
Slow WS syncs, apps
Bandwidth
Storage capacity, cost
4
Accelerate workspace syncs
Remove network bandwidth bottleneck
Reduce network disk space usage
SLOW Syncs
Vision
5
Network Storage
Network Storage
Workspace C File 1 File 2 File 3
Workspace B File 1 File 2 File 3
Workspace A File 1 File 2 File 3
User A
User B
User C
• Duplicates file storage for every user workspace
• De-‐dupe too slow for fast changing data sets
• Saturates precious Tier 1 storage space
• Under-‐uAlizes local caching
High Demand on Network Storage
Perforce SCM Server
File Data Metadata
File 1 File 2 File 3
6
File Not Yet Opened
User A
User B
User C Local Cache
File 2 0 b
File 1 0 b
File 3 0 b
• Virtual workspaces projected using Perforce db.have table informaAon from fstat
• 100% dynamic, user controlled workspaces
• Local storage agnosAc, e.g. disk, tmpfs, SSD....
Local Cache
File 2 0 b
File 1 0 b
File 3 0 b
Local Cache
File 2 0 b
File 1 0 b
File 3 0 b
Dynamic Virtual Workspaces Instantly displays fully populated workspaces
7
Local Caching Streams actual files on applicaAon / user demand
User A
User B
User C
Local Cache
File 2 0 b
File 1 0 b
File 3 0 b
• Files can be streamed from main Perforce server, Perforce proxy or Perforce replica
• TCP streaming protocol avoids tradiAonal NFS-‐delivery latency
• 2x faster retrieval of cache-‐warm files due to on-‐board access
File Not Yet Opened
File Opened Read-only
Local Cache
File 1 0 b
File 3 0 b File 2
Local Cache
File 2 0 b
File 1 0 b
File 3
8
User A
User B
Local Cache
File 2
Local Cache
File 3 0 b
File 2 0 b
User C Local Cache
File 2 0 b
File 1 0 b File 3
File 1 0 b
Automa0c cache management stays within storage quotas using LRU algorithm
File 3
File
File 1
File Not Yet Opened
File Opened Read-only
Local caching -‐ manages cache quotas
9
Intelligent File RedirecAon: 2x faster file access
Reads
Writes
Intelligent File Filtering approach • Separates reads from writes, stores reads in local cache
• Modified files automaAcally wriWen to NAS (or local cache) Advantages • Gets on-‐board speeds for reads instead of network storage • Modified files safely stored on network disks • Automated -‐ No manual management
Intelligent File Redirect On-‐board speeds for reads
Network Storage
Local Cache
Read/ Write
Read -only
Read -only
Read -only
Read/ Write
Read/ Write
Read -only
Read -only
Read/ Write
Read/ Write
10
User A
User B
Local Cache Local Cache
File 3 0 b
File 2 0 b
User C Local Cache
File 2 0 b
File 1 0 b File 3
File 1 0 b
Newly created files, checked out files saved to network storage volume or kept local
File 3
File
File 2 0 b
Network Storage
TEMP 1
File Opened Read/Write
File 1
File Not Yet Opened
File Opened Read-only
Intelligent file redirect
11
User A
User B
Local Cache Local Cache
File 3 0 b
File 2 0 b
User C Local Cache
File 2 0 b
File 1 0 b File 3
File 1 0 b
Checked in objects automa0cally purged from write storage
Avoids duplicated files, frees up space.
File 3
File
File 2 0 b
Network Storage
File 1 TEMP 1
Real-‐Ame de-‐duplicaAon -‐ frees space a\er Perforce check-‐ins
File Opened Read/Write
File Not Yet Opened
File Opened Read-only
12
IC Manage Views: Accelerates Workspace Syncs & Reduces Network Storage
IC Manage Views
Network storage usage reduced by 4x.
Local Caching,
Real-‐3me De-‐dup
Zero-‐Time Sync Workspaces.
Dynamic Virtual Workspaces
2x faster file access, applicaAon speed up.
Intelligent File Redirec3on
13
IC Manage Views AddiAonal Benefits
• NAS agnosAc -‐ 100% compaAble with exis0ng storage technologies
• Scalable -‐ Savings increase with number of users and the size of databases.
• Flexibility to build Workspaces on Demand. SoVware teams can build workspaces anywhere, avoiding problems and costs associated with disk space alloca0on.
• Reliable. Handles cache recovery in the event of failures or errors.
• Designers maintain workspace file control and stability. No manual management of network cache storage and different versions
• Reduces load on filers and network.