![Page 1: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/1.jpg)
BioHPC Cloud Storage
1
[web] portal.biohpc.swmed.edu
[email] [email protected]
Updated for 2015-06-10
![Page 2: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/2.jpg)
Overview
2/32
What is the cloud service/storage?
Services we provide to BioHPC user?
– BioHPC cluster
– BioHPC Lamella Storage System
– BioHPC File Exchange
How can I use BioHPC cloud storage?
– Basic File Management
– File Synchronization
– File Transfer
– File Sharing
– BioHPC manager account
![Page 3: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/3.jpg)
3/32
Cloud Service ( Cloud Computing and Cloud Storage )
Image retrieved from http://giannisgiataganas.blogspot.com/2010/06/lets-talk-about-cloud.html, http://www.nec.com/en/global/solutions/cloud/portfolio/storage.html
Cloud storage (one of the fundamental components of cloud computing) is a service model in which data is maintained, managed and backed up remotely and made available to users over a network (typically the Internet).
Cloud computing refers to the delivery of computing resources over the network.
![Page 4: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/4.jpg)
4/32
Image retrieved from http://www.bcni.ca/cloud-services/
• Security
• Reliability
• Speed
• Ease-of-use
• Storage space
How to Evaluate a Cloud Storage Service
![Page 5: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/5.jpg)
7 Best Cloud Storage Service 2015
5/32
Ranking retrieved from http://www.pcadvisor.co.uk/features/internet/3506734/best-cloud-storage-dropbox-google-drive-onedrive-icloud/– by Martyn Casserly, 09 February 15
![Page 6: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/6.jpg)
Security
Storage space
Other Limitations of Web-based Storage System
6/32
Why We Need BioHPC Cloud Service
UTSouthwestern General Cloud Policy: UTSW information must never be stored in unapproved online file storage systems.
Dropbox: 2~18 GB (5 TB @ $75/month)Google Drive: 15 GB (30 TB @ $299.99/month)BioHPC Cloud: 150GB + access to BioHPC storage system (~ 1 PBs)
• Max upload/download speed ~ 10 MB/s• Web browser do not support folder uploading• Upload/download limit for single file (size)
![Page 7: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/7.jpg)
BioHPC Cloud Storage
7/32
As a BioHPC user, you will have access to:
• BioHPC Cluster/home2/username 50 GB quota per user/work/department/group/username 5 TB quota per group/project/department/group/username 5 TB quota per group
• BioHPC File Exchange (web-interface)https://cloud.biohpc.swmed.edu 50 GB quota per user
• BioHPC Lamella Cloud Storage (web-interface), on campus only, private cloudhttps://lamella.biohpc.swmed.edu 100 GB quota per userGateway to BioHPC Cluster (via FTP, SAMBA or WebDAV)
![Page 8: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/8.jpg)
Initial Setup on Lamella
8/32
* Mount External Storage
— Setting up Lamella to access BioHPC cluster (home, project, and work space)
lysosome username password homeproject
For home leave blank
For private project space:department/lab/user
For lab shared project space:department/lab/shared
BioHPC Endosome/Lysosome
![Page 9: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/9.jpg)
What can we do with BioHPC cloud storage system
9/32
• Basic File Managementview, move, rename, download, upload, delete your files and etc.
• File Synchronizationowncloud client
• File TransferClusterSambaFTPWebDAV (not recommended)
• File SharingShared FolderBioHPC File Exchange (https://cloud.biohpc.swmed.edu)BioHPC Lamella Cloud Storage (https://lamella.biohpc.swmed.edu)
• BioHPC manager accountA lab manager can share data to non-BioHPC users
![Page 10: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/10.jpg)
Navigation bar:
– Allows navigation between different parts
New/Upload button:
– Allows you to create new files or upload existing ones from your device
– you can also drop files from Explorer or Finder onto the file view.
– With New button, you can create a text file, folder or download a file from the provided URL.
Search/Settings:
– Search allows you to look for files and directories
New/Upload button:
– Clicking on small thumbnails or icons of the files/directory
– Select multiple files can e deleted or downloaded as ZIP
Viewing Files
– Can display text files, PDFs, and image files
– If displaying of a file is not possible, a download process will start
Moving files
– Dragging and dropping files/directory
Basic File Management
9/32
![Page 11: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/11.jpg)
File Synchronization
11/32
BioHPC Cluster(work or project folder)
Lamella Cloud Storage
< 100 GB
> 100 GB
![Page 12: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/12.jpg)
12/32
* File Synchronization
File Synchronization
Before setting up file synchronization:
1) Estimate the total size of your dataif the total size of your data is always < 100GB, you may direct
synchronize your data to lamella, otherwise use your BioHPC project directory.
2) Estimate the total free space on your local diskif the total size of your data is always < free space on your local disk,
you may synchronize everything from lamella, otherwise chose what to sync.
Rule of thumb : Your data size should always smaller than your container
![Page 13: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/13.jpg)
File Synchronization
13/32
download and install owncloud client from https://portal.biohpc.swmed.edu/content/software/
connect to lamella.biohpc.swmed.edu -> username/password
* File Synchronization
![Page 14: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/14.jpg)
File Synchronization
14/32
Chose What to Sync
total data size < local disk space
total data size > local disk space
Total data size = actual file stored in lamella locally (up to 100 GB) + mounted folders (up to 10 TB)
Anything your deleted from your PC will be removed from BioHPC cloud storage server as well.
* File Synchronization
![Page 15: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/15.jpg)
15/32
File Transfer
Lysosome
Lamella Cloud Storage
![Page 16: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/16.jpg)
16/32
If your are using a BioHPC thin client or workstation you have direct access to your BioHPC storage.
File Transfer
Separate training session – July 14th.
Accessing BioHPC Storage Directly from thin client, workstation
![Page 17: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/17.jpg)
17/32
File Transfer
Accessing BioHPC Storage Directly via SSH
Host Name: nucleus.biohpc.swmed.eduPort: 22
Download and install PuTTY from https://portal.biohpc.swmed.edu/content/software/
![Page 18: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/18.jpg)
18/32
Computer -> Map Network Drive
Folder is: \\lamella.biohpc.swmed.edu\username home\\lamella.biohpc.swmed.edu\project\department\group project\\lamella.biohpc.swmed.edu\work\department\group work
Check ‘Connect using different credentials’Enter your BioHPC username and password when prompted.
File Transfer
Accessing BioHPC Storage Directly from Windows via Samba
![Page 19: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/19.jpg)
Accessing BioHPC Storage Directly from Mac OSX via Samba
19/32
Finder -> Go -> Connect to Server
Folder is: smb://lamella.biohpc.swmed.edu/username homesmb://lamella.biohpc.swmed.edu/project/department/group projectsmb://lamella.biohpc.swmed.edu/work/department/group work
Enter your BioHPC username and password when prompted.
File Transfer
![Page 20: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/20.jpg)
File Transfer
20/32
Host: lamella.biohpc.swmed.eduUsername: Your BioHPC usernamePassword: Your BioHPC passwordPort: Empty
Transfer data to/from BioHPC cloud storage via FTP
![Page 21: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/21.jpg)
21/32
Accessing your Files from Windows via WebDAV
1) Increase the maximum size of WebDav downloads to 4GB
regedit -> HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters\FileSizelimitinBytes
By default Windows limits the maximum size of WebDav downloads to ~50 MB
File Transfer
Not recommended but still useful when you need direct access to the local
storage of BioHPC Lamella or File Exchange system from your PC
![Page 22: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/22.jpg)
22/32
2) Mount your lamella storage to PC
Start -> Computer -> Map Network Drive
* You could mount BioHPC File Exchange storage to your PC in a similar way .
(https//cloud.biohpc.swmed.edu/remote.php/webdav)
File Transfer
Accessing your Files from Windows via WebDAV
![Page 23: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/23.jpg)
23/32
File Transfer
Cyberduck – A Client support FTP, WebDAV for both Mac and Windows
• Download and install Cyberduck from https://portal.biohpc.swmed.edu/content/software/
• Once installed into your 'Applications' folder, start Cyberduck, choose 'Open Connection' and configure as in the image below:
Accessing your Files from Mac OSX via WebDAV
![Page 24: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/24.jpg)
24/32
Uploading / Downloading Speed
Max Uploading andDownloading Speed*
Lamella Cloud Storage(Samba)
400MB/s
BioHPC File-Exchange(webDav)
10 MB /s
Lamella Cloud Storage(webDav)
100 MB /s
Lamella Cloud Storage(FTP )
300 MB ~ 1.4 GB/s
• Upload and download speed depends on your actual network connection• We don’t set up a limitation on max concurrent sessions
File Transfer
![Page 25: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/25.jpg)
File Sharing
25/32
• Sharing data with all BioHPC user/work/shared /project/shared
• Sharing data inside your department/work/department_name/shared /project/department_name/shared
• Sharing data inside your group/work/department_name/group_name/shared /project/department_name/group_name/shared
If you have a workstation connected to our BioHPC cloud or you know how to use
command line to connect to BioHPC cloud, you can share your data on cluster directly.
Shared folders had already been set up at different levels
![Page 26: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/26.jpg)
26/32
Sharing data with UTSW collaborators
File Sharing
BioHPC Lamella Cloud Storage
https://lamella.biohpc.swmed.edu
Share with linkShare with BioHPC user or group(Email notification)
![Page 27: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/27.jpg)
27/32
Sharing data with non UTSW collaborators
File Sharing
BioHPC File Exchange
https://cloud.biohpc.swmed.edu
Share with link(Email notification, setup password & expiration date)
![Page 28: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/28.jpg)
28/32
A lab manager ( BioHPC manager account ) sharing data to non-BioHPC users
Lysosome
Lab Manager Lab manger account : only provide storage space under
/project/department/group/manager_name
,and this is the home directory of manager account
Lab Facilities, e.g.: scanning EM
images
non-BioHPC user A
non-BioHPC user B
non-BioHPC user C
directly saved to manager’s home directory
![Page 29: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/29.jpg)
29/32
A lab manager ( BioHPC user ) sharing data to non-BioHPC users
Lysosome
Lamella Cloud Storage
Lab Manager
non-BioHPC user A
non-BioHPC user B
non-BioHPC user C
Share data for user to download
![Page 30: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/30.jpg)
A lab manager sharing data to non-BioHPC users
30/32
Step 1: mount manager’s home directory to your lab facility to save data
Support PC, Mac and Linux/Unix
![Page 31: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/31.jpg)
A lab manager sharing data to non-BioHPC users
31/32
Step 2: Sharing data with non BioHPC users with a linkpassword protection (recommended) set up an expiration date (recommended) email notification
![Page 32: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/32.jpg)
A lab manager sharing data to non-BioHPC users
32/32
Step 3: non BioHPC users download data by providing correct password
![Page 33: BioHPC Cloud Storage · 2019. 6. 27. · What can we do with BioHPC cloud storage system 9/32 • Basic File Management view, move, rename, download, upload, delete your files and](https://reader033.vdocuments.us/reader033/viewer/2022051901/5fefd14a8834686ae51d6c16/html5/thumbnails/33.jpg)
For More Information
• Read related articles from https://portal.biohpc.swmed.eduLamella - Internal Cloud StorageBioHPC File Exchange - External Cloud Storage
• Email the ticket system: [email protected]