How do I set up a storage volume?

We have a provisioning page to create Turbo volumes. Go to ITS Service Request System (SRS). Turbo can be ordered in 1TB increments. We provide Turbo Ordering Instructions to help with how to fill out the form.

How do I make changes to a volume?

  • Go to the ITS Service Request System (SRS)
  • Select Request Service at the top of the screen.
  • Select Modify for either NFS or CIFS and on the following page choose the Turbo volume you wish to modify
  • Enter in any changes you wish to make.
  • Near the bottom of the page, select Next to move on to the next page.
  • Continue making changes on each page until you reach the Review page.
  • After reviewing all your changes to ensure they are correct click on Submit Now to submit the changes to your volume.
  • The changes will be sent to the ARC Storage team so they can process the request. This can take up to a business day to complete. They will update you as soon as the changes have taken effect.

What is replication?

With replication, data is written to storage at a separate location for purposes of data protection and service continuity.

What are Snapshots?

Snapshots provide a read-only copy of your Turbo volume at a specific point in time. To a certain extent, this will enable recovery of lost files, however; only files that have been snap-shotted overnight can be recovered. Files that are lost on the same day in which they were created are not guaranteed recoverable. Please be aware that Snapshots will take up space on your volume. If you have 10TB of storage and have 5TB of data, you’ll potentially have 5TB of Snapshots and have used up all of your storage space.

Your snapshots can be found in /nfs/turbo/<volume_name>/.shapshots/

Snapshots can be requested at no additional charge, and are available with the following options:

  • Daily snapshot, retained for 1 day
  • Daily snapshot, retained for 3 days
  • Daily snapshot, retained for 7 days
  • Daily snapshot, retained for 7 days, and 1 bi-weekly snapshot

Can I store sensitive data on my volume?

Yes. All sensitive data must comply with our Data Security policies.

Is there a special rate for researchers from Michigan Medicine or the College of Literature, Science, and the Arts?

Both the Med School and the College of Literature, Science, and the Arts offer their PIs subsidized storage volumes. Please refer to the following pages for more information:

How do I move files to and from my new storage volume?

Moving data into your storage volume is as easy as mounting it to a local workstation or server and copying or moving the data to it.

If you have only made the storage volume available only to the Great Lakes HPC Cluster or some other cluster, we recommend one of the following options. We highly recommend using the Globus Connect file management service when moving data in larger volumes than 5TB, but any of the methods listed will suffice, depending on your operating system.

Globus

*** Be aware that if you have a Multi-protocol or NFS Turbo volume that is mounted on Great Lakes there is no need to set up a separate Globus endpoint for it ***

Transferring files to and from Great Lakes:

  1. Install Globus Connect Personal. Select the OS you are using and follow the documentation to install Globus Connect Personal, and setup a local Globus endpoint.
  2. Go to https://www.globus.org and click on “Login” in the upper right corner.
  3. Select “University of Michigan” as your organization in the drop-down field and select “Continue.”
  4. This will prompt you for your uniqname and UMICH (Level-1) password. Enter these and select “Login,” then authorize with Duo two-factor authentication.
  5. You should be redirected to the “File Manager” tab on the left side of the screen.
  6. Select the “Collection” field and type your local Globus endpoint.
  7. On the right side of the screen, select “Transfer or Sync to…”.
  8. Select the new black “Collection” field and type in the Great Lakes Globus endpoint. The endpoint is umich#greatlakes.
  9. By default, you will see into your Home directory. You can change your path under the “Path” field.
  10. When you’ve selected the files you wish to transfer, select the “Start” button near the bottom of the screen. A notification for the transfer will appear in the “Activity” tab on the left side of the screen.

Cyberduck

Cyberduck can be used on Mac or Windows 

  1. Open Cyberduck and click the Open connection button.
  2. Set the “Server:” to: be greatlakes-xfer.arc-ts.umich.edu.
  3. Set your Username: to be your uniqname.
  4. Enter your Kerberos password.
  5. Click connect.
  6. Drag and drop files between the two systems. Click the Disconnect button when completed.

WinSCP

Connecting to a Remote System

  • Select the protocol you would like to use (recommend sftp or scp).
  • In the host name field, type the hostname you would like to connect to.
    • e.g.: greatlakes.arc-ts.umich.edu
  • The port number will auto-populate based on the protocol you set.
  • In the user name field, type your uniqname.
  • In the password field, type your UMICH (Level-1) password.
  • Click the “Login” button.
  • If “Continue connecting to an unknown server and add its host key to a cache?” pops up, click “Yes.”
  • A Duo two-factor authentication prompt will appear. Select your preferred method and authenticate with Duo.

Transferring Files

  • The left panel of winSCP will be your local machine, and the right panel of winSCP will be the remote machine you’ve connected to.
  • On your local machine, navigate to the file/directory you wish to transfer to the remote machine.
  • Drag the file/directory you wish to transfer to the remote machine’s panel. This will initiate the transfer.

SCP

From the command line:

SFTP

From the command line

Connect to a Remote Server Using SFTP

  • sftp uniqname@hostname
  • sftp user1@greatlakes.arc-ts.umich.edu

Transfer Local File/Dir to Remote System

  • get file /remote-directory

Transfer Remote File/Dir to Local Machine

  • get /remote-directory

Transfer Multiple Files/Dirs to Remote System

  • mput files /remote-directory

Transfer Multiple Files/Dirs to Local Machine

  • mget /remote-directory

What is the difference between NFS and CIFS?

  • NFS – “Network File System” is used for Unix and Linux based operating systems.
  • CIFS – “Common Internet File System” is used for Windows and OS X operating systems.

How do I have my volume mounted on Great Lakes?

Select the “Great Lakes” option under “Make Available On” section of the SRS form.

Is it possible to distribute charges for a volume across multiple Shortcodes?

Yes. If you are from Michigan Medicine (Med) or the College of Literature, Science, and the Arts (LSA) please contact your local unit support team to assist you with this process. If you are not in one of these units please contact ARC at arcts-support@umich.edu.

How do I add or remove people from my Turbo volume?

Send these requests to arcts-support@umich.edu. If your volume uses an AD group please indicate that in your email.

What are Unix Groups for?

In order to set appropriate permissions, and provide access for multiple users on our clusters, local Unix groups must be created by ARC to properly administer your NFS Turbo volume. Any user addition or deletions to Unix groups on the Great Lakes HPC Cluster or other ARC-controlled Unix system must be requested by contacting us at arcts-support@umich.edu.

What are Active Directory Groups for?

In order to set appropriate permissions for Windows centric volumes, and provide access for multiple users, Active Directory groups must be created by your local IT department to properly administer your CIFS volume.

We have functionality to control the user lists for an Active Directory group via the membership of an MCommunity Group. If you would like to set this functionality up so you can self-manage membership, please send us email at arcts-support@umich.edu.

How do I set permissions on my new Turbo volume?

The PI associated with the Turbo volume has full control to create directories and set permissions at the lowest level of the volume.

NFS

For NFS volumes, this can be done with standard Unix permission commands, such as:

How do I recover lost files from old snapshots?

Turbo volumes that are configured to use snapshots will save previous versions of files. Please be aware, only files that have been “snap-shotted” overnight can be recovered.

Linux: To recover files lost in your Turbo volume, navigate to the “.snapshots” directory at the root of your volume. Ex: /nfs/turbo/VOLUME-NAME/.snapshots/

$ cd /nfs/turbo/flux-admin/.snapshot
$ ls -1
daily_2019-08-24-_23-30
daily_2019-08-25-_23-30
daily_2019-08-26-_23-30
daily_2019-08-27-_23-30
daily_2019-08-28-_23-30
daily_2019-08-29-_23-30
daily_2015-08-30-_23-30

Windows:  You can recover lost files from snapshots natively:

  • Open the directory that the deleted file was held in.
  • Right click in the directory that the file or folder was stored and select “Properties”.
  • Click on the “Previous Versions” tab when the Properties window opens.
  • A list of snapshots will be displayed.
  • Select the snapshot from which you wish to restore data.
  • In the new window, locate the file(s) you wish to restore.
  • Simply drag the file(s) or folder to their correct locations

I’m new to storing data across multiple services like Turbo, Locker and Data Den.  How do I tell what service is best for the majority of the data I currently hold?

If you are unsure which of our storage services should be used to host your data, we have written some software that you can download and execute to analyze your files to understand how much of your data is stored in large files, how much of your data has been accessed recently, and the distribution of file sizes and access times. The software is accessible here.

This software doesn’t examine the contents of any data files, it merely scans file attributes, it also does not store any file names after searching through the filesystem.

If you have any questions on this software, or if you are unsure about any of the recommendations the tool sends you, please contact us at arcts-support@umich.edu.

Configuration
Turbo User Guide
Order Service
Back to Overview