Tag

Locker

Flux transfer servers will be decommissioned on Jan. 9; use new endpoints

By | Feature, General Interest, News

To provide faster transfers of data from ARC services, ARC will be decommissioning the Flux-Xfer servers on January 9, 2023. You will need to update how you migrate your data. 

For everyone who uses ARC storage services, especially Data Den users: This message is VERY important! This change includes the use of scp from Flux-Xfer, as well as the Globus endpoint umich#flux. Any shared endpoints that you have created on umich#flux will be automatically migrated to a new Globus collection on January 9. Those who use Data Den should take special interest in Item 1 listed below. 

Action item – Use the new endpoints

  1. If you currently use globus and umich#flux to access your Locker or Data Den volume, you should use the Globus Collection ‘UMich ARC Locker Non-Sensitive Volume Collection’ for Locker, and the Globus Collection ‘UMich ARC Data Den Non-Sensitive Volume Collection’ for Data Den. 
  2. If you currently use globus and umich#flux to access your Turbo volumes, you should use the Globus Collection ‘UMich ARC Turbo Non-Sensitive Volume Collection’.
  3. If you currently use globus and umich#flux to access other storage volumes, you should use the Globus Collection ‘umich#greatlakes’.  
  4. If you currently use scp on flux-xfer to copy data to/from Turbo, you should use ‘globus-xfer1.arc-ts.umich.edu’.

User guide 

How can we help you?

For assistance or questions, please contact ARC at arc-support@umich.edu

ARC-TS seeks pilot users for two new research storage services

By | General Interest, Happenings, HPC, News

Advanced Research Computing – Technology Services (ARC-TS) is seeking pilot users for two new research storage services.

The first, Locker, is group project storage focused on large data sets, and is available at a cost less than half that of current primary storage services. Locker still provides encryption, replication, snapshots, and workstation access. Example use cases for Locker are research projects in climate studies, genomics, imaging, and other data-intensive sciences.

The second service, Data Den, provides archive class storage for research data that is not actively used. As our lowest cost research storage offering, Data Den provides “cold storage” for massive amounts of data with 20 petabytes of encrypted and replicated capacity. Data Den allows researchers to preserve data between rounds of funding and management plans, and to free up space in more expensive primary storage by moving valuable, but not currently used, data.

Those interested in participating in the pilots should contact ARC-TS at hpc-support@umich.edu.

Early user buy-in available for new Locker large-file storage service

By | General Interest, HPC, News

Advanced Research Computing – Technology Services (ARC-TS) will be deploying a new research storage service aimed at serving faculty in the Big Data era.  This service, called Locker, complements our existing Turbo general purpose storage service and planned archive service Data Den.

Locker is a large-file cost-optimized storage and is not good for general purpose / small file use.

Faculty can now buy in, at a one-time cost to bootstrap the service.  Faculty interested in this option ahead of the general service will need to commit to 200TB un-replicated or 100TB replicated, of space or more, at a one-time cost of $175/TB un-replicated, $350/TB replicated, for 5 years. This would be a minimum purchase of $35,000, with no further costs for 5 years.

If you are interested, please contact ARC-TS by July 10, 2017, at hpc-support@umich.edu.

Locker Service Timeline - Timeline(4)Frequently Asked Questions

Q: When will Locker be ready if I contribute funds to its launch?

A: Locker aims to be on site and ready for data by Fall semester (2017).

 

Q: When will Locker be ready as a monthly service?

A: The current timeline aims for early November 2017.

 

Q: What if I need less than 100TB replicated or 200TB un-replicated?

A: After the early period smaller allocations will be available.  Contact us at hpc-support@umich.edu to discuss your needs.

 

Q: Can I keep data beyond 5 years?

A: Yes. Options will exist beyond the first 5 years with new ongoing costs for support of the system.

 

Q: What is a large file for Locker?

A: A file size averaging 1MB or larger.  If files are smaller than this, Turbo or MiStorage are preferred options due to their SSD and other caching functionality.

 

Q: With what methods does one access Locker?

A: Locker will support NFS (v3/v4) and CIFS/SMB to workstations, servers, and clusters.

 

Q: Can I use Locker with Sensitive Data such as HIPAA/PHI?

A: Locker comes with encryption at rest and will eventually support HIPAA/PHI data and more.  It will NOT support sensitive data during the early user period. Sensitive data clearance work will start once the system is in place, and should be ready 2-3 months later.

 

Q: Can I pay for Locker monthly rather than up front?

A: Locker will eventually be a monthly service similar to Turbo, but during the early period we are looking for faculty to commit to a minimum amount of storage at a one-time cost (hardware only) to bootstrap the service and keep future prices low.

 

Q: Can I add more capacity?

A: Yes, you can request more capacity at any time.  Because of the design, larger requests will require a few weeks lead time.  To keep costs low, Locker does not maintain significant extra capacity idle, but can grow at anytime to sizes in the 10s of PB.

 

Q: What optional features exist?

A: The features are:

  • Optional Geographic Replication
  • Optional Snapshots

 

Q: Can I use this for clinical care / enterprise use cases?

A: No. Locker has a 9:00 a.m. – 5:00 p.m. support window and is not architected for enterprise availability.  We recommend using MiStorage for enterprise or comparable services in HITS for clinical care.

 

Q: Does Locker include backups?

A: Locker does not include backups.  It does include optional geographic replication and snapshots, which provide some protection against user deletion and major disaster but do not protect against software or administrator error the same way backups do.  For backups we recommend MiBackup.

 

Q: Who should use Locker?

A: Researchers whose datasets typical file size exceed 1 MB can use Locker to store their data more cost efficiently than other options at the University.

 

Q: Why should I contribute to the launch of Locker?

A: Locker aims to provide a cost effective solution for big data storage.  To do this a minimum amount of space needs to be allocated.  By contributing you secure the option of low cost storage for research going forward.