Category

News

Flux transfer servers will be decommissioned on Jan. 9; use new endpoints

By | Feature, General Interest, News

To provide faster transfers of data from ARC services, ARC will be decommissioning the Flux-Xfer servers on January 9, 2023. You will need to update how you migrate your data. 

For everyone who uses ARC storage services, especially Data Den users: This message is VERY important! This change includes the use of scp from Flux-Xfer, as well as the Globus endpoint umich#flux. Any shared endpoints that you have created on umich#flux will be automatically migrated to a new Globus collection on January 9. Those who use Data Den should take special interest in Item 1 listed below. 

Action item – Use the new endpoints

  1. If you currently use globus and umich#flux to access your Locker or Data Den volume, you should use the Globus Collection ‘UMich ARC Locker Non-Sensitive Volume Collection’ for Locker, and the Globus Collection ‘UMich ARC Data Den Non-Sensitive Volume Collection’ for Data Den. 
  2. If you currently use globus and umich#flux to access your Turbo volumes, you should use the Globus Collection ‘UMich ARC Turbo Non-Sensitive Volume Collection’.
  3. If you currently use globus and umich#flux to access other storage volumes, you should use the Globus Collection ‘umich#greatlakes’.  
  4. If you currently use scp on flux-xfer to copy data to/from Turbo, you should use ‘globus-xfer1.arc-ts.umich.edu’.

User guide 

How can we help you?

For assistance or questions, please contact ARC at arc-support@umich.edu

2023 Winter Maintenance & Globus File Transfer upgrade 

By | Feature, General Interest, Great Lakes, HPC, News, Systems and Services

Winter maintenance is coming up! See the details below. Reach out to arc-support@umich.edu with questions or if you need help. 

These services will be unavailable: 

  • Great Lakes – We will be updating Great Lakes on a rolling basis throughout December and beginning of January, and if successful, there should be no downtime or impact, with the following exceptions: 
    • Single precision GPUs (SPGPU) will be down Jan. 4-5 for networking maintenance. Those nodes will return back to production when maintenance has been completed and the nodes have been reloaded.
    • Customers will be notified via email of any changes to Great Lakes maintenance that will require downtime.
    • If unsuccessful, the Great Lakes maintenance will begin on Jan. 4-5, starting at 8am.  In either case, we will email everyone with the updated maintenance status.
  • Globus on the storage transfer nodes: Jan. 17-18.

Maintenance notes:

  • No downtime for ARC storage systems maintenance (Turbo, Locker, and Data Den).
  • Open OnDemand (OOD) users will need to re-login. Any existing jobs will continue to run and can be reconnected in the OOD portal.
  • Login servers will be updated, and the maintenance should not have any effect on most users. Those who are affected will be contacted directly by ARC. 
  • Copy any data and files that may be needed during maintenance to your local drive using Globus File Transfer before maintenance begins. 
  • Slurm email will be improved, providing  more detailed information about completed jobs.

Countdown to maintenance 

For Great Lakes HPC jobs, use the command “maxwalltime” to discover the amount of time remaining until maintenance begins. 

Jobs that request more walltime than remains until maintenance will automatically be queued and start once maintenance is complete. If the plan for Great Lakes maintenance is successful, any queued jobs will be able to run as usual (except for the SPGPU nodes as discussed above). Customers will be notified via email if downtime is required for Great Lakes.

Status updates and additional information

How can we help you?

For assistance or questions, please contact ARC at arc-support@umich.edu.

ARC Director Brock Palen spoke about Globus at SC22

By | General Interest, News

At the recent International Conference for High Performance Computing, Networking, Storage, and Analysis in Dallas (SC22) in Dallas, Tx., ARC Director Brock Palen spoke about how the University of Michigan is using Globus File Transfer. Globus allows the transfer of data between different storage systems, lab computers, and personal desktops/laptops.

“U-M is the largest public research university in North America by research spend. There are over a hundred top 10 graduate programs, and this drives a lot of interesting collaborations,” said Palen.  

Globus is a robust, cloud-based, file transfer service designed to move many large files, ranging from 10s of GBs to 10s of TBs. ARC is a Globus Subscription Provider for the U-M community, which allows U-M resources to serve as endpoints or collections for file transfers.

“Globus is the tool we use for data transfer, and it’s pretty much ubiquitous across Internet2, the national sites, other academic sites, and it interoperates everywhere,” Palen noted.  

Watch the YouTube video featuring Brock Palen

Check out the Globus user guide on the ARC website.

XSEDE is now ACCESS 

By | Feature, HPC, News

Access: advancing innovation and decorative imageThe national HPC resource known as XSEDE has now fully transitioned to ACCESS, Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support. 

ARC staff members, Todd Raeker and Shelly Johnson, are here to help you understand what is happening and how you can take advantage of the national resources and allocations.  

ACCESS has streamlined the allocation application process. Many projects that formerly had to submit an annual allocation proposal under XSEDE may now fit within one of the new ACCESS opportunities for small or medium-sized projects. You can now make one request with a much shorter allocations proposal. You can also make these requests at any time, rather than wait for the quarterly deadlines. Allocations are awarded for the duration of a supporting grant or for 12 months, with possible extensions up to five years.

There are new tiers as well. With the new Discover tier, there are 1.5 million available credits, and is closer in line to annual allocation from XSEDE. Researchers can submit the one-page application anytime. Check the overview of the ACCESS opportunities to see where your project fits. If you find that your research activities are considered to be a large-scale project, you will need to submit a proposal for a Maximize ACCESS award. Requests for Maximize ACCESS are accepted every six months. More details are available on the Maximize ACCESS web page.

Are you a grad student researcher? You now qualify to apply as the principal investigator (PI) for an ACCESS allocation in the Explore ACCESS tier to obtain resources to help complete your dissertation. Graduate students will need to include a letter from their advisor to support the request, and advisors must be added as a co-PI.

If you are interested in assisting the national research and educational community by reviewing ACCESS allocation requests, you can sign up to join the ACCESS allocations review committee.

Need help? Contact arc-support@umich.edu, or visit the ACCESS web page on the ARC website. You can also keep up with ongoing updates on the ACCESS webpage, access-ci.org.

Attend a listening session with ARC Director Brock Palen

By | Feature, HPC, News

You’re invited! ARC Director Brock Palen would like to hear from researchers. 

Are ARC services meeting their needs? What is not working well for them? Is something technical impeding their ability to do their research? Do they like Turbo Research Storage, the HPC web interface Open OnDemand, or the no-cost allocations offered by the U-M Research Computing Package

This is an open, virtual, drop-in office hour. All are welcome. 

There are three sessions are available: 

Researchers can also email Brock Palen at brockp@umich.edu or reach out to ARC at arc-support@umich.edu.

Precision Health and ARC team up on a self-service tool for genetic research

By | Great Lakes, HPC, News

Encore is a self-serve genetic analysis tool that researchers can now run using a point-and-click interface without the need to directly manipulate the genetic data. Only a phenotype file is needed to build a GWAS model with SAIGE (genetics analysis software), launch and monitor job progress, and interactively explore results.

It is geared for a range of disciplines and specialties including biostatistics, epidemiology, neuroscience, gastroenterology, anesthesiology, clinical pharmacy, and bioinformatics.

The tool was developed at the U-M School of Public Health Center for Statistical Genetics and is managed by Precision Health and supported by ITS’s Advanced Research Computing (ARC).  

Brock Palen, ARC director, “When someone uses Encore they are actually running on Great Lakes, and we are happy to provide the computational performance behind Encore.”

Using Encore is easy. No coding, command-line/Linux knowledge is required to run GWAS in Encore. Researchers also do not need to have knowledge of batch job submission or scheduling, or have direct access to a high-performance computing cluster. Encore automatically prepares job submission scripts and submits the analysis to the Great Lakes High-Performance Computing Cluster. 

Great Lakes is the university’s flagship open-science high-performance computing cluster. It is much faster and more powerful than a laptop, and provides quicker answers and optimized support for simulation, genomics, machine learning, life science, and more. The platform provides a balanced combination of computing power, I/O performance, storage capability, and accelerators.

Visit the Encore wiki page to learn more

To get started, send an email to PHDataHelp@umich.edu

For questions about Great Lakes, contact arc-support@umich.edu

Dailey receives U-M Robotics’ first-ever alumni award 

By | General Interest, Happenings, News, Research, Uncategorized

Meghan Dailey will be presenting The Future of Machine Learning in Robotics on September 23 at 2 p.m., at FMCRB or Zoom

Meghan Dailey is the U-M Robotics department’s first Alumni Merit Award recipient!

Dailey is a member of the first-ever class in U-M Robotics. She earned a Masters of Science degree in 2015 with a focus in artificial intelligence. She is currently a machine learning specialist with Advanced Research Computing (ARC), a division of Information and Technology Service (ITS)

You’re invited 

In honor of the award, Dailey will be presenting “The Future of Machine Learning in Robotics” on Friday, September 23, 2 p.m., Ford Robotics Building (FMCRB) or on Zoom (meeting ID: 961 1618 4387, passcode: 643563). Machine learning is becoming widely prevalent in many different fields, including robotics. In a future where robots and humans assist each other in completing tasks, what is the role of machine learning, and how should it evolve to effectively serve both humans and robots? Dailey will discuss her past experiences in robotics and machine learning, and how she envisions machine learning contributing to the growth of the robotics field.

About Dailey

A member of the ARC Scientific Computing and Research Consulting Services team, Dailey helps researchers with machine learning and artificial intelligence programming. She has consulted with student and faculty teams to build neural networks for image analysis and classification. She also has extensive experience in natural language processing and has worked on many projects analyzing text sentiment and intent.

Image courtesy U-M Robotics

New Resource Management Portal feature for Armis2 HPC Clusters

By | Armis2, HPC, News

Advanced Research Computing (ARC), a division of Information and Technology Services (ITS), has been developing a self-service tool called the Resource Management Portal (RMP) to give researchers and their delegates the ability to directly manage the IT research services they consume from ARC. 

Customers who use the Armis2 High-Performance Computing Cluster now have the ability to view their account information via the RMP, including the account name, resource limits (CPUs and GPUs), and the user access list.

“We are proud to be able to offer this tool for customers who use the HIPAA-certified Armis2 cluster,” said Brock Palen, ARC director. 

The RMP is a self-service-only user portal with tools and APIs for research managers, unit support staff, and delegates to manage their ARC IT resources. The RMP team is slowly adding capabilities over time. 

To get started or find help, contact arc-support@umich.edu.

Preserving Michigan’s musical history and culture

By | Feature, News, Research

From Kentucky bluegrass to Louisiana Zydeco to German hurdy-gurdy to East European Klezmer to Indian Manipuri dancing to Native American pow wows, and much more, these musical traditions from around the country and around the world have found their way to Michigan. Beginning in 2014, the Musical Heritage Project has been documenting Michigan’s folk music history.

Lester Monts

Lester Monts Lester Monts specializes in ethnomusicology and has been documenting Michigan’s folk cultural heritage since 2014. (Image courtesy Lester Monts)

The project is led by ethnomusicologist Dr. Lester P. Monts, Arthur F. Thurnau Professor Emeritus of Music, who began his musical journey as an orchestral trumpet player. He earned bachelor’s and master’s degrees in trumpet performance and teaching trumpet at the college level before completing the doctoral degree in ethnomusicology and embarking on a research career. In the mid-1970s, Monts began to focus his research on music and culture in Liberia and Sierra Leone in West Africa. The fourteen-year Liberian civil war thwarted his fieldwork in that region.

Noting that there has been no systematic effort to collect and archive Michigan’s rich folk music heritage, the Michigan Musical Heritage Project was launched. Monts has embraced the study of music from the cultural and social aspects of the people who make it. He notes that “music brings people together; it has the power to create community, and we witnessed this occurring throughout our many journeys around the state.”

Using his charm, passion, likeability, and keen musical knowledge to cultivate trust with his interviewees, Monts captured more than 400 hours of audio and video data over the years, amassing a total of 80 terabytes of data. He believes this to be the most extensive collection of Michigan folk music in the state and that U-M is the right place to house this collection.

The Michigan Musical Heritage Project crew.

The Michigan Musical Heritage Project crew wraps up at the end of recording session. (Image courtesy Lester Monts)

With a videography crew consisting primarily of former U-M students, Monts traveled all around the state to record performances at folk music festivals and cultural gatherings, such as the Celtic Festival (Saline), Irish Folk Music Festival (Muskegon) Hispanic Heritage Festival (Hart), Hiawatha Traditional Music Festival (Marquette), Port Sanilac Blues Festival (Port Sanilac), Africa World Festival (Detroit), Aura Jamboree (Aura), Oldtime Fiddlers Convention and Traditional Music Festival (Hillsdale).

He says, “The creative talents of the state’s outstanding musicians must be preserved, not only for my research but for that of others as well. If properly preserved, I’m confident that in the future, the ethnomusicology program and the American Cultures department will find these data provide important insights into Michigan’s diverse musical heritage.”

How technology supports this project 

Monts’ crew includes a strong partnership with Tom Bray, converging technologies consultant and adjunct assistant professor of Art and Design, Penny W. Stamps School of Art and Design. Bray has been instrumental in pairing the right technology for the long-term preservation of this collection, which includes converting older footage to digital media. 

Tom Bray

Tom Bray (image courtesy LSA)

Bray has collaborated with Monts to convert older technologies, such as VHS, 8mm, and high-8 video, to digital files. The files are both compressed and uncompressed and are very large and of high resolution.

All of this wonderful and important audio and video footage needs to be preserved somewhere. But where do you turn when you have 80 terabytes of data? Monts said, “I’ve been desperately searching for a way to archive the video data collected under the auspices of the Michigan Musical Heritage Project.” 

Enter the U-M Research Computing Package (UMRCP) and the team from Advanced Research Computing (ARC), a division of Information and Technology Services. The UMRCP offers researchers across all campuses several resources at no additional cost to researchers, including 100 terabytes of long-term storage.

Bray said, “I had to read the UMRCP email announcement twice because I couldn’t believe my eyes. I was so excited that ITS and the university are supporting researchers in this way. We jumped on this opportunity right away.” 

ARC Director Brock Palen is excited about this work, too. “This is super interesting, and not like the usual types of research ARC normally sees, like climate and genomics. We’re happy to help Dr. Monts and Mr. Bray, and anyone who needs it, anytime. The archive is intentionally built for holding large-volume, raw data such as 4k video, and we are proud to be their go-to for this important cultural preservation project.” 

Old media in Dr. Monts' office

Hours and hours of media is being converted to a digital format. (Photo by Stephanie Dascola)

ARC replicates and encrypts in two secure locations that are miles apart, so those who use ARC services will not have to worry about crashes that they might experience if they are using their own equipment. The UMRCP also includes technical expertise by talented ARC staff to further remove barriers so researchers can do what they do best.

Monts and Bray also leverage the university’s network and WiFi services to transfer the files from their studio in the Duderstadt Center to storage. The network is designed to minimize bottlenecks so that data transfers quickly and efficiently. 

Dr. Monts said, “Although the pandemic temporarily disrupted my plans to complete the video documentary, I take solace in knowing that the many hours of data we collected is in a much safer environment than we had. The UMRCP storage resource is truly a boon!”

Related links

An old reel-to-reel tape player.

A reel-to-reel tape player. (Photo by Stephanie Dascola)

Lester Monts plays footage from a special women's only dance in Iberia.

Dr. Monts shows footage from a special women-only dance in Iberia. He earned permission to record this rarely-documented group of women. (Photo by Stephanie Dascola)

No-cost research computing allocations now available

By | HPC, News, Research, Systems and Services, Uncategorized

U-M Research Computing PackageResearchers on all university campuses can now sign up for the U-M Research Computing Package, a new package of no-cost supercomputing resources provided by Information and Technology Services.

As of Sept. 1, university researchers have access to a base allocation for 80,000 CPU hours of high-performance computing and research storage services at no cost. This includes 10 terabytes of high-speed and 100 terabytes of archival storage.

These base allocations will meet the needs of approximately 75 percent of current high-performance-computing users and 90 percent of current research storage users. Researchers must sign up on ITS’s Advanced Research Computing website to receive the allocation.

“With support from President (Mark) Schlissel and executive leadership, this initiative provides a unified set of resources, both on campus and in the cloud, that meet the needs of the rich diversity of disciplines. Our goal is to encourage the use, support and availability of high-performance computing resources for the entire research community,” said Ravi Pendse, vice president for information technology and chief information officer.

The computing package was developed to meet needs across a diversity of disciplines and to provide options for long-term data management, sharing and protecting sensitive data, and more competitive cost structures that give faculty and research teams more flexibility to procure resources on short notice.

“It is incredibly important that we provide our research community with the tools necessary so they can use their experience and expertise to solve problems and drive innovation,” said Rebecca Cunningham, vice president for research and the William G. Barsan Collegiate Professor of Emergency Medicine. “The no-cost supercomputing resources provided by ITS and Vice President Pendse will greatly benefit our university community and the countless individuals who are positively impacted by their research.”

Ph.D. students may qualify for their own UMRCP resources depending on who is overseeing their research and their adviser relationship. Students should consult with their Ph.D. program administrator to determine their eligibility. ITS will confirm this status when a UMRCP request is submitted.

Undergraduate and master’s students do not currently qualify for their own UMRCP, but they can be added as users or administrators of another person’s UMRCP. Students can also access other ITS programs such as Great Lakes for Course Accounts, and Student Teams.

“If you’re a researcher at Michigan, these resources are available to you without financial impact. We’re going to make sure you have what you need to do your research. We’re investing in you as a researcher because you are what makes Michigan Research successful,” Brock Palen, Advanced Research Computing director.

Services that are needed beyond the base allocation provided by the UMRCP are available at reduced rates and are automatically available for all researchers on the Ann Arbor, Dearborn, Flint and Michigan Medicine campuses.

More Information