CML: Difference between revisions

From UMIACS
Jump to navigation Jump to search
No edit summary
 
(118 intermediate revisions by 8 users not shown)
Line 1: Line 1:
The Center for Machine Learning ([https://ml.umd.edu CML]) at the University of Maryland is located within the Institute for Advanced Computer Studies.  The CML has a cluster of computational (CPU/GPU) resources that are available to be scheduled.
The Center for Machine Learning ([https://ml.umd.edu CML]) at the University of Maryland is located within the Institute for Advanced Computer Studies.  The CML has a cluster of computational (CPU/GPU) resources that are available to be scheduled.


=Data Storage=
<span style="font-size:150%">'''As of the [[MonthlyMaintenanceWindow | August 2023 maintenance window]], all compute nodes have moved into the [[Nexus]] cluster.''' Please see [[Nexus/CML]] for more details.</span>


Until the final storage investment arrives we have made available a temporary allocation of storage.  There are 3 types of storage available to users in the CML, home directories, project directories and scratch directories.
=Getting Started=
 
* [[SLURM/JobSubmission | Submitting Jobs]]
== Home Directories ==
* [[SLURM/JobStatus | Checking Job Status]]
* [[Nexus/CML#Storage | Data Storage]]
Home directories in the CML computational infrastructure are available from the Institutes [[NFShomes]].  These home directories have very limited storage and are intended for your personal files, configuration and source code.  Your home directory is '''not''' intended for data sets or other large scale data holdings.  Users are encouraged to utilize our [[GitLab]] infrastructure to host your code repositories.
 
This data has both snapshots and is backed up nightly.
 
== Project Directories ==
 
Users within the CML compute infrastructure can request project based allocations for up to 1TB for up to 120 days from staff@umiacs.umd.edu with approval from a CML faculty member and the director.  These allocations will be available from '''/fs/cml-projects''' under a name that you provide when you request the allocation.  Once the allocation period is over the user will be contacted and give a window of opportunity to clean and secure their data before staff will remove the allocation.
 
This data is backed up nightly.
 
== Scratch Directories ==
 
There are two types of scratch directories in the CML compute infrastructure, network and local scratch directories.  Scratch data has no data protection including no snapshots and the data is not backed up.
 
=== Network Scratch Directory===
Users granted access to the CML compute infrastructure are each allocated 200GB of network attached scratch.  This is available as <code>/fs/cml-scratch/USERNAME</code> where USERNAME is your username.
 
=== Local Scratch Directory===
Each computational node that a user can schedule compute jobs on has one or more local scratch directories.  These are always named <code>/scratch0</code>, <code>/scratch1</code>, etc.  These are almost always more performant than any other storage available to the job.  However users must stage their data within the confine of their job and stage the data out before the end of their job.

Latest revision as of 22:31, 17 August 2023

The Center for Machine Learning (CML) at the University of Maryland is located within the Institute for Advanced Computer Studies. The CML has a cluster of computational (CPU/GPU) resources that are available to be scheduled.

As of the August 2023 maintenance window, all compute nodes have moved into the Nexus cluster. Please see Nexus/CML for more details.

Getting Started

Retrieved from "https://wiki.umiacs.umd.edu/umiacs/index.php?title=CML&oldid=11225"