UMIACS - User contributions [en]

WebSpace

2025-10-30T20:32:51Z

Ekr597:

UMIACS provides static web space hosting for research/lab pages and user pages.

== '''Hosting websites in UMIACS Object Store ''(preferred method)''''' ==
Please refer to the section "Hosting a Website in your Bucket" on the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store Help Page] or visit [[OBJ/WebHosting]]. This is currently our most updated and reliable method for hosting websites.

==Main Website and Lab Pages==
<pre>http://www.umiacs.umd.edu</pre>

You can access the main website and lab sites for editing in two ways:
* From Unix as /fs/www - and can be remotely accessed by [[SFTP]] to a supported Unix host (e.g., [[Nexus]]).
* From Windows using [[WinSCP]].

Faculty members and authorized users can modify their own public profiles on the main UMIACS homepage. For instructions, see [[ContentManagement]].

==Personal Web Space==
Your personal website URL at UMIACS is

<pre>http://www.umiacs.umd.edu/~username</pre>

where '''username''' is your UMIACS username. You can set this page to redirect to any page of your choice by setting the '''Home Page''' attribute in your UMIACS [https://intranet.umiacs.umd.edu/directory/info/ directory entry].

In general, large files or directories for distribution related to a lab's research should go into the specific lab's web tree, not your individual web tree. Remember that your webpage is not permanently continued upon your departure from UMIACS.

UMIACS currently supports hosting a personal website on the Object Store.

===UMIACS Object Store===
This is the preferred method of hosting a personal website at UMIACS. Please see the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store (OBJ) Help Page] for more information on creating a website within OBJ. Once you create your website in OBJ, you will need to set your directory '''Home Page''' to the bucket's URL (the URL that ends in <code>umiacs.io</code>).

===Nexus File Space===
{{Note|'''''This service has been deprecated.'''''}}

This is primarily a legacy method for users who already have their websites configured this way. If you believe that your circumstances require your personal website to be hosted on this file space, please contact the [[HelpDesk | Help Desk]]. (This does not affect existing users who already have websites hosted on the Nexus file space.)

You will need set your directory '''Home Page''' attribute to <code>http://users.umiacs.umd.edu/~username</code>, where '''username''' is your UMIACS username (similar to your personal URL above). You can access your website for editing in two ways:

* From Unix as /fs/www-users/username - and can be remotely accessed by [[SFTP]] to a supported Unix host (e.g., [[Nexus]]).
* From Windows as \\umiacs-webftp\www-users\username

==Adding A Password Protected Folder To Your Web Space==
{{Note|'''''This method will NOT work in the UMIACS Object Store.'''''}}

1) Create the directory you want to password protect or <tt>cd</tt> into the directory you want to password protect.

2) Create a file called ''.htaccess'' (<tt> vi .htaccess</tt>) in the directory you wish to password protect.

3) In the file you just created, type the following lines

<pre>
AuthUserFile "/your/directory/here/".htpasswd
AuthName "Secure Document"
AuthType Basic
require user username
</pre>

For example, if you were going to protect the <tt>/fs/www-users/username/private</tt> directory and you want the required name to be <tt>class239</tt>, then your file would look like this:
<pre>
AuthUserFile /fs/www-users/username/private/.htpasswd
AuthName "Secure Document"
AuthType Basic
require user class239
</pre>

4) Create a file called ''.htpasswd'' in the same directory as ''.htaccess''. You create this file by typing in <tt>htpasswd -c .htpasswd ''username''</tt> in the directory area to be protected.

In the example above, the username is <tt>class239</tt> so you would type <tt>htpasswd -c .htpasswd class239</tt>

You will be prompted to enter the password you want. The ''.htpasswd'' file will be created in the current directory and will contain an encrypted version of the password.

To later change the username, edit the ''.htaccess'' file and change the username. If you want to later change the password, just retype the above line in step 4 and enter the new password at the prompt.

==Restricting Content based on IP address==
It is possible to have pages on your webspace only accessible to clients connecting from certain IP addresses. In order to accomplish this, cd in to the directory you wish to restrict, and edit your ''.htaccess'' or ''httpd.conf'' file. The example below shows how to make content only viewable to clients connecting from the UMD WiFi in Apache 2.2.

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">SetEnvIF X-Forwarded-For "^128\.8\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^129\.2\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^192\.168\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^206\.196\.(?:1[6-9][0-9]|2[0-5][0-9])\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^10\.\d+\.\d+\.\d+$" UMD_NETWORK
Order Deny,Allow
Deny from all
Allow from env=UMD_NETWORK
</pre>

The SetEnvIF directive will modify one's environment if the specified attribute matches the provided regular expression. In this example, IP addresses that are forwarded from an IP within UMD's IP space are tagged with UMD_NETWORK. Then, all traffic to the example directory is blocked unless it has the UMD_NETWORK tag. See the following pages for a more in depth explanation of the commands used.

[https://httpd.apache.org/docs/2.2/howto/htaccess.html .htaccess], [https://httpd.apache.org/docs/2.2/mod/mod_setenvif.html#setenvif SetEnvIf], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#order Order], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#deny Deny], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#allow Allow]

FilesystemDataStorage

2025-09-24T19:28:55Z

Ekr597:

Filesystem [[Data Storage | (data) storage]] refers to all data that is stored physically at UMIACS, i.e., on hard drives either in servers in datacenters managed by [[HelpDesk | UMIACS staff]], or in UMIACS-supported workstations. The opposite of this is [[CloudDataStorage | cloud storage]] which is stored on third-party providers' data hosting platforms.

The below sections outline the different categories of filesystem storage available at UMIACS. Although technically filesystem storage by the above definition, UMIACS also hosts an [[OBJ | Object Store]] that is documented outside the scope of this page.

==Network Home Directory Filesystem Storage==
We provide network home directory filesystem storage to each of our users through [[NFShomes]] home directories.

This home directory can be accessed via <code>/nfshomes/<username></code> on supported UNIX machines or by using [[WinSCP]] on Windows machines.

Each home directory is backed up nightly using the Institute's [[TSM]] backup system. It also has [[Snapshots]] enabled for easy user restores.

Users are given a 30GB, non-expandable [[Quota]]. You will need to use either platform-specific filesystem storage, directly-attached filesystem storage, or other network-attached filesystem storage for increased space.

On user account closure, the account's NFShomes home directory goes into our [[Archives]].

==UNIX Filesystem Storage==
UNIX hosts use redundant, backed-up network file shares for user home directories ([[#Network Home Directory Filesystem Storage |above section]]). Research data storage ([[#Network-Attached Filesystem Storage |below section]]) is also stored on redundant, possibly-backed-up network file shares and is generally available under /fs/

All UNIX hosts also have filesystem storage available for transitory use. These directories may be used to store temporary '''''COPIES''''' of data that is permanently stored elsewhere or as a staging point for output.

These directories may not, '''''under any circumstances''''', be used as permanent storage for unique, important data. They are not backed up or archived by UMIACS. UMIACS staff cannot recover damaged or deleted data from these directories and will not be responsible for data loss if they are misused. Additionally, on our [[SLURM]] compute clusters, these volumes may have an automated cleanup routine that will delete unmodified data after some number of days. You can check the page for the specific cluster you are using for more information.

Please note that '''/tmp''' in particular is at risk for data loss or corruption as that directory is regularly used by system processes and services for temporary storage.

These directories include:

- /tmp
- /scratch0, /scratch1, ... (/scratch#)
- any directory named in whole or in part "tmp", "temp", or "scratch".

==Windows and macOS Filesystem Storage==
Windows and macOS hosts at UMIACS store user directories on their primary internal drives (<tt>C:\Users</tt> for Windows, <tt>/Users</tt> for macOS). Supported, UMIACS-managed hosts automatically back up user data on these drives nightly using the Institute's [[TSM]] backup system. If you have a supported, UMIACS-managed host that has other internal or external hard drives attached to it, or other partitions on its primary internal drive, please be aware that these drives/partitions '''are not''' backed up. Laptops and non-standard hosts are not automatically backed up and should be manually backed up by their users.

On host decommission, user directories go into our [[Archives]].

==Direct-Attached Filesystem Storage==
Direct-attached filesystem storage refers to devices like USB flash drives and USB hard drives, which are very popular for easily expanding storage capacity on a host. However, these devices are significantly more vulnerable to data loss or theft than internal or networked data storage. In general, UMIACS discourages the use of direct-attached filesystem storage when any other option is available. Please note that these devices are prone to high rates of failure and additional steps should be taken to ensure that the data is backed up and that critical or confidential data is not lost or stolen.

Direct-attached filesystem storage is not backed up or archived by UMIACS.

==Network-Attached Filesystem Storage==
Some labs have network-attached filesystem storage space dedicated for datasets, models, and project storage. These shares are typically named in the form <tt>/fs/<lab>-<purpose></tt> (i.e., <tt>/fs/cml-models</tt> or <tt>/fs/vulcan-projects</tt>).

Network-attached filesystem storage may or may not be backed up and/or archived by UMIACS. Details of a specific share's retention policy should be stated along with the documentation of the share's access / usage policy. If you find documentation for a network-attached filesystem storage space in this wiki that does not state its retention policy, please [[HelpDesk | contact staff]].

===Network-Attached Filesystem Scratch Storage===
One specific sub-category of network-attached filesystem storage is network-attached filesystem scratch storage. These shares are named similarly to UNIX filesystem storage, but with the lab's name included (i.e., <tt>/fs/cbcb-scratch</tt> or <tt>/gammascratch</tt>), are intended for scratch/temporary storage, and are subject to the same policies as filesystem scratch/tmp directories, discussed above.

Network-attached filesystem scratch storage is not backed up or archived by UMIACS.

==UNIX Filesystem Storage Commands==
Below are a few different CLI commands that may prove useful for monitoring your filesystem storage usage and performance. For additional information, run <code>[command] --help</code> or <code>man [command]</code>

df - Shows descriptive file system information
<pre>
Usage: df [OPTION]... [FILE]...
Show information about the file system on which each FILE resides,
or all file systems by default.
</pre>
For example, to check how much space is available at a directory:
<pre>df -h ./</pre>

du - Shows disk usage of specific files. Use the -d flag for better depth control.
<pre>
Usage: du [OPTION]... [FILE]...
or: du [OPTION]... --files0-from=F
Summarize disk usage of each FILE, recursively for directories.
</pre>
For example, to check how much space each file in a directory takes up:
<pre>du -ah -d 1 ./</pre>

free - Shows current memory(RAM) usage. Use the -h flag for a human readable format.
<pre>
Usage:
free [options]
</pre>

quota - Shows quota information, this is useful for viewing per filesystem limits in places such as a home directory.
<pre>
quota: Usage: quota [-guqvswim] [-l | [-Q | -A]] [-F quotaformat]
quota [-qvswim] [-l | [-Q | -A]] [-F quotaformat] -u username ...
quota [-qvswim] [-l | [-Q | -A]] [-F quotaformat] -g groupname ...
quota [-qvswugQm] [-F quotaformat] -f filesystem ...
</pre>

iostat - Shows drive utilization, as well as other utilizations. Pair this with the <code>watch</code> command for regular updates.
<pre>
Usage: iostat [ options ] [ <interval> [ <count> ] ]
Options are:
[ -c ] [ -d ] [ -h ] [ -k | -m ] [ -N ] [ -t ] [ -V ] [ -x ] [ -y ] [ -z ]
[ -j { ID | LABEL | PATH | UUID | ... } ]
[ [ -T ] -g <group_name> ] [ -p [ <device> [,...] | ALL ] ]
[ <device> [...] | ALL ]
</pre>

FileTransferProtocol

2025-09-24T18:33:37Z

Ekr597:

{{Note|'''Our FTP service is deprecated in favor of the UMIACS Object Store. Please see [[OBJ]]. Depending on when your account was installed, you may or may not have FTP access.'''}}

UMIACS provides FTP services for transferring data to external collaborators. Since the FTP protocol is conducted entirely in plaintext, external users login to the service as anonymous, and internal users can access the file directories internally. Users will never authenticate over FTP with their UMIACS account. Please see [[SFTP]] for more information on a secure file transfer protocol.

==Publishing datasets via FTP==
Users can place data to be externally accessible in their public FTP space, which is located from the FTP service as

<pre>ftp://ftp.umiacs.umd.edu/pub/<username>/</pre>

To upload data to your public site, you can use
* '''/fs/ftp/pub/<username>''' from supported UNIX machines
* [[WinSCP]] from supported Windows machines

NASProjects

2025-09-24T18:28:58Z

Ekr597:

===Project Directories===
UMIACS labs distribute shared data to workgroups onsite and outside UMIACS using a wide variety of protocols including [[FTP]], [[SFTP]], [[SCP]], and [[NFS]].

On-site clients can access project directories through the Network File System ([[NFS]]) protocol. Supported UNIX workstations map project directories to

/fs/

Windows workstations can access these directories using a [[SFTP]] client, such as [[WinSCP]].

Project data storage may not be shared by default with every host. Please send mail to [[HelpDesk | staff]] to configure a new share.

Clients can also access data using authenticated File Transfer Protocol ([[FTP]]), Secure File Transfer Protocol ([[SFTP]]), and Secure Copy ([[SCP]]) through UMIACS file servers.

===Usage Guidelines===
Project Directories using Network attached Storage ([[NAS]]) are tuned for your workgroup's requirements and preferences.

Please avoid storing personal data in project storage allocations. Separating project data from personal data will simplify administration and data management for both researchers and staff.

UMobj/Example

2025-09-11T18:47:27Z

Ekr597: Deprecated, see MinIO Client

{{Note|'''UMobj has been deprecated in favor of the MinIO Client. Please see [[MinIO_Client]]'''}}

In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]).
<pre>
export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>"
export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>"
export OBJ_SERVER="obj.umiacs.umd.edu"
</pre>

We will be uploading this 4.4GB file.
<pre>
# stat CASIA-WebFace.tar.gz
File: `CASIA-WebFace.tar.gz'
Size: 4018119666 Blocks: 7847896 IO Block: 262144 regular file
Device: 13h/19d Inode: 87161 Links: 1
Access: (0644/-rw-r--r--) Uid: (10001/username) Gid: (18001/groupname)
Access: 2015-11-24 19:34:05.715695000 -0500
Modify: 2015-11-23 14:23:35.872087000 -0500
Change: 2015-11-23 16:07:10.062306000 -0500
</pre>

If the bucket does not exist yet you need to create the bucket first.
<pre>
# mkobj janus_datasets:
Created bucket janus_datasets.
</pre>

Now you can copy the file to the bucket.
<pre>
# cpobj CASIA-WebFace.tar.gz janus_datasets:
100% |#########################################################################################################################################|
</pre>

To verify the object store has the data expected we can compare the checksums. First you can calculate the md5sum locally and then use cmpobj to stream the data from the object store and create a md5sum. They should match.

<pre>
# md5sum CASIA-WebFace.tar.gz
004c2475e5ed66771e8873be67a93105 CASIA-WebFace.tar.gz
# cmpobj janus_datasets:CASIA-WebFace.tar.gz
004c2475e5ed66771e8873be67a93105
</pre>

UMobj

2025-09-11T18:46:43Z

Ekr597: Deprecated, see MinIO Client

{{Note|'''UMobj has been deprecated in favor of the MinIO Client. Please see [[MinIO_Client]]'''}}

The UMobj suite of utilities provide command-line access to the [https://obj.umiacs.umd.edu/obj UMIACS Object Store]. UMobj is preinstalled on all UMIACS-supported RHEL8 machines. For other operating systems or non UMIACS-supported hosts, we encourage use of one of the many [[S3Clients#Command_Line_Clients | third-party command line clients]] that exist.

==When to use UMobj==
Use umobj if:
* you have a large number of files to upload (hundreds or thousands of files)
* you are uploading large files (files greater than 4GB in size)

==Setup==
We highly recommend setting a few environmental variables containing your credentials for
convenience. When logged into the Object Store web interface (see list above), you can
find these credentials on the user page. E.g. https://obj.umiacs.umd.edu/obj/user/

For example, if you use the <tt>bash</tt> shell, you can add something like the following to your
<tt>.bashrc</tt> or <tt>.bash_profile</tt>.

<pre>
export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>"
export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>"
export OBJ_SERVER="obj.umiacs.umd.edu"
</pre>

Or, in tcsh, you can do the following or add it into your <tt>.tcshrc</tt>.

<pre>
setenv OBJ_ACCESS_KEY_ID "<ACCESS_KEY>"
setenv OBJ_SECRET_ACCESS_KEY "<SECRET_KEY>"
setenv OBJ_SERVER "obj.umiacs.umd.edu"
</pre>

(substituting in your actual <ACCESS_KEY> and <SECRET_KEY> for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]).

==Detailed Usage==
For an example of how to use UMobj, please see [[UMobj/Example]].

For complete usage information, please see the [[GitLab]] page for [https://gitlab.umiacs.umd.edu/staff/umobj/blob/master/README.md#umobj umobj].

MinIO Client

2025-09-11T18:38:18Z

Ekr597: Created page

The MinIO Client is a comprehensive single binary (Go) command line client for cloud based storage services.

==Installing the MinIO Client(mc)==
mc is available on all UMIACS-supported RHEL machines through adding it via our software [[Modules|module tree]].
:<pre>module add mc</pre>
For other operating systems or non UMIACS-supported hosts, the instructions to download and install it are found at https://docs.min.io/enterprise/aistor-object-store/reference/cli/#quickstart.

==Setup the MinIO Client(mc)==
To connect to Obj you'll need to configure mc by adding a host with your ACCESS_KEY and SECRET_KEY.
* The ACCESS_KEY and SECRET_KEY can either be for your personal account or a [[OBJ#LabGroups | LabGroup]]. You can find these in the [https://obj.umiacs.umd.edu/obj/user/ Object Store].

Running the following command to create the host "obj".
:<pre>mc config host add obj http://obj.umiacs.umd.edu <ACCESS_KEY> <SECRET_KEY></pre>

You can see what host(s) you have configured with the command <code>mc config host ls</code>.

<pre>
$ mc config host ls
...
obj
URL : http://obj.umiacs.umd.edu
AccessKey : (redacted)
SecretKey : (redacted)
API : s3v4
Path : auto
...
</pre>

==Running the MinIO Client(mc)==
To use most commands, specify the host you created and bucket name. For example the following to list the contents of the bucket "iso".
<pre>
$ mc ls obj/iso
[2017-02-10 16:45:04 EST] 3.5GiB rhel-server-7.3-x86_64-dvd.iso
[2017-02-13 12:21:33 EST] 4.0GiB rhel-workstation-7.3-x86_64-dvd.iso
</pre>

The full MinIO Client documentation can be found here: https://docs.min.io/enterprise/aistor-object-store/reference/cli/#command-quick-reference.

==Troubleshooting==
If you receive an error message, please check if it matches one of the error messages below. If you don't see the error message below, please reach out to the [[HelpDesk]].
* The provided 'x-amz-content-sha256' header does not match what was computed.
** The MinIO Client has issues handling empty files. You can have it use an older api to handle these files by re-running the config command and specifying the api. <pre>mc config host add obj http://obj.umiacs.umd.edu <ACCESS_KEY> <SECRET_KEY> --api S3v2</pre>

File:WinSCP edit properties.png

2025-06-25T19:05:03Z

Ekr597:

File:WinSCP remote properties.png

2025-06-25T19:03:38Z

Ekr597:

WinSCP

2025-06-25T18:11:25Z

Ekr597:

WinSCP is a free Windows file transfer application used to copy data to and from a remote host.

WinSCP is installed on all UMIACS-supported Windows workstations.

For all other Windows hosts, WinSCP can be downloaded from https://winscp.net/eng/index.php.

==Logging Into WinSCP==
When launching WinSCP, it asks you to log into a remote host.
* If you would like to upload files to [[OBJ]], follow the login instructions on the [[S3Clients#WinSCP | S3 Clients]] page.

To access a UMIACS directory, such as your [[NFShomes]] directory, [[WebSpace | legacy web page directories]], or [[FileTransferProtocol | legacy FTP directories]]:
* For the File protocol, select SFTP.
* For the host name, enter your [[Nexus#Access | Nexus submission node]].
* Enter your UMD username and password.

[[Image:WinSCP_sftp_login.png|500px]]

After logging in, WinSCP shows two directories.

The left side shows a directory on your computer and the right side shows a directory on the remote host.

==Changing Directories==
===Changing the Local Directory===
To change the directory on your computer, select "Open Directory" from the "Local" menu.

This controls the section on the left-hand side.

[[Image:WinSCP_change_local_dir.png|500px]]

===Changing the Remote Directory===
To change the UMIACS directory, select "Open Directory" from the "Remote" menu.

This controls the section on the right-hand side.

[[Image:WinSCP_change_remote_dir.png|600px]]

When changing the remote directory, you have to put in the path to that directory.

Below is a list of frequently used directories and their paths.

If you would like to access a specific directory and you don't know the path, please contact the [[HelpDesk]].

{| class="wikitable sortable" cellpadding="20"
! Name
! Path
|-
| NFShomes
| /fs/nfshomes/<username>
|-
| Nexus Scratch
| /fs/nexus-scratch/<username>
|-
| www-umiacs
| /fs/www
|-
| www-users
| /fs/www-users
|-
| ftp-umiacs
| /fs/ftp
|}

==Transferring Files==
To transfer files between your computer and the remote directory, simply drag the files from one side to the other.

You can also select the files and use the corresponding button.
===Uploading Files===
To upload files from your computer to the remote directory, select one or more files on the left-hand side and then click the Upload button.

[[Image:WinSCP_upload_local_file.png|300px]]

After clicking the Upload button, another window pops up. You can enter a different directory to change where the files are uploaded. Click OK to start the upload.

===Downloading Files===
To download files from the remote directory to your computer, select one or more files on the right-hand side and then click the Download button.

[[Image:WinSCP_download_remote_file.png|300px]]

After clicking the Download button, another window pops up. You can enter a different directory to change where the files are downloaded. Click OK to start the download.

==Opening Files==
Double clicking on a file will open the file in a text editor. This works for small, simple files, such as .txt and .html files.

To open more complex files, such as .pdf files, select the file and then click "Open" from the "Files" menu.

[[Image:WinSCP_open_file.png|500px]]

==Saving the Workspace==
If you frequently move files from your computer to a specific directory, you can save the workspace so it will open to that directory when you launch WinSCP.

First, change the local and remote directories to the directories you want using the instructions above.

Then, click "Save Workspace" under the "Tabs" menu.

[[Image:WinSCP_save_workspace.png|300px]]

Another window pops up letting you set the name of the workspace.

You can check the box next to "Create desktop shortcut" so it will save an icon on your desktop that will open up to the saved directories.

[[Image:WinSCP_save_workspace_window.png|300px]]

WinSCP

2025-06-25T15:08:18Z

Ekr597:

WinSCP is a free Windows file transfer application used to copy data to and from a remote host.

WinSCP is installed on all UMIACS-supported Windows workstations.

For all other Windows hosts, WinSCP can be downloaded from https://winscp.net/eng/index.php.

==Logging Into WinSCP==
When launching WinSCP, it asks you to log into a remote host.
* If you would like to upload files to [[OBJ]], follow the login instructions on the [[S3Clients#WinSCP | S3 Clients]] page.

To access a UMIACS directory, such as /fs/nexus-scratch:
* For the File protocol, select SFTP.
* For the host name, enter your [[Nexus#Access | Nexus submission node]].
* Enter your UMD username and password.

[[Image:WinSCP_sftp_login.png|500px]]

After logging in, WinSCP shows two directories.

The left side shows a directory on your computer and the right side shows a directory on the remote host.

==Changing Directories==
===Changing the Local Directory===
To change the directory on your computer, select "Open Directory" from the "Local" menu.

This controls the section on the left-hand side.

[[Image:WinSCP_change_local_dir.png|500px]]

===Changing the Remote Directory===
To change the UMIACS directory, select "Open Directory" from the "Remote" menu.

This controls the section on the right-hand side.

[[Image:WinSCP_change_remote_dir.png|600px]]

==Transferring Files==
To transfer files between your computer and the remote directory, simply drag the files from one side to the other.

You can also select the files and use the corresponding button.
===Uploading Files===
To upload files from your computer to the remote directory, select one or more files on the left-hand side and then click the Upload button.

[[Image:WinSCP_upload_local_file.png|300px]]

After clicking the Upload button, another window pops up. You can enter a different directory to change where the files are uploaded. Click OK to start the upload.

===Downloading Files===
To download files from the remote directory to your computer, select one or more files on the right-hand side and then click the Download button.

[[Image:WinSCP_download_remote_file.png|300px]]

After clicking the Download button, another window pops up. You can enter a different directory to change where the files are downloaded. Click OK to start the download.

==Opening Files==
Double clicking on a file will open the file in a text editor. This works for small, simple files, such as .txt and .html files.

To open more complex files, such as .pdf files, select the file and then click "Open" from the "Files" menu.

[[Image:WinSCP_open_file.png|500px]]

==Saving the Workspace==
If you frequently move files from your computer to a specific directory, you can save the workspace so it will open to that directory when you launch WinSCP.

First, change the local and remote directories to the directories you want using the instructions above.

Then, click "Save Workspace" under the "Tabs" menu.

[[Image:WinSCP_save_workspace.png|300px]]

Another window pops up letting you set the name of the workspace.

You can check the box next to "Create desktop shortcut" so it will save an icon on your desktop that will open up to the saved directories.

[[Image:WinSCP_save_workspace_window.png|300px]]

File:WinSCP save workspace window.png

2025-06-25T15:04:25Z

Ekr597:

File:WinSCP save workspace.png

2025-06-25T14:47:47Z

Ekr597:

WinSCP

2025-06-25T05:32:37Z

Ekr597:

File:WinSCP open file.png

2025-06-25T05:29:40Z

Ekr597:

WinSCP

2025-06-25T03:55:34Z

Ekr597:

WinSCP

2025-06-24T23:43:48Z

Ekr597: Created page with "WinSCP is a free Windows file transfer application used to copy data to and from a remote host. WinSCP is installed on all UMIACS-supported Windows workstations. For all other Windows hosts, WinSCP can be downloaded from https://winscp.net/eng/index.php. ==Using WinSCP== When launching WinSCP, it will ask you to login to a remote host. * If you would like to upload files to OBJ, follow the login instructions on the S3 Clients page. To acce..."

File:WinSCP download remote file.png

2025-06-24T23:42:28Z

Ekr597:

File:WinSCP upload local file.png

2025-06-24T23:39:30Z

Ekr597:

File:WinSCP change remote dir.png

2025-06-24T17:53:37Z

Ekr597: Showing where to change the remote directory in WinSCP

== Summary ==
Showing where to change the remote directory in WinSCP

File:WinSCP change local dir.png

2025-06-24T17:53:01Z

Ekr597: Showing where to change the local directory in WinSCP

== Summary ==
Showing where to change the local directory in WinSCP

File:WinSCP sftp login.png

2025-06-24T17:51:31Z

Ekr597: WinSCP login prompt, logging in as SFTP

== Summary ==
WinSCP login prompt, logging in as SFTP

S3Clients

2024-10-10T20:39:09Z

Ekr597:

Many popular S3 desktop clients can be used to access the [[OBJ | UMIACS Object Store]]. These tools complement the [[UMobj]] command line utilities and the built-in web interface by providing integration with the native file explorer on your desktop machine.

'''Note''': Many of these clients have features that are not supported by our Object Store in UMIACS. One prominent example of this is permissions. We suggest you instead manage permissions from the [https://obj.umiacs.umd.edu/obj built-in web application] for the Object Store.

=Graphical Clients=

==Cyberduck==
https://cyberduck.io/

This is a free Windows and Mac S3 browser (it is however nagware that asks for a donation). It supports our S3 Object Store using the "S3 (Amazon Simple Storage Service)" drop down menu choice in the add bookmark dialog.

[[Image:Cyberduck.png|400px]]

The following fields are required:
* '''Server''' - This is your object store (<code>obj.umiacs.umd.edu</code>)
* '''Access Key ID''' - This is your access key as provided to you in the object store
* '''Password''' - This is your secret key as provided to you in the object store

You will be prompted for your secret key when you connect and may choose to save the password.

==WinSCP==
* https://winscp.net/eng/index.php

This is a free Windows file transfer application. It supports our S3 Object Store using the "Amazon S3" drop down menu choice under File protocol when logging in.

[[Image:WinSCPS3.png|400px]]

The following fields are required:
* '''Host name''' - This is your object store (<code>obj.umiacs.umd.edu</code>)
* '''Access key ID''' - This is your access key as provided to you in the object store
* '''Secret access key''' - This is your secret key as provided to you in the object store

==Transmit==
* http://panic.com/transmit/
This is a paid file transfer application for Mac. It supports our S3 Object Store using the "S3" menu choice after clicking the plus sign to add a favorite.

[[Image:Transmit.png|400px]]

The following fields are required:
* '''Server''' - This is your object store (<code>obj.umiacs.umd.edu</code>)
* '''Access Key ID''' - This is your access key as provided to you in the object store
* '''Secret''' - This is your secret key as provided to you in the object store

These settings can be saved as a favorite for easy access. Transmit also allows you to mount your Obj buckets as local disks, which will support easy drag-and-drop of files.

=Command Line Clients=
==s3cmd==
Command line client for accessing S3-like services.

* http://s3tools.org/s3cmd

You need to configure a file like <code>~/.s3cfg</code> that looks like the following with your ACCESS_KEY and SECRET_KEY substituted.

<pre>
[default]
access_key = <ACCESS_KEY>
host_base = obj.umiacs.umd.edu
host_bucket = %(bucket)s.obj.umiacs.umd.edu
secret_key = <SECRET_KEY>
use_https = True
</pre>

==mc==
The MinIO Client is a comprehensive single binary (Go) command line client for cloud based storage services.

* https://min.io/download

You can run this client on supported UMIACS systems through adding it via our software [[Modules|module]].
<pre>module add mc</pre>

You will need to setup a cloud provider for Obj by running the following command (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]).

<pre>
mc config host add obj http://obj.umiacs.umd.edu <ACCESS_KEY> <SECRET_KEY>
</pre>

You can see what host(s) you have configured with the command <code>mc config host ls</code>.

<pre>
$ mc config host ls
...
obj
URL : http://obj.umiacs.umd.edu
AccessKey : (redacted)
SecretKey : (redacted)
API : s3v4
Path : auto
...
</pre>

You can then use the normal <code>mc</code> commands like the following to list the contents of a bucket.

<pre>
$ mc ls obj/iso
[2017-02-10 16:45:04 EST] 3.5GiB rhel-server-7.3-x86_64-dvd.iso
[2017-02-13 12:21:33 EST] 4.0GiB rhel-workstation-7.3-x86_64-dvd.iso
</pre>

There is also the ability to search for file globs of specific files using the <code>find</code> sub-command for <code>mc</code>.

<pre>
$ mc find derek/derek_support --name "*.log"
derek/derek_support/mds_20170918/ceph-mds.objmds01.log
derek/derek_support/satellite.log
derek/derek_support/umiacs-49168.log
</pre>

The full MinIO Client documentation can be found here: https://min.io/docs/minio/linux/reference/minio-mc.html.

Template:Nfshomes

2024-08-22T17:21:12Z

Ekr597: Updated the command to not use quota

You have 30GB of home directory storage available at <code>/nfshomes/<username></code>. It has both [[Snapshots]] and [[TSM | Backups]] enabled.

Home directories are intended to store personal or configuration files only. We encourage you to not share any data in your home directory. You are encouraged to utilize our [[GitLab]] infrastructure to host your code repositories.

'''NOTE''': To check your quota on this directory, use the command <code>df -h ~</code>.

Nexus/GAMMA

2024-06-25T18:25:22Z

Ekr597: Added project limits

The [https://gamma.umd.edu/ GAMMA] lab has a partition of GPU nodes available in the [[Nexus]]. Only GAMMA lab members are able to run non-interruptible jobs on these nodes.

=Access=
You can always find out what hosts you have access to submit via the [[Nexus#Access]] page. The GAMMA lab in particular has a special submission host that has additional local storage available.
* <code>nexusgamma00.umiacs.umd.edu</code>

Please do not run anything on the login node. Always allocate yourself machines on the compute nodes (see instructions below) to run any job.

=Quality of Service=
GAMMA users have access to all of the [[Nexus#Quality_of_Service_.28QoS.29 | standard job QoSes]] in the <code>gamma</code> partition using the <code>gamma</code> account.

The additional job QoSes for the GAMMA partition specifically are:
* <code>huge-long</code>: Allows for longer jobs using higher overall resources.

Please note that the partition has a <code>GrpTRES</code> limit of 100% of the available cores/RAM on the partition-specific nodes in aggregate plus 50% of the available cores/RAM on legacy## nodes in aggregate, so your job may need to wait if all available cores/RAM (or GPUs) are in use.

=Hardware=
{| class="wikitable sortable"
! Nodenames
! Type
! Quantity
! CPU cores per node
! Memory per node
! GPUs per node
|-
|gammagpu[00-04,06-09]
|A5000 GPU Node
|9
|32
|256GB
|8
|-
|gammagpu05
|A4000 GPU Node
|1
|32
|256GB
|8
|- class="sortbottom"
|
!Total
|10
|320
|2560GB
|80
|}

=Example=
From <code>nexusgamma00.umiacs.umd.edu</code> you can run the following example to submit an interactive job. Please note that you need to specify the <code>--partition</code> and <code>--account</code>. Please refer to our [[SLURM]] documentation about about how to further customize your submissions including making a batch submission. The following command will allocate 8 GPUs for 2 days in an interactive session. Change parameters accordingly to your needs. We discourage use of srun and promote use of sbatch for fair use of GPUs.

<pre>
$ srun --pty --gres=gpu:8 --account=gamma --partition=gamma --qos=huge-long bash
$ hostname
gammagpu01.umiacs.umd.edu
$ nvidia-smi -L
GPU 0: NVIDIA RTX A5000 (UUID: GPU-cdfb2e0c-d69f-354b-02f4-15161dc7fa66)
GPU 1: NVIDIA RTX A5000 (UUID: GPU-be53e7a1-b8fd-7089-3cac-7a2fbf4ec7dd)
GPU 2: NVIDIA RTX A5000 (UUID: GPU-774efbb1-d7ec-a0bb-e992-da9d1fa6b193)
GPU 3: NVIDIA RTX A5000 (UUID: GPU-d1692181-c7de-e273-5f95-53ad381614c3)
GPU 4: NVIDIA RTX A5000 (UUID: GPU-ba51fd6c-37bf-1b95-5f68-987c18a6292a)
GPU 5: NVIDIA RTX A5000 (UUID: GPU-c1224a2a-4a3b-ff16-0308-4f36205b9859)
GPU 6: NVIDIA RTX A5000 (UUID: GPU-8d20d6cd-abf5-2630-ab88-6bba438c55fe)
GPU 7: NVIDIA RTX A5000 (UUID: GPU-93170910-5d94-6da5-8a24-f561d7da1e2d)
</pre>

You can also use SBATCH to submit your job. Here are two examples on how to do that.

<pre>
$ sbatch --pty --gres=gpu:8 --account=gamma --partition=gamma --qos=huge-long --time=1-23:00:00 script.sh
</pre>

OR

<pre>
$ sbatch script.sh

// script.sh //

#!/bin/bash
#SBATCH --gres=gpu:8
#SBATCH --account=gamma
#SBATCH --partition=gamma
#SBATCH --qos=huge-long
#SBATCH --time=1-23:00:00

python your_file.py
</pre>

=Storage=
There are 3 types of user storage available to users in GAMMA:
* Home directories
* Project directories
* Scratch directories

There is also read-only storage available for Dataset directories.

GAMMA users can also request [[Nexus#Project_Allocations | Nexus project allocations]].

===Home Directories===
{{Nfshomes}}

===Project Directories===
You can request project based allocations for up to 8TB and up to 180 days with approval from a GAMMA faculty member.

To request an allocation, please [[HelpDesk | contact staff]] with the faculty member(s) that approved the project in the conversation. Please include the following details:
* Project Name (short)
* Description
* Size (1TB, 2TB, etc.)
* Length in days (30 days, 90 days, etc.)
* Other user(s) that need to access the allocation, if any

These allocations will be available from '''/fs/gamma-projects''' under a name that you provide when you request the allocation. Near the end of the allocation period, staff will contact you and ask if you would like to renew the allocation (requires re-approval from a GAMMA faculty member).
* If you are no longer in need of the storage allocation, you will need to relocate all desired data within two weeks of the end of the allocation period. Staff will then remove the allocation.
* If you do not respond to staff's request by the end of the allocation period, staff will make the allocation temporarily inaccessible.
** If you do respond asking for renewal but the original faculty approver does not respond within two weeks of the end of the allocation period, staff will also make the allocation temporarily inaccessible.
** If one month from the end of the allocation period is reached without both you and the faculty approver responding, staff will remove the allocation.

This data is backed up nightly.

===Scratch Directories===
Scratch data has no data protection, there are no snapshots and the data is not backed up.
There are two types of scratch directories:
* Network scratch directory
* Local scratch directories

====Network Scratch Directory====
You are allocated 100GB of scratch space via NFS from <code>/gammascratch/$username</code>. '''It is not backed up or protected in any way.'''

This directory is '''automounted''' so you may not see your directory if you run <code>ls /gammascratch</code> but it will be mounted when you <code>cd</code> into your /gammascratch directory.

You may request a permanent increase of up to 200GB total space without any faculty approval by [[HelpDesk | contacting staff]]. If you need space beyond 200GB, you will need faculty approval.

This file system is available on all submission, data management, and computational nodes within the cluster.

====Local Scratch Directories====
These file systems are not available over [[NFS]] and '''there are no backups or snapshots available''' for these file systems.

* Each computational node that you can schedule compute jobs on has one or more local scratch directories. These are always named <code>/scratch0</code>, <code>/scratch1</code>, etc. These directories are local to each node, ie. the <code>/scratch0</code> on two different nodes are completely separate.
** These directories are almost always more performant than any other storage available to the job. However, you must stage data to these directories within the confines of your jobs and stage the data out before the end of your jobs.
** These local scratch directories have a tmpwatch job which will '''delete unaccessed data after 90 days''', scheduled via maintenance jobs to run once a month during our monthly maintenance windows. Again, please make sure you secure any data you write to these directories at the end of your job.
* Gamma has invested in a 20TB NVMe scratch file system on <code>nexusgamma00.umiacs.umd.edu</code> that is available as <code>/scratch1</code>. To utilize this space, you will need to copy data from/to this over SSH from a compute node. To make this easier, you may want to setup [[SSH]] keys that will allow you to copy data without prompting for passwords.
** The <code>/scratch1</code> directory on <code>nexusgamma00.umiacs.umd.edu</code> doesn't have a tmpwatch. The files in this directory need to be manually removed once they are no longer needed.

===Datasets===
We have read-only dataset storage available at <code>/fs/gamma-datasets</code>. If there are datasets that you would like to see curated and available, please see [[Datasets | this page]].

The list of GAMMA datasets we currently host can be viewed [https://info.umiacs.umd.edu/datasets/list/?q=GAMMA here].

Nexus/Vulcan

2024-06-03T18:07:16Z

Ekr597:

The Vulcan standalone cluster's compute nodes have folded into [[Nexus]] as of the scheduled [[MonthlyMaintenanceWindow | maintenance window]] for August 2023 (Thursday 08/17/2023, 5-8pm).

The Nexus cluster already has a large pool of compute resources made possible through college-level funding for UMIACS and CSD faculty. Details on common nodes already in the cluster (Tron partition) can be found [[Nexus/Tron | here]].

Please [[HelpDesk | contact staff]] with any questions or concerns.

==Usage==
You can [[SSH]] to <code>nexusvulcan.umiacs.umd.edu</code> to log in to a submission host.

If you store something in a local directory (/tmp, /scratch0) on one of the two submission hosts, you will need to connect to that same submission host to access it later. The actual submission hosts are:
* <code>nexusvulcan00.umiacs.umd.edu</code>
* <code>nexusvulcan01.umiacs.umd.edu</code>

All partitions, QoSes, and account names from the standalone Vulcan cluster have been moved over to Nexus. However, please note that <code>vulcan-</code> is prepended to all of the values that were present in the standalone Vulcan cluster to distinguish them from existing values in Nexus. The lone exception is the base account that was named <code>vulcan</code> in the standalone cluster (it is also named just <code>vulcan</code> in Nexus).

Here are some before/after examples of job submission with various parameters:

{| class="wikitable"
! Standalone Vulcan cluster submission command
! Nexus cluster submission command
|-
|<code>srun --partition=dpart --qos=medium --account=abhinav --gres=gpu:gtx1080ti:2 --pty bash</code>
|<code>srun --partition=vulcan-dpart --qos=vulcan-medium --account=vulcan-abhinav --gres=gpu:gtx1080ti:2 --pty bash</code>
|-
|<code>srun --partition=cpu --qos=cpu --pty bash</code>
|<code>srun --partition=vulcan-cpu --qos=vulcan-cpu --account=vulcan --pty bash</code>
|-
|<code>srun --partition=scavenger --qos=scavenger --account=vulcan --gres=gpu:4 --pty bash</code>
|<code>srun --partition=vulcan-scavenger --qos=vulcan-scavenger --account=vulcan --gres=gpu:4 --pty bash</code>
|}

Vulcan users (exclusively) can schedule non-interruptible jobs on Vulcan nodes with any non-scavenger job parameters. Please note that the <code>vulcan-dpart</code> partition has a <code>GrpTRES</code> limit of 100% of the available cores/RAM on all vulcan## in aggregate nodes plus 50% of the available cores/RAM on legacy## nodes in aggregate, so your job may need to wait if all available cores/RAM (or GPUs) are in use. It also has a max submission limit of 500 jobs per user simultaneously so as to not overload the cluster. This is codified by the partition QoS named '''vulcan'''.

Please note that the Vulcan compute nodes are also in the institute-wide <code>scavenger</code> partition in Nexus. Vulcan users still have scavenging priority over these nodes via the <code>vulcan-scavenger</code> partition (i.e., all <code>vulcan-</code> partition jobs (other than <code>vulcan-scavenger</code>) can preempt both <code>vulcan-scavenger</code> and <code>scavenger</code> partition jobs, and <code>vulcan-scavenger</code> partition jobs can preempt <code>scavenger</code> partition jobs).

==Nodes==
There are currently 45 [[Nexus/Vulcan/GPUs | GPU nodes]] available running a mixture of NVIDIA RTX A6000, NVIDIA RTX A5000, NVIDIA RTX A4000, NVIDIA Quadro P6000, NVIDIA GeForce GTX 1080 Ti, NVIDIA GeForce RTX 2080 Ti, and NVIDIA Tesla P100 cards. There are also 4 CPU-only nodes available.

All nodes are scheduled with the [[SLURM]] resource manager.

==Partitions==
There are three partitions available to general Vulcan [[SLURM]] users. You must specify a partition when submitting your job.

* '''vulcan-dpart''' - This is the default partition. Job allocations are guaranteed. Only nodes with GPUs from architectures before NVIDIA's [https://www.nvidia.com/en-us/data-center/ampere-architecture/ Ampere architecture] are included in this partition.
* '''vulcan-scavenger''' - This is the alternate partition that allows jobs longer run times and more resources but is preemptable when jobs in other <code>vulcan-</code> partitions are ready to be scheduled.
* '''vulcan-cpu''' - This partition is for CPU focused jobs. Job allocations are guaranteed.

There are a few additional partitions available to subsets of Vulcan users based on specific requirements.

* '''vulcan-ampere''' - This partition contains nodes with GPUs from NVIDIA's [https://www.nvidia.com/en-us/data-center/ampere-architecture/ Ampere architecture]. Job allocations are guaranteed.
*: As of Thursday 02/29/2024 at 12pm, there is a 4 hour time limit on interactive jobs in this partition. If you need to run longer jobs, you will need to modify your workflow into a job that can be submitted as a batch script.
*: As of Thursday 03/21/2024 at 5pm, there is a limit of 4 CPUs and 48G memory maximum per GPU requested by a job. If you need to run jobs with more CPUs/memory, you will either need to request more GPUs in the job or use a different partition.

: Submission is restricted to the Slurm [[#Accounts | accounts]] of the faculty who invested in these nodes:
:* Abhinav Shrivastava (vulcan-abhinav)
:* Jia-Bin Huang (vulcan-jbhuang)
:* Christopher Metzler (vulcan-metzler)
:* Matthias Zwicker (vulcan-zwicker)

==Accounts==
Vulcan has a base SLURM account <code>vulcan</code> which has a modest number of guaranteed billing resources available to all cluster users at any given time. Other faculty that have invested in Vulcan compute infrastructure have an additional account provided to their sponsored accounts on the cluster.

If you do not specify an account when submitting your job, you will receive the '''vulcan''' account. If your faculty sponsor has their own account, it is recommended to use that account for job submission.

The current faculty accounts are:
* vulcan-abhinav
* vulcan-djacobs
* vulcan-jbhuang
* vulcan-lsd
* vulcan-metzler
* vulcan-rama
* vulcan-ramani
* vulcan-yaser
* vulcan-zwicker

<pre>
$ sacctmgr show account format=account%20,description%30,organization%10
Account Descr Org
-------------------- ------------------------------ ----------
... ... ...
vulcan vulcan vulcan
vulcan-abhinav vulcan - abhinav shrivastava vulcan
vulcan-djacobs vulcan - david jacobs vulcan
vulcan-jbhuang vulcan - jia-bin huang vulcan
vulcan-lsd vulcan - larry davis vulcan
vulcan-metzler vulcan - chris metzler vulcan
vulcan-rama vulcan - rama chellappa vulcan
vulcan-ramani vulcan - ramani duraiswami vulcan
vulcan-yaser vulcan - yaser yacoob vulcan
vulcan-zwicker vulcan - matthias zwicker vulcan
... ... ...
</pre>

Faculty can manage this list of users via our [https://intranet.umiacs.umd.edu/directory/secgroup/ Directory application] in the Security Groups section. The security group that controls access has the prefix <code>vulcan_</code> and then the faculty username. It will also list <code>slurm://nexusctl.umiacs.umd.edu</code> as the associated URI.

You can check your account associations by running the '''show_assoc''' command to see the accounts you are associated with. Please [[HelpDesk | contact staff]] and include your faculty member in the conversation if you do not see the appropriate association.

<pre>
$ show_assoc
User Account MaxJobs GrpTRES QOS
---------- ---------------- ------- ------------- --------------------------------------------------------------------------------
... ... ... ...
abhinav vulcan 48 vulcan-cpu,vulcan-default,vulcan-medium,vulcan-scavenger
abhinav vulcan-abhinav 48 vulcan-cpu,vulcan-default,vulcan-high,vulcan-medium,vulcan-scavenger
... ... ... ...
</pre>

You can also see the total number of Track-able Resources (TRES) allowed for each account by running the following command. Please make sure you give the appropriate account that you are looking for. As shown below, there is a concurrent limit of 64 total GPUs for all users not in a contributing faculty group.

<pre>
$ sacctmgr show assoc account=vulcan format=user,account,qos,grptres
User Account QOS GrpTRES
---------- ---------- -------------------- -------------
vulcan gres/gpu=64
... ...
</pre>

==QoS==
You need to decide the QOS to submit with which will set a certain number of restrictions to your job. If you do not specify a QoS when submitting your job using the <code>--qos</code> parameter, you will receive the '''vulcan-default''' QoS assuming you are using a Vulcan account.

The following <code>sacctmgr</code> command will list the current QOS. Either the <code>vulcan-default</code>, <code>vulcan-medium</code>, or <code>vulcan-high</code> QOS is required for the vulcan-dpart partition. Please note that only faculty accounts (see above) have access to the <code>vulcan-high</code> QoS.

The following example will show you the current limits that the QOS have. The output is truncated to show only relevant Vulcan QoS.
<pre>
$ show_qos
Name MaxWall MaxTRES MaxJobsPU MaxTRESPU
-------------------- ----------- ------------------------------ --------- ------------------------------
...
vulcan-cpu 2-00:00:00 cpu=1024,mem=4T 4
vulcan-default 7-00:00:00 cpu=4,gres/gpu=1,mem=32G 2
vulcan-exempt 7-00:00:00 cpu=32,gres/gpu=8,mem=256G 2
vulcan-high 1-12:00:00 cpu=16,gres/gpu=4,mem=128G 2
vulcan-janus 3-00:00:00 cpu=32,gres/gpu=10,mem=256G
vulcan-medium 3-00:00:00 cpu=8,gres/gpu=2,mem=64G 2
vulcan-sailon 3-00:00:00 cpu=32,gres/gpu=8,mem=256G gres/gpu=48
vulcan-scavenger 3-00:00:00 cpu=32,gres/gpu=8,mem=256G
...

$ show_partition_qos
Name MaxSubmitPU MaxTRESPU GrpTRES
-------------------- ----------- ------------------------------ --------------------
...
vulcan 500 cpu=1760,mem=15824G
vulcan-scavenger 500
...
</pre>

==Storage==
Vulcan has the following storage available. Please also review UMIACS [[LocalDataStorage | Local Data Storage]] policies including any volume that is labeled as scratch.

Vulcan users can also request [[Nexus#Project_Allocations | Nexus project allocations]].

===Home Directories===
{{Nfshomes}}

===Scratch Directories===
Scratch data has no data protection including no snapshots and the data is not backed up. There are two types of scratch directories in the Vulcan compute infrastructure:
* Network scratch directory
* Local scratch directories

====Network Scratch Directory====
You have 300GB of scratch storage available at <code>/vulcanscratch/<username></code>. '''It is not backed up or protected in any way.''' This directory is '''automounted''' so you will need to <code>cd</code> into the directory or request/specify a fully qualified file path to access this.

You may request a temporary increase of up to 500GB total space for a maximum of 120 days without any faculty approval by [[HelpDesk | contacting staff]]. Once the temporary increase period is over, you will be contacted and given a one-week window of opportunity to clean and secure your data before staff will forcibly remove data to get your space back under 300GB. If you need space beyond 500GB or for longer than 120 days, you will need faculty approval and/or a project directory.

This file system is available on all submission, data management, and computational nodes within the cluster.

====Local Scratch Directories====
Each computational node that you can schedule compute jobs on has one or more local scratch directories. These are always named <code>/scratch0</code>, <code>/scratch1</code>, etc. These are almost always more performant than any other storage available to the job. However, you must stage their data within the confine of their job and stage the data out before the end of their job.

These local scratch directories have a tmpwatch job which will '''delete unaccessed data after 90 days''', scheduled via maintenance jobs to run once a month at 1am. Different nodes will run the maintenance jobs on different days of the month to ensure the cluster is still highly available at all times. Please make sure you secure any data you write to these directories at the end of your job.

===Datasets===
We have read-only dataset storage available at <code>/fs/vulcan-datasets</code>. If there are datasets that you would like to see curated and available, please see [[Datasets | this page]].

The list of Vulcan datasets we currently host can be viewed [https://info.umiacs.umd.edu/datasets/list/?q=Vulcan here].

===Project Storage===
Users within the Vulcan compute infrastructure can request project based allocations for up to 10TB for up to 180 days by [[HelpDesk | contacting staff]] with approval from the Vulcan faculty manager (Dr. Shrivastava). These allocations will be available from <code>/fs/vulcan-projects</code> and <code>/fs/cfar-projects</code> under a name that you provide when you request the allocation. Near the end of the allocation period, staff will contact you and ask if you would like to renew the allocation for up to another 180 days (requires re-approval from Dr. Shrivastava).
* If you are no longer in need of the storage allocation, you will need to relocate all desired data within two weeks of the end of the allocation period. Staff will then remove the allocation.
* If you do not respond to staff's request by the end of the allocation period, staff will make the allocation temporarily inaccessible.
** If you do respond asking for renewal but the original faculty approver does not respond within two weeks of the end of the allocation period, staff will also make the allocation temporarily inaccessible.
** If one month from the end of the allocation period is reached without both you and the faculty approver responding, staff will remove the allocation.

Project storage is fully protected. It has [[Snapshots | snapshots]] enabled and is [[NightlyBackups | backed up nightly]].

===Object Storage===
All Vulcan users can request project allocations in the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store]. Please [[HelpDesk | contact staff]] with a short project name and the amount of storage you will need to get started.

To access this storage, you'll need to use a [[S3Clients | S3 client]] or our [[UMobj]] command line utilities.

An example on how to use the umobj command line utilities can be found [[UMobj/Example | here]]. A full set of documentation for the utilities can be found on the [https://gitlab.umiacs.umd.edu/staff/umobj/blob/master/README.md#umobj umobj Gitlab page].

==Migration==
===Home Directories===
The [[Nexus]] uses [[NFShomes]] home directories - if your UMIACS account was created before February 22nd, 2023, you were using <code>/cfarhomes/<username></code> as your home directory on the standalone Vulcan cluster. While <code>/cfarhomes</code> is available on Nexus, your shell initialization scripts from it will not automatically load. Please copy over anything you need to your <code>/nfshomes/<username></code> directory at your earliest convenience, as <code>/cfarhomes</code> will be retired in a two phase process:
* Fri 11/17/2023, 5pm: cfarhomes directories are made read-only
* Thu 12/21/2023, 5-8pm ([[MonthlyMaintenanceWindow |monthly maintenance window]]): cfarhomes directories are taken offline

Nexus/CML

2024-02-07T18:27:08Z

Ekr597: Changed the starting /cmlscratch to 200GB

The [[CML]] standalone cluster's compute nodes have folded into [[Nexus]] as of the scheduled [[MonthlyMaintenanceWindow | maintenance window]] for August 2023 (Thursday 08/17/2023, 5-8pm).

The Nexus cluster already has a large pool of compute resources made possible through college-level funding for UMIACS and CSD faculty. Details on common nodes already in the cluster (Tron partition) can be found [[Nexus/Tron | here]].

Please [[HelpDesk | contact staff]] with any questions or concerns.

==Usage==
The Nexus cluster submission nodes that are allocated to CML are <code>nexuscml00.umiacs.umd.edu</code> and <code>nexuscml01.umiacs.umd.edu</code>.

All partitions, QoSes, and account names from the standalone CML cluster have been moved over to Nexus. However, please note that <code>cml-</code> is prepended to all of the values that were present in the standalone CML cluster to distinguish them from existing values in Nexus. The lone exception is the base account that was named <code>cml</code> in the standalone cluster (it is also named just <code>cml</code> in Nexus).

Here are some before/after examples of job submission with various parameters:

{| class="wikitable"
! Standalone CML cluster submission command
! Nexus cluster submission command
|-
|<code>srun --partition=dpart --qos=medium --account=tomg --gres=gpu:rtx2080ti:2 --pty bash</code>
|<code>srun --partition=cml-dpart --qos=cml-medium --account=cml-tomg --gres=gpu:rtx2080ti:2 --pty bash</code>
|-
|<code>srun --partition=cpu --qos=cpu --pty bash</code>
|<code>srun --partition=cml-cpu --qos=cml-cpu --account=cml --pty bash</code>
|-
|<code>srun --partition=scavenger --qos=scavenger --account=scavenger --gres=gpu:4 --pty bash</code>
|<code>srun --partition=cml-scavenger --qos=cml-scavenger --account=cml-scavenger --gres=gpu:4 --pty bash</code>
|}

CML users (exclusively) can schedule non-interruptible jobs on CML nodes with any non-scavenger job parameters. Please note that the <code>cml-dpart</code> partition has a <code>GrpTRES</code> limit of 100% of the available cores/RAM on all cml## nodes in aggregate plus 50% of the available cores/RAM on legacy## nodes in aggregate, so your job may need to wait if all available cores/RAM (or GPUs) are in use. It also has a max submission limit of 500 jobs per user simultaneously so as to not overload the cluster. This is codified by the partition QoS named '''cml'''.

Please note that the CML compute nodes are also in the institute-wide <code>scavenger</code> partition in Nexus. CML users still have scavenging priority over these nodes via the <code>cml-scavenger</code> partition (i.e., all <code>cml-</code> partition jobs (other than <code>cml-scavenger</code>) can preempt both <code>cml-scavenger</code> and <code>scavenger</code> partition jobs, and <code>cml-scavenger</code> partition jobs can preempt <code>scavenger</code> partition jobs).

==Partitions==
There are three partitions available to general CML [[SLURM]] users. You must specify a partition when submitting your job.

* '''cml-dpart''' - This is the default partition. Job allocations are guaranteed.
* '''cml-scavenger''' - This is the alternate partition that allows jobs longer run times and more resources but is preemptable when jobs in other <code>cml-</code> partitions are ready to be scheduled.
* '''cml-cpu''' - This partition is for CPU focused jobs. Job allocations are guaranteed.

There is one additional partition available solely to Dr. Furong Huang's sponsored accounts.

* '''cml-furongh''' - This partition is for exclusive priority access to Dr. Huang's purchased A6000 node. Job allocations are guaranteed.

==Accounts==
The Center has a base SLURM account <code>cml</code> which has a modest number of guaranteed billing resources available to all cluster users at any given time. Other faculty that have invested in the cluster have an additional account provided to their sponsored accounts on the cluster, which provides a number of guaranteed billing resources corresponding to the amount that they invested.

If you do not specify an account when submitting your job, you will receive the '''cml''' account. If your faculty sponsor has their own account, it is recommended to use that account for job submission.

The current faculty accounts are:
* cml-abhinav
* cml-cameron
* cml-furongh
* cml-hajiagha
* cml-john
* cml-ramani
* cml-sfeizi
* cml-tokekar
* cml-tomg
* cml-zhou

<pre>
$ sacctmgr show account format=account%20,description%30,organization%10
Account Descr Org
-------------------- ------------------------------ ----------
... ... ...
cml cml cml
cml-abhinav cml - abhinav shrivastava cml
cml-cameron cml - maria cameron cml
cml-furongh cml - furong huang cml
cml-hajiagha cml - mohammad hajiaghayi cml
cml-john cml - john dickerson cml
cml-ramani cml - ramani duraiswami cml
cml-scavenger cml - scavenger cml
cml-sfeizi cml - soheil feizi cml
cml-tokekar cml - pratap tokekar cml
cml-tomg cml - tom goldstein cml
cml-zhou cml - tianyi zhou cml
... ... ...
</pre>

Faculty can manage this list of users via our [https://intranet.umiacs.umd.edu/directory/secgroup/ Directory application] in the Security Groups section. The security group that controls access has the prefix <code>cml_</code> and then the faculty username. It will also list <code>slurm://nexusctl.umiacs.umd.edu</code> as the associated URI.

You can check your account associations by running the '''show_assoc''' command to see the accounts you are associated with. Please [[HelpDesk | contact staff]] and include your faculty member in the conversation if you do not see the appropriate association.

<pre>
$ show_assoc
User Account MaxJobs GrpTRES QOS
---------- ---------------- ------- ------------- --------------------------------------------------
... ... ...
tomg cml cml-cpu,cml-default,cml-medium
tomg cml-scavenger cml-scavenger
tomg cml-tomg cml-default,cml-high,cml-medium
... ... ...
</pre>

You can also see the total number of Track-able Resources (TRES) allowed for each account by running the following command. Please make sure you give the appropriate account that you are looking for. The billing number displayed here is the sum of [[SLURM/Priority#Modern | resource weightings]] for all nodes appropriated to that account.

<pre>
$ sacctmgr show assoc account=cml format=user,account,qos,grptres
User Account QOS GrpTRES
---------- ---------- -------------------- -------------
cml billing=7732
... ...
</pre>

==QoS==
CML currently has 5 QoS for the '''cml-dpart''' partition (though <code>high_long</code> and <code>very_high</code> may not be available to all faculty accounts), 1 QoS for the '''cml-scavenger''' partition, and 1 QoS for the '''cml-cpu''' partition. If you do not specify a QoS when submitting your job using the <code>--qos</code> parameter, you will receive the '''cml-default''' QoS assuming you are using a CML account.

The important part here is that in different QoS you can have a shorter/longer maximum wall time, a different total number of jobs running at once, and a different maximum number of track-able resources (TRES) for the job. In the cml-scavenger QoS, one more constraint that you are restricted by is the total number of TRES per user (over multiple jobs).

<pre>
$ show_qos
Name MaxWall MaxTRES MaxJobsPU MaxTRESPU
-------------------- ----------- ------------------------------ --------- ------------------------------
...
cml-cpu 7-00:00:00 8
cml-default 7-00:00:00 cpu=4,gres/gpu=1,mem=32G 2
cml-high 1-12:00:00 cpu=16,gres/gpu=4,mem=128G 2
cml-high_long 14-00:00:00 cpu=32,gres/gpu=8 8 gres/gpu=8
cml-medium 3-00:00:00 cpu=8,gres/gpu=2,mem=64G 2
cml-scavenger 3-00:00:00 gres/gpu=24
cml-very_high 1-12:00:00 cpu=32,gres/gpu=8,mem=256G 8 gres/gpu=12
...

$ show_partition_qos
Name MaxSubmitPU MaxTRESPU GrpTRES
-------------------- ----------- ------------------------------ --------------------
...
cml 500 cpu=1128,mem=11381G
cml-scavenger 500 gres/gpu=24
...
</pre>

==Storage==
There are 3 types of user storage available to users in the CML:
* Home directories
* Project directories
* Scratch directories

There are also 2 types of read-only storage available for common use among users in the CML:
* Dataset directories
* Model directories

CML users can also request [[Nexus#Project_Allocations | Nexus project allocations]].

===Home Directories===
Home directories in the CML computational infrastructure are available from the Institute's [[NFShomes]] as <code>/nfshomes/USERNAME</code> where USERNAME is your username. These home directories have very limited storage (30GB, cannot be increased) and are intended for your personal files, configuration and source code. Your home directory is '''not''' intended for data sets or other large scale data holdings. Users are encouraged to utilize our [[GitLab]] infrastructure to host your code repositories.

'''NOTE''': To check your quota on this directory you will need to use the <code>quota -s</code> command.

Your home directory data is fully protected and has both [[Snapshots | snapshots]] and is [[NightlyBackups | backed up nightly]].

===Project Directories===
You can request project based allocations for up to 6TB for up to 120 days with approval from a CML faculty member and the director of CML.

To request an allocation, please [[HelpDesk | contact staff]] with the faculty member(s) that the project is under involved in the conversation. Please include the following details:
* Project Name (short)
* Description
* Size (1TB, 2TB, etc.)
* Length in days (30 days, 90 days, etc.)
* Other user(s) that need to access the allocation, if any

These allocations will be available from '''/fs/cml-projects''' under a name that you provide when you request the allocation. Near the end of the allocation period, staff will contact you and ask if you would like to renew the allocation for up to another 120 days (requires re-approval from a CML faculty member and the director of CML). If you are no longer in need of the storage allocation, you will need to relocate all desired data within two weeks of the end of the allocation period. Staff will then remove the allocation. If you do not respond to staff's request by the end of the allocation period, staff will make the allocation temporarily inaccessible. If you do respond asking for renewal but the original faculty approver does not respond within two weeks of the end of the allocation period, staff will also make the allocation temporarily inaccessible. If one month from the end of the allocation period is reached without both you and the faculty approver responding, staff will remove the allocation.

This data is backed up nightly.

===Scratch Directories===
Scratch data has no data protection including no snapshots and the data is not backed up. There are two types of scratch directories in the CML compute infrastructure:
* Network scratch directory
* Local scratch directories

====Network Scratch Directory====
You are allocated 200GB of scratch space via NFS from <code>/cmlscratch/$username</code>. '''It is not backed up or protected in any way.''' This directory is '''automounted''' so you will need to <code>cd</code> into the directory or request/specify a fully qualified file path to access this.

You may request a permanent increase of up to 800GB total space without any faculty approval by [[HelpDesk | contacting staff]]. If you need space beyond 800GB, you will need faculty approval and/or a project directory. Space increases beyond 800GB also have a maximum request period of 120 days (as with project directories), after which they will need to be renewed with re-approval from a CML faculty member and the director of CML.

This file system is available on all submission, data management, and computational nodes within the cluster.

====Local Scratch Directories====
Each computational node that you can schedule compute jobs on has one or more local scratch directories. These are always named <code>/scratch0</code>, <code>/scratch1</code>, etc. These are almost always more performant than any other storage available to the job. However, you must stage data to these directories within the confines of your jobs and stage the data out before the end of your jobs.

These local scratch directories have a tmpwatch job which will '''delete unaccessed data after 90 days''', scheduled via maintenance jobs to run once a month during our monthly maintenance windows. Again, please make sure you secure any data you write to these directories at the end of your job.

===Datasets===
We have read-only dataset storage available at <code>/fs/cml-datasets</code>. If there are datasets that you would like to see curated and available, please see [[Datasets | this page]].

The list of CML datasets we currently host can be viewed [https://info.umiacs.umd.edu/datasets/list/?q=CML here].

===Models===
We have read-only model storage available at <code>/fs/cml-models</code>. If there are models that you would like to see downloaded and made available, please see [[Datasets | this page]].

Nexus/GAMMA

2024-01-24T18:42:59Z

Ekr597:

The [https://gamma.umd.edu/ GAMMA] lab has a partition of GPU nodes available in the [[Nexus]]. Only GAMMA lab members are able to run non-interruptible jobs on these nodes.

=Access=
You can always find out what hosts you have access to submit via the [[Nexus#Access]] page. The GAMMA lab in particular has a special submission host that has additional local storage available.
* <code>nexusgamma00.umiacs.umd.edu</code>

Please do not run anything on the login node. Always allocate yourself machines on the compute nodes (see instructions below) to run any job.

=Quality of Service=
GAMMA users have access to all of the [[Nexus#Quality_of_Service_.28QoS.29 | standard job QoSes]] in the <code>gamma</code> partition using the <code>gamma</code> account.

The additional job QoSes for the GAMMA partition specifically are:
* <code>huge-long</code>: Allows for longer jobs using higher overall resources.

Please note that the partition has a <code>GrpTRES</code> limit of 100% of the available cores/RAM on the partition-specific nodes in aggregate plus 50% of the available cores/RAM on legacy## nodes in aggregate, so your job may need to wait if all available cores/RAM (or GPUs) are in use.

=Hardware=
{| class="wikitable sortable"
! Nodenames
! Type
! Quantity
! CPUs
! Memory
! GPUs
|-
|gammagpu[00-04,06-09]
|A5000 GPU Node
|9
|32
|256GB
|8
|-
|gammagpu05
|A4000 GPU Node
|1
|32
|256GB
|8
|- class="sortbottom"
|
!Total
|10
|320
|2560GB
|80
|}

=Example=
From <code>nexusgamma00.umiacs.umd.edu</code> you can run the following example to submit an interactive job. Please note that you need to specify the <code>--account</code>, <code>--partition</code> and <code>--qos</code>. Please refer to our [[SLURM]] documentation about about how to further customize your submissions including making a batch submission. The following command will allocate 8 GPUs for 2 days in an interactive session. Change parameters accordingly to your needs. We discourage use of srun and promote use of sbatch for fair use of GPUs.

<pre>
$ srun --pty --gres=gpu:8 --account=gamma --partition=gamma --qos=huge-long bash
$ hostname
gammagpu01.umiacs.umd.edu
$ nvidia-smi -L
GPU 0: NVIDIA RTX A5000 (UUID: GPU-cdfb2e0c-d69f-354b-02f4-15161dc7fa66)
GPU 1: NVIDIA RTX A5000 (UUID: GPU-be53e7a1-b8fd-7089-3cac-7a2fbf4ec7dd)
GPU 2: NVIDIA RTX A5000 (UUID: GPU-774efbb1-d7ec-a0bb-e992-da9d1fa6b193)
GPU 3: NVIDIA RTX A5000 (UUID: GPU-d1692181-c7de-e273-5f95-53ad381614c3)
GPU 4: NVIDIA RTX A5000 (UUID: GPU-ba51fd6c-37bf-1b95-5f68-987c18a6292a)
GPU 5: NVIDIA RTX A5000 (UUID: GPU-c1224a2a-4a3b-ff16-0308-4f36205b9859)
GPU 6: NVIDIA RTX A5000 (UUID: GPU-8d20d6cd-abf5-2630-ab88-6bba438c55fe)
GPU 7: NVIDIA RTX A5000 (UUID: GPU-93170910-5d94-6da5-8a24-f561d7da1e2d)
</pre>

You can also use SBATCH to submit your job. Here are two examples on how to do that.

<pre>
$ sbatch --pty --gres=gpu:8 --account=gamma --partition=gamma --qos=huge-long --time=1-23:00:00 script.sh
</pre>

OR

<pre>
$ sbatch script.sh

// script.sh //

#!/bin/bash
#SBATCH --gres=gpu:8
#SBATCH --account=gamma
#SBATCH --partition=gamma
#SBATCH --qos=huge-long
#SBATCH --time=1-23:00:00

python your_file.py
</pre>

=Storage=
There are 3 types of user storage available to users in GAMMA:
* Home directories
* Project directories
* Scratch directories

There is also read-only storage available for Dataset directories.

GAMMA users can also request [[Nexus#Project_Allocations | Nexus project allocations]].

===Home Directories===
Home directories are available from the Institute's [[NFShomes]] as <code>/nfshomes/USERNAME</code> where USERNAME is your UMIACS username. These home directories have very limited storage (30GB, cannot be increased) and are intended for your personal files, configuration and source code. Your home directory is '''not''' intended for data sets or other large scale data holdings. Users are encouraged to utilize our [[GitLab]] infrastructure to host your code repositories.

Your home directory data is fully protected and has both [[Snapshots | snapshots]] and is [[NightlyBackups | backed up nightly]].

===Project Directories===
You can request project based allocations with approval from a GAMMA faculty member.

To request an allocation, please [[HelpDesk | contact staff]] with the faculty member(s) that approved the project in the conversation. Please include the following details:
* Project Name (short)
* Description
* Size (1TB, 2TB, etc.)
* Length in days (30 days, 90 days, etc.)
* Other user(s) that need to access the allocation, if any

These allocations will be available from '''/fs/gamma-projects''' under a name that you provide when you request the allocation. Near the end of the allocation period, staff will contact you and ask if you would like to renew the allocation (requires re-approval from a GAMMA faculty member). If you are no longer in need of the storage allocation, you will need to relocate all desired data within two weeks of the end of the allocation period. Staff will then remove the allocation. If you do not respond to staff's request by the end of the allocation period, staff will make the allocation temporarily inaccessible. If you do respond asking for renewal but the original faculty approver does not respond within two weeks of the end of the allocation period, staff will also make the allocation temporarily inaccessible. If one month from the end of the allocation period is reached without both you and the faculty approver responding, staff will remove the allocation.

This data is backed up nightly.

===Scratch Directories===
Scratch data has no data protection, there are no snapshots and the data is not backed up.
There are two types of scratch directories:
* Network scratch directory
* Local scratch directories

====Network Scratch Directory====
You are allocated 100GB of scratch space via NFS from <code>/gammascratch/$username</code>. '''It is not backed up or protected in any way.'''

This directory is '''automounted''' so you may not see your directory if you run <code>ls /gammascratch</code> but it will be mounted when you <code>cd</code> into your /gammascratch directory.

You may request a permanent increase of up to 200GB total space without any faculty approval by [[HelpDesk | contacting staff]]. If you need space beyond 200GB, you will need faculty approval.

This file system is available on all submission, data management, and computational nodes within the cluster.

====Local Scratch Directories====
These file systems are not available over [[NFS]] and '''there are no backups or snapshots available''' for these file systems.

* Each computational node that you can schedule compute jobs on has one or more local scratch directories. These are always named <code>/scratch0</code>, <code>/scratch1</code>, etc. These directories are local to each node, ie. the <code>/scratch0</code> on two different nodes are completely separate.
** These directories are almost always more performant than any other storage available to the job. However, you must stage data to these directories within the confines of your jobs and stage the data out before the end of your jobs.
** These local scratch directories have a tmpwatch job which will '''delete unaccessed data after 90 days''', scheduled via maintenance jobs to run once a month during our monthly maintenance windows. Again, please make sure you secure any data you write to these directories at the end of your job.
* Gamma has invested in a 20TB NVMe scratch file system on <code>nexusgamma00.umiacs.umd.edu</code> that is available as <code>/scratch1</code>. To utilize this space, you will need to copy data from/to this over SSH from a compute node. To make this easier, you may want to setup [[SSH]] keys that will allow you to copy data without prompting for passwords.
** The <code>/scratch1</code> directory on <code>nexusgamma00.umiacs.umd.edu</code> doesn't have a tmpwatch. The files in this directory need to be manually removed once they are no longer needed.

===Datasets===
We have read-only dataset storage available at <code>/fs/gamma-datasets</code>. If there are datasets that you would like to see curated and available, please see [[Datasets | this page]].

The list of GAMMA datasets we currently host can be viewed [https://info.umiacs.umd.edu/datasets/list/?q=GAMMA here].

Nexus/GAMMA

2024-01-05T21:01:19Z

Ekr597:

The [https://gamma.umd.edu/ GAMMA] lab has a partition of GPU nodes available in the [[Nexus]]. Only GAMMA lab members are able to run non-interruptible jobs on these nodes.

=Access=
You can always find out what hosts you have access to submit via the [[Nexus#Access]] page. The GAMMA lab in particular has a special submission host that has additional local storage available.
* <code>nexusgamma00.umiacs.umd.edu</code>

Please do not run anything on the login node. Always allocate yourself machines on the compute nodes (see instructions below) to run any job.

=Quality of Service=
GAMMA users have access to all of the [[Nexus#Quality_of_Service_.28QoS.29 | standard job QoSes]] in the <code>gamma</code> partition using the <code>gamma</code> account.

The additional job QoSes for the GAMMA partition specifically are:
* <code>huge-long</code>: Allows for longer jobs using higher overall resources.

Please note that the partition has a <code>GrpTRES</code> limit of 100% of the available cores/RAM on the partition-specific nodes in aggregate plus 50% of the available cores/RAM on legacy## nodes in aggregate, so your job may need to wait if all available cores/RAM (or GPUs) are in use.

=Hardware=
{| class="wikitable sortable"
! Nodenames
! Type
! Quantity
! CPUs
! Memory
! GPUs
|-
|gammagpu[00-04,06-09]
|A5000 GPU Node
|9
|32
|256GB
|8
|-
|gammagpu05
|A4000 GPU Node
|1
|32
|256GB
|8
|- class="sortbottom"
|
!Total
|10
|320
|2560GB
|80
|}

=Example=
From <code>nexusgamma00.umiacs.umd.edu</code> you can run the following example to submit an interactive job. Please note that you need to specify the <code>--account</code>, <code>--partition</code> and <code>--qos</code>. Please refer to our [[SLURM]] documentation about about how to further customize your submissions including making a batch submission. The following command will allocate 8 GPUs for 2 days in an interactive session. Change parameters accordingly to your needs. We discourage use of srun and promote use of sbatch for fair use of GPUs.

<pre>
$ srun --pty --gres=gpu:8 --account=gamma --partition=gamma --qos=huge-long bash
$ hostname
gammagpu01.umiacs.umd.edu
$ nvidia-smi -L
GPU 0: NVIDIA RTX A5000 (UUID: GPU-cdfb2e0c-d69f-354b-02f4-15161dc7fa66)
GPU 1: NVIDIA RTX A5000 (UUID: GPU-be53e7a1-b8fd-7089-3cac-7a2fbf4ec7dd)
GPU 2: NVIDIA RTX A5000 (UUID: GPU-774efbb1-d7ec-a0bb-e992-da9d1fa6b193)
GPU 3: NVIDIA RTX A5000 (UUID: GPU-d1692181-c7de-e273-5f95-53ad381614c3)
GPU 4: NVIDIA RTX A5000 (UUID: GPU-ba51fd6c-37bf-1b95-5f68-987c18a6292a)
GPU 5: NVIDIA RTX A5000 (UUID: GPU-c1224a2a-4a3b-ff16-0308-4f36205b9859)
GPU 6: NVIDIA RTX A5000 (UUID: GPU-8d20d6cd-abf5-2630-ab88-6bba438c55fe)
GPU 7: NVIDIA RTX A5000 (UUID: GPU-93170910-5d94-6da5-8a24-f561d7da1e2d)
</pre>

You can also use SBATCH to submit your job. Here are two examples on how to do that.

<pre>
$ sbatch --pty --gres=gpu:8 --account=gamma --partition=gamma --qos=huge-long --time=1-23:00:00 script.sh
</pre>

OR

<pre>
$ sbatch script.sh

// script.sh //

#!/bin/bash
#SBATCH --gres=gpu:8
#SBATCH --account=gamma
#SBATCH --partition=gamma
#SBATCH --qos=huge-long
#SBATCH --time=1-23:00:00

python your_file.py
</pre>

=Storage=
There are 3 types of user storage available to users in GAMMA:
* Home directories
* Project directories
* Scratch directories

There is also read-only storage available for Dataset directories.

GAMMA users can also request [[Nexus#Project_Allocations | Nexus project allocations]].

===Home Directories===
Home directories are available from the Institute's [[NFShomes]] as <code>/nfshomes/USERNAME</code> where USERNAME is your UMIACS username. These home directories have very limited storage (30GB, cannot be increased) and are intended for your personal files, configuration and source code. Your home directory is '''not''' intended for data sets or other large scale data holdings. Users are encouraged to utilize our [[GitLab]] infrastructure to host your code repositories.

Your home directory data is fully protected and has both [[Snapshots | snapshots]] and is [[NightlyBackups | backed up nightly]].

===Project Directories===
You can request project based allocations with approval from a GAMMA faculty member.

To request an allocation, please [[HelpDesk | contact staff]] with the faculty member(s) that approved the project in the conversation. Please include the following details:
* Project Name (short)
* Description
* Size (1TB, 2TB, etc.)
* Length in days (30 days, 90 days, etc.)
* Other user(s) that need to access the allocation, if any

These allocations will be available from '''/fs/gamma-projects''' under a name that you provide when you request the allocation. Near the end of the allocation period, staff will contact you and ask if you would like to renew the allocation (requires re-approval from a GAMMA faculty member). If you are no longer in need of the storage allocation, you will need to relocate all desired data within two weeks of the end of the allocation period. Staff will then remove the allocation. If you do not respond to staff's request by the end of the allocation period, staff will make the allocation temporarily inaccessible. If you do respond asking for renewal but the original faculty approver does not respond within two weeks of the end of the allocation period, staff will also make the allocation temporarily inaccessible. If one month from the end of the allocation period is reached without both you and the faculty approver responding, staff will remove the allocation.

This data is backed up nightly.

===Scratch Directories===
Scratch data has no data protection, there are no snapshots and the data is not backed up.
There are two types of scratch directories:
* Network scratch directory
* Local scratch directories

====Network Scratch Directory====
You are allocated 50GB of scratch space via NFS from <code>/gammascratch/$username</code>. '''It is not backed up or protected in any way.'''

This directory is '''automounted''' so you may not see your directory if you run <code>ls /gammascratch</code> but it will be mounted when you <code>cd</code> into your /gammascratch directory.

You may request a permanent increase of up to 200GB total space without any faculty approval by [[HelpDesk | contacting staff]]. If you need space beyond 200GB, you will need faculty approval.

This file system is available on all submission, data management, and computational nodes within the cluster.

====Local Scratch Directories====
These file systems are not available over [[NFS]] and '''there are no backups or snapshots available''' for these file systems.

* Each computational node that you can schedule compute jobs on has one or more local scratch directories. These are always named <code>/scratch0</code>, <code>/scratch1</code>, etc. These directories are local to each node, ie. the <code>/scratch0</code> on two different nodes are completely separate.

** These directories are almost always more performant than any other storage available to the job. However, you must stage data to these directories within the confines of your jobs and stage the data out before the end of your jobs.

** These local scratch directories have a tmpwatch job which will '''delete unaccessed data after 90 days''', scheduled via maintenance jobs to run once a month during our monthly maintenance windows. Again, please make sure you secure any data you write to these directories at the end of your job.

* Gamma has invested in a 20TB NVMe scratch file system on <code>nexusgamma00.umiacs.umd.edu</code> that is available as <code>/scratch1</code>. To utilize this space, you will need to copy data from/to this over SSH from a compute node. To make this easier, you may want to setup [[SSH]] keys that will allow you to copy data without prompting for passwords.

** The <code>/scratch1</code> directory on <code>nexusgamma00.umiacs.umd.edu</code> doesn't have a tmpwatch. The files in this directory need to be manually removed once they are no longer needed.

===Datasets===
We have read-only dataset storage available at <code>/fs/gamma-datasets</code>. If there are datasets that you would like to see curated and available, please see [[Datasets | this page]].

The list of GAMMA datasets we currently host can be viewed [https://info.umiacs.umd.edu/datasets/list/?q=GAMMA here].

Datasets

2024-01-02T22:03:30Z

Ekr597: Add GAMMA

UMIACS hosts a number of datasets in read-only mode on some shared filesystems used by our [[SLURM]] computing clusters. The motivation behind this is to provide publicly accessible datasets in a well-defined location in order to de-duplicate their use elsewhere, thus reducing overall storage usage.

==Dataset Directories==
* [[Nexus/CML | CML]] (<code>/fs/cml-datasets</code>)
** [https://info.umiacs.umd.edu/datasets/list/?q=CML List of datasets] -- faculty approver Tom Goldstein
* [[Nexus/GAMMA | GAMMA]] (<code>/fs/gamma-datasets</code>)
** [https://info.umiacs.umd.edu/datasets/list/?q=GAMMA List of datasets] -- faculty approver Dinesh Manocha
* [[Nexus]] (<code>/fs/nexus-datasets</code>)
** [https://info.umiacs.umd.edu/datasets/list/?q=Nexus List of datasets]
* [[Nexus/Vulcan | Vulcan]] (<code>/fs/vulcan-datasets</code>)
** [https://info.umiacs.umd.edu/datasets/list/?q=Vulcan List of datasets] -- faculty approver Abhinav Shrivastava

==Requesting a new dataset==
You can request a new dataset by [[HelpDesk | contacting staff]] with a link to the dataset's official download location. Torrents or other peer-to-peer re-hosting are not allowed unless sanctioned by the dataset owners.
* '''CML/GAMMA/Vulcan''': If the uncompressed/final dataset size is over 100GB, staff will first contact the faculty approver for the cluster to ensure they approve of using the storage space. If the size is under 100GB, no faculty approval is required. Staff will then inspect the dataset and see if there are any terms and conditions that must be agreed to before downloading.
* '''Nexus''': Please let staff know which faculty member's research you are working on that requires use of the dataset you are requesting. Then, if the uncompressed/final dataset size is over 50GB, the [https://www.umiacs.umd.edu/people/computing-staff Director of Computing Facilities] must first approve of using the storage space. If the size is under 50GB, no approval is required. Staff will then inspect the dataset and see if there are any terms and conditions that must be agreed to before downloading.

If there are no terms and conditions, staff will download/extract the dataset, copy it to the appropriate location depending on what cluster you are requesting it for, and let you know when it is available for use on that cluster.

If there are terms and conditions in excess of well-defined [https://creativecommons.org/about/cclicenses/ Creative Commons licenses], staff will first need to have the dataset's terms and conditions approved by [https://ora.umd.edu/ UMD's Office of Research Administration (ORA)]. Since the dataset will be hosted by UMIACS, as an institution, in a location accessible by all users of a cluster, not all of whom will have individually agreed to the terms and conditions, having a single person, staff member or not, agree to a set of terms and conditions is not sufficient to host it. After approval by ORA, staff will perform the same steps as mentioned above (download/extract, copy to appropriate location, and let you know when ready).

==Dataset use==
All datasets are read-only to users. Any intermediate data generated from a dataset will need to be stored in a location other than the shared filesystem hosting the dataset and other datasets.

Exceptions may be granted if there is a set of intermediate data generated from a dataset that you believe will be useful to a subset of a cluster's users. If you suspect this is the case for some of your generated data, please [[HelpDesk | contact staff]]. We will follow the above procedure and upon whatever approvals may be necessary, copy the intermediate data into the shared filesystem.

Nexus/GAMMA

2024-01-02T21:56:15Z

Ekr597: Added info about GAMMA's network storage

The [https://gamma.umd.edu/ GAMMA] lab has a partition of GPU nodes available in the [[Nexus]]. Only GAMMA lab members are able to run non-interruptible jobs on these nodes.

=Access=
You can always find out what hosts you have access to submit via the [[Nexus#Access]] page. The GAMMA lab in particular has a special submission host that has additional local storage available.
* <code>nexusgamma00.umiacs.umd.edu</code>

Please do not run anything on the login node. Always allocate yourself machines on the compute nodes (see instructions below) to run any job.

=Quality of Service=
GAMMA users have access to all of the [[Nexus#Quality_of_Service_.28QoS.29 | standard job QoSes]] in the <code>gamma</code> partition using the <code>gamma</code> account.

The additional job QoSes for the GAMMA partition specifically are:
* <code>huge-long</code>: Allows for longer jobs using higher overall resources.

Please note that the partition has a <code>GrpTRES</code> limit of 100% of the available cores/RAM on the partition-specific nodes in aggregate plus 50% of the available cores/RAM on legacy## nodes in aggregate, so your job may need to wait if all available cores/RAM (or GPUs) are in use.

=Hardware=
{| class="wikitable sortable"
! Nodenames
! Type
! Quantity
! CPUs
! Memory
! GPUs
|-
|gammagpu[00-04,06-09]
|A5000 GPU Node
|9
|32
|256GB
|8
|-
|gammagpu05
|A4000 GPU Node
|1
|32
|256GB
|8
|- class="sortbottom"
|
!Total
|10
|320
|2560GB
|80
|}

=Example=
From <code>nexusgamma00.umiacs.umd.edu</code> you can run the following example to submit an interactive job. Please note that you need to specify the <code>--account</code>, <code>--partition</code> and <code>--qos</code>. Please refer to our [[SLURM]] documentation about about how to further customize your submissions including making a batch submission. The following command will allocate 8 GPUs for 2 days in an interactive session. Change parameters accordingly to your needs. We discourage use of srun and promote use of sbatch for fair use of GPUs.

<pre>
$ srun --pty --gres=gpu:8 --account=gamma --partition=gamma --qos=huge-long bash
$ hostname
gammagpu01.umiacs.umd.edu
$ nvidia-smi -L
GPU 0: NVIDIA RTX A5000 (UUID: GPU-cdfb2e0c-d69f-354b-02f4-15161dc7fa66)
GPU 1: NVIDIA RTX A5000 (UUID: GPU-be53e7a1-b8fd-7089-3cac-7a2fbf4ec7dd)
GPU 2: NVIDIA RTX A5000 (UUID: GPU-774efbb1-d7ec-a0bb-e992-da9d1fa6b193)
GPU 3: NVIDIA RTX A5000 (UUID: GPU-d1692181-c7de-e273-5f95-53ad381614c3)
GPU 4: NVIDIA RTX A5000 (UUID: GPU-ba51fd6c-37bf-1b95-5f68-987c18a6292a)
GPU 5: NVIDIA RTX A5000 (UUID: GPU-c1224a2a-4a3b-ff16-0308-4f36205b9859)
GPU 6: NVIDIA RTX A5000 (UUID: GPU-8d20d6cd-abf5-2630-ab88-6bba438c55fe)
GPU 7: NVIDIA RTX A5000 (UUID: GPU-93170910-5d94-6da5-8a24-f561d7da1e2d)
</pre>

You can also use SBATCH to submit your job. Here are two examples on how to do that.

<pre>
$ sbatch --pty --gres=gpu:8 --account=gamma --partition=gamma --qos=huge-long --time=1-23:00:00 script.sh
</pre>

OR

<pre>
$ sbatch script.sh

// script.sh //

#!/bin/bash
#SBATCH --gres=gpu:8
#SBATCH --account=gamma
#SBATCH --partition=gamma
#SBATCH --qos=huge-long
#SBATCH --time=1-23:00:00

python your_file.py
</pre>

=Storage=
There are 3 types of user storage available to users in GAMMA:
* Home directories
* Project directories
* Scratch directories

There is also read-only storage available for Dataset directories.

GAMMA users can also request [[Nexus#Project_Allocations | Nexus project allocations]].

===Home Directories===
Home directories are available from the Institute's [[NFShomes]] as <code>/nfshomes/USERNAME</code> where USERNAME is your UMIACS username. These home directories have very limited storage (30GB, cannot be increased) and are intended for your personal files, configuration and source code. Your home directory is '''not''' intended for data sets or other large scale data holdings. Users are encouraged to utilize our [[GitLab]] infrastructure to host your code repositories.

Your home directory data is fully protected and has both [[Snapshots | snapshots]] and is [[NightlyBackups | backed up nightly]].

===Project Directories===
You can request project based allocations with approval from a GAMMA faculty member.

To request an allocation, please [[HelpDesk | contact staff]] with the faculty member(s) that approved the project in the conversation. Please include the following details:
* Project Name (short)
* Description
* Size (1TB, 2TB, etc.)
* Length in days (30 days, 90 days, etc.)
* Other user(s) that need to access the allocation, if any

These allocations will be available from '''/fs/gamma-projects''' under a name that you provide when you request the allocation. Near the end of the allocation period, staff will contact you and ask if you would like to renew the allocation (requires re-approval from a GAMMA faculty member). If you are no longer in need of the storage allocation, you will need to relocate all desired data within two weeks of the end of the allocation period. Staff will then remove the allocation. If you do not respond to staff's request by the end of the allocation period, staff will make the allocation temporarily inaccessible. If you do respond asking for renewal but the original faculty approver does not respond within two weeks of the end of the allocation period, staff will also make the allocation temporarily inaccessible. If one month from the end of the allocation period is reached without both you and the faculty approver responding, staff will remove the allocation.

This data is backed up nightly.

===Scratch Directories===
Scratch data has no data protection, there are no snapshots and the data is not backed up.
There are two types of scratch directories:
* Network scratch directory
* Local scratch directories

====Network Scratch Directory====
You are allocated 100GB of scratch space via NFS from <code>/gammascratch/$username</code>. '''It is not backed up or protected in any way.'''

This directory is '''automounted''' so you may not see your directory if you run <code>ls /gammascratch</code> but it will be mounted when you <code>cd</code> into your /gammascratch directory.

You may request a permanent increase of up to 200GB total space without any faculty approval by [[HelpDesk | contacting staff]]. If you need space beyond 200GB, you will need faculty approval.

This file system is available on all submission, data management, and computational nodes within the cluster.

====Local Scratch Directories====
These file systems are not available over [[NFS]] and '''there are no backups or snapshots available''' for these file systems.

* Each computational node that you can schedule compute jobs on has one or more local scratch directories. These are always named <code>/scratch0</code>, <code>/scratch1</code>, etc. These directories are local to each node, ie. the <code>/scratch0</code> on two different nodes are completely separate.

** These directories are almost always more performant than any other storage available to the job. However, you must stage data to these directories within the confines of your jobs and stage the data out before the end of your jobs.

** These local scratch directories have a tmpwatch job which will '''delete unaccessed data after 90 days''', scheduled via maintenance jobs to run once a month during our monthly maintenance windows. Again, please make sure you secure any data you write to these directories at the end of your job.

* Gamma has invested in a 20TB NVMe scratch file system on <code>nexusgamma00.umiacs.umd.edu</code> that is available as <code>/scratch1</code>. To utilize this space, you will need to copy data from/to this over SSH from a compute node. To make this easier, you may want to setup [[SSH]] keys that will allow you to copy data without prompting for passwords.

** The <code>/scratch1</code> directory on <code>nexusgamma00.umiacs.umd.edu</code> doesn't have a tmpwatch. The files in this directory need to be manually removed once they are no longer needed.

===Datasets===
We have read-only dataset storage available at <code>/fs/gamma-datasets</code>. If there are datasets that you would like to see curated and available, please see [[Datasets | this page]].

The list of GAMMA datasets we currently host can be viewed [https://info.umiacs.umd.edu/datasets/list/?q=GAMMA here].

Datasets

2023-10-20T18:10:28Z

Ekr597: Updated the links for CML and Vulcan

UMIACS hosts a number of datasets in read-only mode on some shared filesystems used by some of our [[SLURM]] computing clusters. The motivation behind this is to provide commonly-used datasets in a well-defined location in order to de-duplicate their use elsewhere, thus reducing overall storage usage.

==Dataset Directories==
* [[Nexus/CML | CML]] (<code>/fs/cml-datasets</code>)
** [[Nexus/CML#Datasets | List of datasets]] -- faculty approver Tom Goldstein
* [[Nexus]] (<code>/fs/nexus-datasets</code>)
** [[Nexus#Datasets | List of datasets]]
* [[Nexus/Vulcan | Vulcan]] (<code>/fs/vulcan-datasets</code>)
** [[Nexus/Vulcan#Datasets | List of datasets]] -- faculty approver Abhinav Shrivastava

==Requesting a new dataset==
You can request a new dataset by [[HelpDesk | contacting staff]] with a link to the dataset's official download location. Torrents or other peer-to-peer re-hosting are not allowed unless sanctioned by the dataset owners.
* '''CML/Vulcan''': If the uncompressed/final dataset size is over 100GB, staff will first contact the faculty approver for the cluster to ensure they approve of using the storage space. If the size is under 100GB, no faculty approval is required. Staff will then inspect the dataset and see if there are any terms and conditions that must be agreed to before downloading.
* '''Nexus''': Please let staff know which faculty member's research you are working on that requires use of the dataset you are requesting. Then, if the uncompressed/final dataset size is over 50GB, the [https://www.umiacs.umd.edu/people/computing-staff Director of Computing Facilities] must first approve of using the storage space. If the size is under 50GB, no approval is required. Staff will then inspect the dataset and see if there are any terms and conditions that must be agreed to before downloading.

If there are no terms and conditions, staff will download/extract the dataset, copy it to the appropriate location depending on what cluster you are requesting it for, and let you know when it is available for use on that cluster.

If there are terms and conditions in excess of well-defined [https://creativecommons.org/about/cclicenses/ Creative Commons licenses], staff will first need to have the dataset's terms and conditions approved by [https://ora.umd.edu/ UMD's Office of Research Administration (ORA)]. Since the dataset will be hosted by UMIACS (as an institution) in a location accessible by all users of a cluster, not all of which will have individually agreed to the terms and conditions, having a single person (staff member or not) agree to a set of terms and conditions is not sufficient to host it. After approval, staff will perform the same steps as mentioned above (download/extract, copy to appropriate location, and let you know when ready).

==Dataset use==
All datasets are read-only to users. Any intermediate data generated from a dataset will need to be stored in a location other than the shared filesystem hosting the dataset and other datasets.

Exceptions may be granted if there is a set of intermediate data generated from a dataset that you believe will be useful to a subset of a cluster's users. If you suspect this is the case for some of your generated data, please [[HelpDesk | contact staff]]. We will follow the above procedure and upon whatever approvals may be necessary, copy the intermediate data into the shared filesystem.

ClassAccounts

2023-10-16T14:49:38Z

Ekr597:

==Overview==
UMIACS Class Accounts are currently intended to support classes for all of UMIACS/CSD via the [[Nexus]] cluster. All new class accounts are serviced solely through this cluster. Faculty may request that a class be supported by following the instructions [[ClassAccounts/Manage | here]].

==Getting an account==
Your TA will request an account for you. Once this is done, you will be notified by email that you have an account to redeem. If you have not received an email, please contact your TA. '''You must redeem the account within 7 days or else the redemption token will expire.''' If your redemption token does expire, please contact your TA to have it renewed.

Once you do redeem your account, you will need to wait until you get a confirmation email that your account has been installed. This is typically done once a day on days that the University is open for business.

===Registering for Duo===
UMIACS requires that all Class accounts be registered for MFA (multi-factor authentication) under our [[Duo]] instance (note that this is different than UMD's general Duo instance). '''You will not be able to log onto the class submission host until you register.'''

If you see the following error in your SSH client you have not yet enrolled/registered in Duo.

<pre>
Access is not allowed because you are not enrolled in Duo. Please contact your organization's IT help desk.
</pre>

In order to register, [https://intranet.umiacs.umd.edu/directory visit our directory app] and log in with your Class username and password. You will then receive a prompt to enroll in Duo. For assistance in enrollment, you can visit our [[Duo | Duo help page]].

Once notified that your account has been installed and you have registered in our Duo instance, you can access the following class submission host(s) using [[SSH]] with your assigned username and your chosen password:
* <code>nexusclass00.umiacs.umd.edu</code> or <code>nexusclass01.umiacs.umd.edu</code>

==Cleaning up your account before the end of the semester==
Class accounts for a given semester are liable to be archived and deleted after that semester's completion as early as the following:
* Winter semesters: February 1st of same year
* Spring semesters: June 1st of same year
* Summer semesters: September 1st of same year
* Fall semesters: January 1st of next year

It is your responsibility to ensure you have backed up anything you want to keep from your class account's personal or group storage (below sections) prior to the relevant date.

==Personal Storage==
Your home directory has a quota of 30GB and is located at:
<pre>
/fs/classhomes/<semester><year>/<coursecode>/<username>
</pre>

where <code><semester></code> is either "spring", "summer", "fall", or "winter", <code><year></code> is the current year e.g., "2021", <coursecode> is the class' course code as listed in UMD's [https://app.testudo.umd.edu/soc/ Schedule of Classes] in all lowercase e.g., "cmsc999z", and <code><username></code> is the username mentioned in the email you received to redeem the account e.g., "c999z000".

You can request up to another 100GB of personal storage if you would like by having your TA [[HelpDesk | contact staff]]. This storage will be located at
<pre>
/fs/class-projects/<semester><year>/<coursecode>/<username>
</pre>

==Group Storage==
You can also request group storage if you would like by having your TA [[HelpDesk | contact staff]] to specify the usernames of the accounts that should be in the group. Only other class accounts in the same class can be added to the group. The quota will be 100GB multiplied by the number of accounts in the group and will be located at
<pre>
/fs/class-projects/<semester><year>/<coursecode>/<groupname>
</pre>

where <code><groupname></code> is composed of:
* the abbreviated course code as used in the username e.g., "c999z"
* the character "g"
* the number of the group (starting at 0 for the first group for the class requested to us) prepended with 0s to make the total group name 8 characters long

e.g., "c999zg00".

==Cluster Usage==
'''You may not run computational jobs on any submission host.''' You must schedule your jobs with the [[SLURM]] workload manager. You can also find out more with the public documentation for the [https://slurm.schedmd.com/quickstart.html SLURM Workload Manager].

'''Any questions or issues with the cluster must be first made through your TA.'''

Class accounts only have access to the following submission parameters in SLURM. You may be required to explicitly set each of these in your submission parameters.

* Partition - <code>class</code>
* Account - <code>class</code>
* QoS - <code>default</code>, <code>medium</code>, and <code>high</code>

Please note that you will be restricted to 32 total cores, 256GB total RAM, and 4 total GPUs across all jobs you have running at once.

===Example===
Here is a basic example to schedule a interactive job running bash with a single GPU in the partition <code>class</code> with the account <code>class</code> running with the QoS of <code>default</code>.

<pre>
$ srun --pty --partition=class --account=class --qos=default --gres=gpu:1 bash
</pre>

<pre>
bash-4.4$ hostname
tron14.umiacs.umd.edu
bash-4.4$ nvidia-smi -L
GPU 0: NVIDIA RTX A4000 (UUID: GPU-55f2d3b7-9162-8b02-50de-476a012c626c)
</pre>

===Available Nodes===
You can list the available nodes and their current state with the <code>show_nodes -p class</code> command. This list of nodes is not completely static as nodes may be pulled out of service to repair/replace GPUs or other components.

<pre>
$ show_nodes -p class
NODELIST CPUS MEMORY AVAIL_FEATURES GRES STATE PARTITION
tron06 16 128520 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron07 16 128520 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron08 16 128520 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron09 16 128520 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron10 16 128524 rhel8,Zen,EPYC-7313P gpu:rtxa4000:4 idle class
tron11 16 128524 rhel8,Zen,EPYC-7313P gpu:rtxa4000:4 idle class
tron12 16 128525 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron13 16 128520 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron14 16 128520 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron15 16 128520 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron16 16 128524 rhel8,Zen,EPYC-7313P gpu:rtxa4000:4 idle class
tron17 16 128524 rhel8,Zen,EPYC-7313P gpu:rtxa4000:4 idle class
tron18 16 128524 rhel8,Zen,EPYC-7313P gpu:rtxa4000:4 idle class
tron19 16 128524 rhel8,Zen,EPYC-7313P gpu:rtxa4000:4 idle class
tron20 16 128524 rhel8,Zen,EPYC-7313P gpu:rtxa4000:4 idle class
tron21 16 128525 rhel8,AMD,EPYC-7302P gpu:rtxa4000:4 idle class
tron22 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron23 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron24 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron25 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron26 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron27 16 128521 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron28 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron29 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron30 16 128521 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron31 16 128521 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron32 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron33 16 128521 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron34 16 128524 rhel8,Zen,EPYC-7313P gpu:rtxa4000:4 idle class
tron35 16 128521 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron36 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron37 16 128521 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron38 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron39 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron40 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron41 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron42 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron43 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron44 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
tron45 16 128525 rhel8,AMD,EPYC-7302 gpu:rtxa4000:4 idle class
</pre>

You can also find more granular information about an individual node with the <code>scontrol show node</code> command.

<pre>
$ scontrol show node tron27
NodeName=tron27 Arch=x86_64 CoresPerSocket=16
CPUAlloc=0 CPUTot=16 CPULoad=0.00
AvailableFeatures=rhel8,AMD,EPYC-7302
ActiveFeatures=rhel8,AMD,EPYC-7302
Gres=gpu:rtxa4000:4
NodeAddr=tron27 NodeHostName=tron27 Version=21.08.8-2
OS=Linux 4.18.0-372.19.1.el8_6.x86_64 #1 SMP Mon Jul 18 11:14:02 EDT 2022
RealMemory=128521 AllocMem=0 FreeMem=125650 Sockets=1 Boards=1
State=IDLE ThreadsPerCore=1 TmpDisk=0 Weight=10 Owner=N/A MCS_label=N/A
Partitions=class,scavenger,tron
BootTime=2022-08-18T17:34:44 SlurmdStartTime=2022-08-19T13:10:47
LastBusyTime=2022-08-22T11:20:18
CfgTRES=cpu=16,mem=128521M,billing=173,gres/gpu=4,gres/gpu:rtxa4000=4
AllocTRES=
CapWatts=n/a
CurrentWatts=0 AveWatts=0
ExtSensorsJoules=n/s ExtSensorsWatts=0 ExtSensorsTemp=n/s
</pre>

Gurobi

2023-08-29T16:53:23Z

Ekr597:

Gurobi Optimizer is a suite of solvers for mathematical programming. It can be accessed through our module tree with the command <code>module add gurobi</code>. More information on our [[Modules | Modules page]]

Documentation can be found at https://www.gurobi.com/documentation/

==Error 10009==
Gurobi will not work on the [[Nexus]] submission nodes since they have a public IP address and will give the error message "Error 10009: Server must be on the same subnet".

If you encounter this error, try running the command in a [[SLURM]] job. If you're still having issues, please contact the [[HelpDesk]].

OBJ

2023-06-12T14:20:41Z

Ekr597: Increased faculty allocation to 10TB

= UMIACS Object Store =
An object store is a web-based storage solution focused on reliability, scalability and security. It is best suited for public content storage/distribution, archiving data or secure data sharing between users. Our Object Storage can be used through the [https://obj.umiacs.umd.edu/obj web interface], the command line [[UMobj]] utilities, third-party graphical [[S3Clients | clients]], and even programmatically using many popular programming languages. We support a subset of the Amazon Simple Storage Services [http://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html (S3) API], built around a technology called [http://ceph.com/ Ceph].

= Terminology =
S3-like storage thinks in terms of buckets and keys. Keys are analogous to files. A bucket is simply a container to a set of keys. There is no actual hierarchy inside of a bucket, but the standard UNIX path separator, a forward slash (/) at the end of a Key name, is interpreted by many clients (including this web site and our umobj utilities) as being a directory delimiter. This allows you to copy data from your local filesystems to your buckets through umobj or third-party clients. You may specify who has what types of access to your buckets via Access Control Lists (ACLs) at the bucket level or the individual key level.

Your data is protected from individual machine failure via replication within the cluster. All data is checksummed in accordance with the Amazon S3 protocol to ensure that data in transit is valid before it is accepted by the cluster. However, there are no backups or snapshots of this data in the cluster, so '''if a user deletes a key or bucket in the object store, there is no way to restore that information'''.

= Getting Started =
UMIACS users are allocated 50GB of storage. Faculty are allocated 10TB. To get started, [https://obj.umiacs.umd.edu/obj log in] and you will be redirected to the initial help page. You can also find the link from our https://intranet.umiacs.umd.edu site as "OBJbox Object Store".

= Buckets =
You can create and browse your buckets (containers that hold data) by visiting your [https://obj.umiacs.umd.edu/obj/buckets/ buckets] page. You can also set bucket-level Access Control Lists (ACLs) from this page. Bucket-level ACLs get implicitly inherited to all the keys within the bucket. However, individual keys can have additional specific ACLs applied for more granular control.

Bucket names must be unique. When you create a bucket it will notify you if the name is already taken.

= Keys (files) =
After selecting a bucket, you will be able to create folders and upload files within that bucket. Listed files can be downloaded, deleted, or assigned a specific ACL by the key owner/creator.

'''Please note: Local file system ownership and permissions, and special files (such as symlinks) can not be represented in the object store. We highly suggest that if you are securing data into the object store and need these to be faithfully maintained that you use a local archive tool (tar, zip, etc..) to collect the data and then upload the resulting archive file(s).'''

= Hosting a Website in your Bucket =
Please visit [[OBJ/WebHosting]] for more information.

= Deleting Keys (files) =
Within the web interface you can delete files one-by-one. If you want to remove a bunch of files, you will need to use a different client as described below.

'''This is dangerous as there are no backups of files in the object store. Be careful to only delete the data you intend to delete.'''

= Clients =
There are several clients that can be used (sometimes with a limited set of features) on your desktop to gain access to the Object Store. All supported UMIACS systems have a copy of our [https://gitlab.umiacs.umd.edu/staff/umobj/blob/master/README.md#umobj umobj] utilities which provide command line access to the object store. We also have an article in our wiki on [[S3Clients | 3rd party clients]] that lists and explains the details. These clients need to be configured with your Access and Secret Keys as described below.

= Access Key and Secret Key =
Each user has one or more pairs of Access Keys and Secret Keys that are used as a credential to not expose your password when using the object store. These can be obtained by clicking on your [https://obj.umiacs.umd.edu/obj/user/ username] in the upper right-hand corner. You'll use these to identify and authenticate yourself to the Object Store.

When using the [https://gitlab.umiacs.umd.edu/staff/umobj/blob/master/README.md#umobj umobj] utilities, you will need to make sure you have added these credentials in your local shell initialization files. There are links to files that have these automatically generated for the 3 most popular UNIX shell families (bash/sh, csh/tcsh, and zsh). Please make sure that whatever file(s) you copy these credentials into can not be read by other users (eg. chmod 600 filename).

'''Note:''' Each Access Key and Secret Key are specific to a particular object store, so if you are accessing multiple object stores you may want to write the credentials for each to separate files and then source each file when you want to use the associated object store. Please [[HelpDesk | contact staff]] if you have any questions.

= Lab Groups =
Lab Groups allow a group of users to share data while avoiding the need for complex ACLs by maintaining group ownership. Designated Lab Group managers can grant granular access (read, write, full control, manager) to buckets owned by the Lab Group. All objects owned by a Lab Group count against the group quota. Lab Groups can be navigated using the menu with username in the top right corner of the page. '''Note:''' this will only appear if you are a member of at least one Lab Group. At this point, you can browse the Object Store as the Lab Group and obtain your unique Access Key and Secret Key pair using the instructions above. To switch to another Lab Group or back to your own buckets, click the menu again and select another user or group.

= Managing Lab Groups =
Lab Groups have many different levels of membership: '''Managers, FULL_CONTROL, READ/WRITE,''' and '''READ'''. Managers can add or remove Lab Group Members while every other access level cannot. If you hold the Manager role in a Lab Group, you can add and remove users using the [https://obj.umiacs.umd.edu/obj/labgroup/list/ Manage LabGroups] page, which is available under the Manage menu at the top of the page. After selecting a Lab Group, you can add users by typing their username into the search field and selecting a membership role.

= Requesting a Lab Group =
To request a Lab Group for your project, please [[HelpDesk | contact staff]].

SecureShell

2020-05-05T23:44:29Z

Ekr597: Updated steps for verifying SSH fingerprints to reflect the updates to OpenSSH (ssh-keygen defaults to SHA256 instead of MD5)

Secure Shell (or [http://en.wikipedia.org/wiki/Secure_Shell SSH]) is a network protocol allowing two computers to exchange data securely over an insecure network. By default, use of SSH brings the user to a terminal, but the protocol can be used for other types of data transfer such as [[SFTP]] and [[SCP]].

==Connecting to an SSH Server==
Under Linux and macOS the following command from a terminal will connect a client computer to the UMIACS [[OpenLAB]].
# ssh bkirz@openlab.umiacs.umd.edu
This will give you access to a terminal on any one of the [[OpenLAB]] servers. Note that by default you will not have access to applications that require X11 to run.

All UMIACS Windows hosts are installed with either the SSH Secure Shell Client or [http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY].

==X11 Forwarding==
By default, SSH only gives the user shell access to a host. Enabling X11 Forwarding allows users to run applications with Graphical User Interfaces.

Under Linux and macOS, the following command from a terminal will connect a client computer to the UMIACS [[OpenLAB]] using X11 Forwarding. Please note that under macOS, [http://xquartz.macosforge.org/landing/ xQuartz] is required on the client machine to forward X sessions from the remote session.
# ssh '''-Y''' bkirz@openlab.umiacs.umd.edu

Under Windows, you will need to forward X through [http://sourceforge.net/projects/vcxsrv/ VcXsrv] or [http://www.straightrunning.com/XmingNotes/ Xming].

First, enable X forwarding on PuTTY. The option is under Connection > SSH > X11, shown below.

[[Image:Putty-x-forwarding.png]]

Next, configure your SSH session and click open to start a SSH session.

After this has been done, every time you want to use X forwarding, you need to make sure VcXsrv or Xming has been started (it will appear in your task tray) through the start menu programs.
Now, you will be able to use Xwindow programs from your ssh client.

==SSH Tunneling==

You can tunnel one or more ports through an SSH connection such that your packets will look like they are coming from the host you are tunneling to. This is helpful for services that you would be normally blocked by a firewall.

Please see the [[SecureShellTunneling]] page for more information.

==SSH Keys (and Passwordless SSH)==

There are some situations where it is important to be able to ssh without entering a password. This is mostly required when working in clusters. This is done using ssh keys. Instead of authenticating with a password, ssh can use a pre-defined set of encryption keys to establish an authorized connection.
To setup passwordless ssh, do the following.

First, you will need to create a ssh [http://en.wikipedia.org/wiki/Key_pair key pair]. It is possible to use a password that you will need to enter at the beginning of your work session. This is preferable as it is more secure but may cause problems for some clustered work. If you simply hit '''[enter]''', you will never be prompted for a password when ssh'ing which can lead to security problems.

* To create a '''''passwordless''''' key, type the following, and then hit enter to place the keys in the default directory.
<pre> # ssh-keygen -N ""</pre>
* Alternatively, to create a '''''passphrase-protected''''' (more-secure) key, type the following.
<pre> # ssh-keygen</pre>

This will produce two files, '''id_rsa''' and '''id_rsa.pub''', the private and public keys, respectively. The default location will be ~/.ssh/. For the purposes of this tutorial we'll assume this default. Once you've created the keys, you will need to put them into place as follows:
# chmod 700 ~/.ssh
# chmod 600 ~/.ssh/id_rsa
# touch ~/.ssh/authorized_keys
# chmod 600 ~/.ssh/authorized_keys
# cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# rm ~/.ssh/id_rsa.pub

*It is '''very''' important that you keep your private key secure! Ensure that it is chmod'd to 600 and that you do not put it anywhere visible to other users!
*You must also make sure that no other users may write to your .ssh directory. This includes making sure that your home directory is not writable by group. Your home directory should be chmod'd to 750 or 700 to make sure of this. If the group write bit is set, your ssh keys '''WILL NOT WORK'''

If you did not select a passphrase when you generated your keys, you can now ssh without a password. If you did select a passphrase, you will need to activate the keys as follows:

# ssh-agent [SHELL]
# ssh-add -t [TIME]

In this case, "[SHELL]" is your preferred shell and "[TIME]" is the amount of time you'd like the key to be active in seconds. So, the following would start a bash shell with passwordless ssh active for 30 minutes:

# ssh-agent bash
# ssh-add -t 1800

You will be prompted for your passphrase and, when entered correctly, you will be able to ssh without entering a password.

To disable this functionality, simply delete your private key file ('''~/.ssh/id_rsa''') and remove the public key from your '''~/.ssh/authorized_keys''' file.

==Verify remote host SSH fingerprint==
The SSH protocol relies on host keys to verify the identify of a given host. Each host as a unique key for the various different protocols supported.

When connecting to a remove host for the first time, or when the remote host's local host key configuration has changed, you may see the following prompt:
<pre>
$ ssh sabobbin@openlab
The authenticity of host 'openlab (128.8.132.247)' can't be established.
RSA key fingerprint is 25:83:aa:df:f5:ad:5f:08:c9:8a:a3:5d:97:8b:48:1f.
Are you sure you want to continue connecting (yes/no)?
</pre>
It is considered best practice to verify the key fingerprint with the actual key of the host. UMIACS maintains a reference of SSH key fingerprints available at the following link:
https://gitlab.umiacs.umd.edu/staff/ssh-fingerprints/blob/master/fingerprints

It is important to note that each key type has a different fingerprint. Depending on your local configuration, your client may prefer a specific type of key. The following commands can be used to determine the fingerprint of a given key type on a remote host:

<pre>
$ ssh-keyscan -t rsa openlab.umiacs.umd.edu > key
# openlab.umiacs.umd.edu:22 SSH-2.0-OpenSSH_8.0
$ ssh-keygen -l -E md5 -f key
2048 MD5:25:83:aa:df:f5:ad:5f:08:c9:8a:a3:5d:97:8b:48:1f openlab.umiacs.umd.edu (RSA)
</pre>

If you have any questions, or notice a discrepancy, please submit a request to staff@umiacs.umd.edu.

===Windows / PuTTY Verification===
If you use PuTTY to connect to remote hosts, the prompt will be similar to the following:

[[File:Putty ssh host key prompt.png]]

If the host key reported by PuTTY matches the [https://gitlab.umiacs.umd.edu/staff/ssh-fingerprints/blob/master/fingerprints Documented entry for that host], it is safe to click 'yes'. If they do not match, please report the issue to [mailto:staff@umiacs.umd.edu staff@umiacs.umd.edu].

===Other Platforms===
* [https://winscp.net/eng/docs/faq_hostkey WinSCP]

==Long Running Processes==
If you are dealing with a long running process that is inhibiting your ability to work regularly, you may want to run your processes inside a screen on the host that you're connecting to. This way, if the connection is dropped for any reason the screen session will automatically detach on the host and will continue running so that you can reattach it at a later time when you've connected again. Please see our documentation on [[Screen | GNU Screen]] for more information.

==Further Information==
[http://www.openssh.org/ OpenSSH]

[http://www.openssh.com/windows.html Windows Clients]

Mathematica

2019-07-26T19:48:50Z

Ekr597: Changed instructions to use modules instead of directing users to /opt/common

Mathematica is freely available to all University-owned machines.

On our UMIACS-supported Linux hosts, Mathematica can be accessed through our [[Modules]].
*The command <code>module add mathematica</code> will add the default version of Mathematica to your Environment.
*To see the versions of Mathematica that are available use the command <code>module avail mathematica</code>.
*To add a specific version of Mathematica to your Environment (i.e. Mathematica 12.0) use the command <code>module add mathematica/12.0</code>
*Further information can be found on our [[Modules | Modules page]].

For UMIACS-supported Windows machines, or other self-supported University-owned equipment, please contact [[HelpDesk | staff]].

==Activation==
There is no automated way to activate Mathematica across our domain. As a result, each computer will have to be registered once against our hosted license server. Any user can go through this process, and it should persist until the host is reinstalled.

* Upon being prompted, click "Other ways to activate" in the bottom row: [[Image:math1.jpg| 500px| Mathematica 10 Activation Screen 1]] 
* Click "Connect to a network license server": [[Image:math2.jpg| 500px| Mathematica 10 Activation Screen 2]] 
* Enter "licserv.umiacs.umd.edu" as the license server. Click "Activate": [[Image:math3.jpg| 500px| Mathematica 10 Activation Screen 3]] 
* Accept the terms, and click ok: [[Image:math4.jpg| 500px| Mathematica 10 Activation Screen 4]]
* Mathematica should now be activated for that machine.