UMIACS - User contributions [en]

Iribe/ConferenceRooms/Moderated

2019-09-06T13:17:18Z

Dkontyko: added room name column

These signature [[Iribe/ConferenceRooms | rooms]] have a primary coordinator and a backup moderator. The touch panels will not allow walk up reservations for these rooms; all reservations need to go through the moderation process. You may [[Iribe/ConferenceRooms/Reserve | create a reservation]] by adding the room in Google Calendar. The moderator will be notified and can approve your request.

Instructions on reserving a room are [[Iribe/ConferenceRooms/Reserve | here]].

{| class="wikitable"
! Room
! Name
! Occupancy
! Primary Moderator
! Backup
|-
| IRB-1127
|
| 12
| Sharron McElroy
| Regis Boykin
|-
| IRB-2137
|
| 12
| Sharron McElroy
| Regis Boykin
|-
| IRB-3137
|
| 24
| Elizabeth Hontz
| Danae Johnson
|-
| IRB-3256
|
| 8
| Barbara Lewis
| Danae Johnson
|-
| IRB-4105
|
| 48
| Sharron McElroy
| Elizabeth Hontz
|-
| IRB-4107
|
| 20
| Sharron McElroy
| Elizabeth Hontz
|-
| IRB-4109
|
| 20
| Sharron McElroy
| Elizabeth Hontz
|-
| IRB-4137
|
| 12
| Elizabeth Hontz
| Danae Johnson
|-
| IRB-4237
|
| 12
| Janice Perrone
| Danae Johnson
|-
| IRB-5105
|
| 24
| Sharron McElroy
| Danae Johnson
|-
| IRB-5137
|
| 18
| Sharron McElroy
| Danae Johnson
|-
| IRB-5165
|
| 16
| Sharron McElroy
| Danae Johnson
|-
| IRB-5237
|
| 12
| Dana Purcell
| Danae Johnson
|}

=== Large Seminar Capabilities ===
'''3137''' and '''4105''' are the two large seminar rooms.

* Dual Display (Projector and LCD)
* Dual camera conferencing via room PC
* Laptop presentation via HDMI or Mersive Solstice
* Blu-ray playback, tuner
* Lectern

===Small Seminar and Large Conference Room Capabilities ===
Every other room on this page has the following setup.

* Single Display (LCD)
* Single camera conferencing via room PC
* Laptop presentation via HDMI or Mersive Solstice

SLURM/JobSubmission

2019-08-19T18:58:50Z

Dkontyko: /* Common srun arguments */

=Job Submission=

SLURM offers a variety of ways to run jobs. It is important to understand the different options available and how to request the resources required for a job in order for it to run successfully. All job submission should be done from submit nodes; any computational code should be run in a job allocation on compute nodes. The following commands outline how to allocate resources on the compute nodes and submit processes to be run on the allocated nodes.

==srun==
<code>srun</code> is the command used to run a process on the compute nodes in the cluster. It works by passing it a command (this could be a script) which will be run on a compute node and then <code>srun</code> will return. <code>srun</code> accepts many command line options to specify the resources required by the command passed to it. Some common command line arguments are listed below and full documentation of all available options is available in the man page for <code>srun</code>, which can be accessed by running <code>man srun</code>.
<pre>
tgray26@opensub01:srun --mem=100mb --time=1:00:00 bash -c 'echo "Hello World from" `hostname`'
Hello World from openlab06.umiacs.umd.edu
</pre>
It is important to understand that <code>srun</code> is an interactive command. By default input to <code>srun</code> is broadcast to all compute nodes running your process and output from the compute nodes is redirected to <code>srun</code>. This behavior can be changed; however, '''srun will always wait for the command passed to finish before exiting, so if you start a long running process and end your terminal session, your process will stop running on the compute nodes and your job will end'''. To run a non-interactive submission that will remain running after you logout, you will need to wrap your <code>srun</code> commands in a batch script and submit it with [[#sbatch | sbatch]]
===Common srun arguments===
* <code>--mem=1gb</code> ''if no unit is given MB is assumed''
* <code>--nodes=2</code> ''if passed to srun, the given command will be run concurrently on each node''
* <code>--qos=dpart</code> ''to see the available QOS options on a cluster, run'' <code>show_qos</code>
* <code>--time=hh:mm:ss</code> ''time needed to run your job''
* <code>--job-name=helloWorld</code>
* <code>--output filename</code> ''file to redirect stdout to''
* <code>--error filename</code> ''file to redirect stderr''
* <code>--partition $PNAME</code> ''request job run in the $PNAME partition''
* <code>--ntasks 2</code> ''request 2 "tasks" which map to cores on a CPU, if passed to srun the given command will be run concurrently on each core''

===Interactive Shell Sessions===
An interactive shell session on a compute node can be useful for debugging or developing code that isn't ready to be run as a batch job. To get an interactive shell on a node, use <code>srun</code> to invoke a shell:
<pre>
tgray26@opensub01:srun --pty --mem 1gb --time=01:00:00 bash
tgray26@openlab06:
</pre>
'''Please do not leave interactive shells running for long periods of time when you are not working. This blocks resources from being used by everyone else.'''

==salloc==
The salloc command can also be used to request resources be allocated without needing a batch script. Running salloc with a list of resources will allocate the resources you requested, create a job, and drop you into a subshell with the environment variables necessary to run commands in the newly created job allocation. When your time is up or you exit the subshell, your job allocation will be relinquished.
<pre>
tgray26@opensub00:salloc -N 1 --mem=2gb --time=01:00:00
salloc: Granted job allocation 159
tgray26@opensub00:srun /usr/bin/hostname
openlab00.umiacs.umd.edu
tgray26@opensub00:exit
exit
salloc: Relinquishing job allocation 159
</pre>
'''Please note that any commands not invoked with srun will be run locally on the submit node. Please be careful when using salloc.'''

==sbatch==
The sbatch command allows you to write a batch script to be submitted and run non-interactively on the compute nodes. To run a simple Hello World command on the compute nodes you could write a file, helloWorld.sh with the following contents:
<pre>
#!/bin/bash

srun bash -c 'echo Hello World from `hostname`'
</pre>
Then you need to submit the script with sbatch and request resources:
<pre>tgray26@opensub00:sbatch --mem=1gb --time=1:00:00 helloWorld.sh
Submitted batch job 121
</pre>
SLURM will return a job number that you can use to check the status of your job with squeue:
<pre>
tgray26@opensub00:squeue
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
121 dpart helloWor tgray26 R 0:01 2 openlab[00-01]
</pre>
====Advanced Batch Scripts====
You can also write a batch script with all of your resources/options defined in the script itself. This is useful for jobs that need to be run 10s/100s/1000s of times. You can then handle any necessary environment setup and run commands on the resources you requested by invoking commands with srun. The srun commands can also be more complex and be told to only use portions of your entire job allocation, each of these distinct srun commands makes up one "job step". The batch script will be run on the first node allocated as part of your job allocation and each job step will be run on whatever resources you tell them to. In the following example I have a batch job that will request 2 nodes in the cluster, then I load a specific version of Python into my environment and submit two job steps, each one using one node. Since srun is blocks until the command finishes, I use the '&' operator to background the process so that both job steps can run at once; however, this means that I then need to use the wait command to block processing until all background processes have finished.
<pre>
#!/bin/bash

# Lines that begin with #SBATCH specify commands to be used by SLURM for scheduling

#SBATCH --job-name=helloWorld # sets the job name
#SBATCH --output helloWorld.out.%j # indicates a file to redirect STDOUT to; %j is the jobid
#SBATCH --error helloWorld.out.%j # indicates a file to redirect STDERR to; %j is the jobid
#SBATCH --time=00:05:00 # how long you think your job will take to complete; format=hh:mm:ss
#SBATCH --qos=dpart # set QOS, this will determine what resources can be requested
#SBATCH --nodes=2 # number of nodes to allocate for your job
#SBATCH --ntasks=4 # request 4 cpu cores be reserved for your node total
#SBATCH --ntasks-per-node=2 # request 2 cpu cores be reserved per node
#SBATCH --mem 1gb # memory required by job; if unit is not specified MB will be assumed

module load Python/2.7.9 # run any commands necessary to setup your environment

srun -N 1 --mem=512mb bash -c "hostname; python --version" & # use srun to invoke commands within your job; using an '&'
srun -N 1 --mem=512mb bash -c "hostname; python --version" & # will background the process allowing them to run concurrently
wait # wait for any background processes to complete

# once the end of the batch script is reached your job allocation will be revoked
</pre>

Another useful thing to know is that you can pass additional arguments into your sbatch scripts on the command line and reference them as <code>${1}</code> for the first argument and so on.

====More Examples====

* [[SLURM/ArrayJobs]]

===scancel===
The scancel command can be used to cancel job allocations or job steps that are no longer needed. It can be passed individual job IDs or an option to delete all of your jobs or jobs that meet certain criteria.
*<code>scancel 255</code> ''cancel job 255''
*<code>scancel 255.3</code> ''cancel job step 3 of job 255''
*<code>scancel --user tgray26 --partition dpart</code> ''cancel all jobs for tgray26 in the dpart partition''

=Identifying Resources and Features=
The sinfo can show you additional features of nodes in the cluster but you need to ask it to show some non-default options using a command like this
<code>sinfo -o "%15N %10c %10m %25f %10G"</code>.

<pre>
$ sinfo -o "%40N %8c %8m %20f %25G"
NODELIST CPUS MEMORY AVAIL_FEATURES GRES
openlab[30-33] 64 257759 Opteron,6274 (null)
openlab[00-07] 8 7822 Opteron,2354 (null)
openlab[10-11,13-18,20-23,25,27-29] 16 23939 Xeon,x5560 (null)
openlab08 32 128720 Xeon,E5-2690 gpu:k20:2
openlab09 32 128722 Xeon,E5-2690 gpu:m40:1,gpu:k20:2
</pre>

You can also identify further specific information about a node using [https://wiki.umiacs.umd.edu/umiacs/index.php/SLURM/ClusterStatus#scontrol scontrol].

=Requesting GPUs=
If you need to do processing on a GPU, you will need to request that your job have access to GPUs just as you need to request processors or cpu cores. You will also need to make sure that you submit your job to the correct partition since nodes with GPUs are often put into their own partition to prevent the nodes from being tied up by jobs that don't utilize GPUs. In SLURM, GPUs are considered "generic resources" also known as GRES. To request some number of GPUs be reserved/available for your job you can use the flag <code>--gres:gpu:2</code> or if there are multiple types of GPUs available in the cluster and you need a specific type, you can provide the type option to the gres flag <code>--gres:k20:1</code>
<pre>
tgray26@opensub01:srun --pty --partition gpu --qos=gpu --gres=gpu:2 nvidia-smi
Wed Jul 13 15:33:18 2016
+------------------------------------------------------+
| NVIDIA-SMI 361.28 Driver Version: 361.28 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K20c Off | 0000:03:00.0 Off | 0 |
| 30% 24C P0 48W / 225W | 11MiB / 4799MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 Tesla K20c Off | 0000:84:00.0 Off | 0 |
| 30% 23C P0 52W / 225W | 11MiB / 4799MiB | 93% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
</pre>
Please note that your job will only be able to see/access the GPUs you requested. If you only need 1 GPU, please request only 1 GPU and the other one will be left available for other users:
<pre>
tgray26@opensub01:srun --pty --partition gpu --qos=gpu --gres=gpu:k20:1 nvidia-smi
Wed Jul 13 15:31:29 2016
+------------------------------------------------------+
| NVIDIA-SMI 361.28 Driver Version: 361.28 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K20c Off | 0000:03:00.0 Off | 0 |
| 30% 24C P0 50W / 225W | 11MiB / 4799MiB | 92% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
</pre>
The <code>--gres</code> flag may also be passed to [[#sbatch | sbatch]] and [[#salloc | salloc]] rather than directly to [[#srun | srun]]

=MPI example=
<pre>
#!/usr/bin/bash
#SBATCH --job-name=mpi_test # Job name
#SBATCH --nodes=4 # Number of nodes
#SBATCH --ntasks=8 # Number of MPI ranks
#SBATCH --ntasks-per-node=2 # Number of MPI ranks per node
#SBATCH --ntasks-per-socket=1 # Number of tasks per processor socket on the node
#SBATCH --time=00:30:00 # Time limit hrs:min:sec

module load mpi

srun --mpi=openmpi /nfshomes/derek/testing/mpi/a.out
</pre>

SLURM/JobSubmission

2019-08-19T18:18:00Z

Dkontyko: /* Common srun arguments */ adding qos option list command

=Job Submission=

SLURM offers a variety of ways to run jobs. It is important to understand the different options available and how to request the resources required for a job in order for it to run successfully. All job submission should be done from submit nodes; any computational code should be run in a job allocation on compute nodes. The following commands outline how to allocate resources on the compute nodes and submit processes to be run on the allocated nodes.

==srun==
<code>srun</code> is the command used to run a process on the compute nodes in the cluster. It works by passing it a command (this could be a script) which will be run on a compute node and then <code>srun</code> will return. <code>srun</code> accepts many command line options to specify the resources required by the command passed to it. Some common command line arguments are listed below and full documentation of all available options is available in the man page for <code>srun</code>, which can be accessed by running <code>man srun</code>.
<pre>
tgray26@opensub01:srun --mem=100mb --time=1:00:00 bash -c 'echo "Hello World from" `hostname`'
Hello World from openlab06.umiacs.umd.edu
</pre>
It is important to understand that <code>srun</code> is an interactive command. By default input to <code>srun</code> is broadcast to all compute nodes running your process and output from the compute nodes is redirected to <code>srun</code>. This behavior can be changed; however, '''srun will always wait for the command passed to finish before exiting, so if you start a long running process and end your terminal session, your process will stop running on the compute nodes and your job will end'''. To run a non-interactive submission that will remain running after you logout, you will need to wrap your <code>srun</code> commands in a batch script and submit it with [[#sbatch | sbatch]]
===Common srun arguments===
* <code>--mem=1gb</code> ''if no unit is given MB is assumed''
* <code>--nodes=2</code> ''if passed to srun, the given command will be run concurrently on each node''
* <code>--qos=dpart</code> ''to see the available QOS options on a cluster, run'' <code>sacctmgr list qos</code>
* <code>--time=hh:mm:ss</code> ''time needed to run your job''
* <code>--job-name=helloWorld</code>
* <code>--output filename</code> ''file to redirect stdout to''
* <code>--error filename</code> ''file to redirect stderr''
* <code>--partition $PNAME</code> ''request job run in the $PNAME partition''
* <code>--ntasks 2</code> ''request 2 "tasks" which map to cores on a CPU, if passed to srun the given command will be run concurrently on each core''

===Interactive Shell Sessions===
An interactive shell session on a compute node can be useful for debugging or developing code that isn't ready to be run as a batch job. To get an interactive shell on a node, use <code>srun</code> to invoke a shell:
<pre>
tgray26@opensub01:srun --pty --mem 1gb --time=01:00:00 bash
tgray26@openlab06:
</pre>
'''Please do not leave interactive shells running for long periods of time when you are not working. This blocks resources from being used by everyone else.'''

==salloc==
The salloc command can also be used to request resources be allocated without needing a batch script. Running salloc with a list of resources will allocate the resources you requested, create a job, and drop you into a subshell with the environment variables necessary to run commands in the newly created job allocation. When your time is up or you exit the subshell, your job allocation will be relinquished.
<pre>
tgray26@opensub00:salloc -N 1 --mem=2gb --time=01:00:00
salloc: Granted job allocation 159
tgray26@opensub00:srun /usr/bin/hostname
openlab00.umiacs.umd.edu
tgray26@opensub00:exit
exit
salloc: Relinquishing job allocation 159
</pre>
'''Please note that any commands not invoked with srun will be run locally on the submit node. Please be careful when using salloc.'''

==sbatch==
The sbatch command allows you to write a batch script to be submitted and run non-interactively on the compute nodes. To run a simple Hello World command on the compute nodes you could write a file, helloWorld.sh with the following contents:
<pre>
#!/bin/bash

srun bash -c 'echo Hello World from `hostname`'
</pre>
Then you need to submit the script with sbatch and request resources:
<pre>tgray26@opensub00:sbatch --mem=1gb --time=1:00:00 helloWorld.sh
Submitted batch job 121
</pre>
SLURM will return a job number that you can use to check the status of your job with squeue:
<pre>
tgray26@opensub00:squeue
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
121 dpart helloWor tgray26 R 0:01 2 openlab[00-01]
</pre>
====Advanced Batch Scripts====
You can also write a batch script with all of your resources/options defined in the script itself. This is useful for jobs that need to be run 10s/100s/1000s of times. You can then handle any necessary environment setup and run commands on the resources you requested by invoking commands with srun. The srun commands can also be more complex and be told to only use portions of your entire job allocation, each of these distinct srun commands makes up one "job step". The batch script will be run on the first node allocated as part of your job allocation and each job step will be run on whatever resources you tell them to. In the following example I have a batch job that will request 2 nodes in the cluster, then I load a specific version of Python into my environment and submit two job steps, each one using one node. Since srun is blocks until the command finishes, I use the '&' operator to background the process so that both job steps can run at once; however, this means that I then need to use the wait command to block processing until all background processes have finished.
<pre>
#!/bin/bash

# Lines that begin with #SBATCH specify commands to be used by SLURM for scheduling

#SBATCH --job-name=helloWorld # sets the job name
#SBATCH --output helloWorld.out.%j # indicates a file to redirect STDOUT to; %j is the jobid
#SBATCH --error helloWorld.out.%j # indicates a file to redirect STDERR to; %j is the jobid
#SBATCH --time=00:05:00 # how long you think your job will take to complete; format=hh:mm:ss
#SBATCH --qos=dpart # set QOS, this will determine what resources can be requested
#SBATCH --nodes=2 # number of nodes to allocate for your job
#SBATCH --ntasks=4 # request 4 cpu cores be reserved for your node total
#SBATCH --ntasks-per-node=2 # request 2 cpu cores be reserved per node
#SBATCH --mem 1gb # memory required by job; if unit is not specified MB will be assumed

module load Python/2.7.9 # run any commands necessary to setup your environment

srun -N 1 --mem=512mb bash -c "hostname; python --version" & # use srun to invoke commands within your job; using an '&'
srun -N 1 --mem=512mb bash -c "hostname; python --version" & # will background the process allowing them to run concurrently
wait # wait for any background processes to complete

# once the end of the batch script is reached your job allocation will be revoked
</pre>

Another useful thing to know is that you can pass additional arguments into your sbatch scripts on the command line and reference them as <code>${1}</code> for the first argument and so on.

====More Examples====

* [[SLURM/ArrayJobs]]

===scancel===
The scancel command can be used to cancel job allocations or job steps that are no longer needed. It can be passed individual job IDs or an option to delete all of your jobs or jobs that meet certain criteria.
*<code>scancel 255</code> ''cancel job 255''
*<code>scancel 255.3</code> ''cancel job step 3 of job 255''
*<code>scancel --user tgray26 --partition dpart</code> ''cancel all jobs for tgray26 in the dpart partition''

=Identifying Resources and Features=
The sinfo can show you additional features of nodes in the cluster but you need to ask it to show some non-default options using a command like this
<code>sinfo -o "%15N %10c %10m %25f %10G"</code>.

<pre>
$ sinfo -o "%40N %8c %8m %20f %25G"
NODELIST CPUS MEMORY AVAIL_FEATURES GRES
openlab[30-33] 64 257759 Opteron,6274 (null)
openlab[00-07] 8 7822 Opteron,2354 (null)
openlab[10-11,13-18,20-23,25,27-29] 16 23939 Xeon,x5560 (null)
openlab08 32 128720 Xeon,E5-2690 gpu:k20:2
openlab09 32 128722 Xeon,E5-2690 gpu:m40:1,gpu:k20:2
</pre>

You can also identify further specific information about a node using [https://wiki.umiacs.umd.edu/umiacs/index.php/SLURM/ClusterStatus#scontrol scontrol].

=Requesting GPUs=
If you need to do processing on a GPU, you will need to request that your job have access to GPUs just as you need to request processors or cpu cores. You will also need to make sure that you submit your job to the correct partition since nodes with GPUs are often put into their own partition to prevent the nodes from being tied up by jobs that don't utilize GPUs. In SLURM, GPUs are considered "generic resources" also known as GRES. To request some number of GPUs be reserved/available for your job you can use the flag <code>--gres:gpu:2</code> or if there are multiple types of GPUs available in the cluster and you need a specific type, you can provide the type option to the gres flag <code>--gres:k20:1</code>
<pre>
tgray26@opensub01:srun --pty --partition gpu --qos=gpu --gres=gpu:2 nvidia-smi
Wed Jul 13 15:33:18 2016
+------------------------------------------------------+
| NVIDIA-SMI 361.28 Driver Version: 361.28 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K20c Off | 0000:03:00.0 Off | 0 |
| 30% 24C P0 48W / 225W | 11MiB / 4799MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 Tesla K20c Off | 0000:84:00.0 Off | 0 |
| 30% 23C P0 52W / 225W | 11MiB / 4799MiB | 93% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
</pre>
Please note that your job will only be able to see/access the GPUs you requested. If you only need 1 GPU, please request only 1 GPU and the other one will be left available for other users:
<pre>
tgray26@opensub01:srun --pty --partition gpu --qos=gpu --gres=gpu:k20:1 nvidia-smi
Wed Jul 13 15:31:29 2016
+------------------------------------------------------+
| NVIDIA-SMI 361.28 Driver Version: 361.28 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K20c Off | 0000:03:00.0 Off | 0 |
| 30% 24C P0 50W / 225W | 11MiB / 4799MiB | 92% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
</pre>
The <code>--gres</code> flag may also be passed to [[#sbatch | sbatch]] and [[#salloc | salloc]] rather than directly to [[#srun | srun]]

=MPI example=
<pre>
#!/usr/bin/bash
#SBATCH --job-name=mpi_test # Job name
#SBATCH --nodes=4 # Number of nodes
#SBATCH --ntasks=8 # Number of MPI ranks
#SBATCH --ntasks-per-node=2 # Number of MPI ranks per node
#SBATCH --ntasks-per-socket=1 # Number of tasks per processor socket on the node
#SBATCH --time=00:30:00 # Time limit hrs:min:sec

module load mpi

srun --mpi=openmpi /nfshomes/derek/testing/mpi/a.out
</pre>

SLURM/JobSubmission

2019-07-09T18:32:30Z

Dkontyko: /* Requesting GPUs */ Added the qos flag for the gpus

=Job Submission=

SLURM offers a variety of ways to run jobs. It is important to understand the different options available and how to request the resources required for a job in order for it to run successfully. All job submission should be done from submit nodes; any computational code should be run in a job allocation on compute nodes. The following commands outline how to allocate resources on the compute nodes and submit processes to be run on the allocated nodes.

==srun==
<code>srun</code> is the command used to run a process on the compute nodes in the cluster. It works by passing it a command (this could be a script) which will be run on a compute node and then <code>srun</code> will return. <code>srun</code> accepts many command line options to specify the resources required by the command passed to it. Some common command line arguments are listed below and full documentation of all available options is available in the man page for <code>srun</code>, which can be accessed by running <code>man srun</code>.
<pre>
tgray26@opensub01:srun --mem=100mb --time=1:00:00 bash -c 'echo "Hello World from" `hostname`'
Hello World from openlab06.umiacs.umd.edu
</pre>
It is important to understand that <code>srun</code> is an interactive command. By default input to <code>srun</code> is broadcast to all compute nodes running your process and output from the compute nodes is redirected to <code>srun</code>. This behavior can be changed; however, '''srun will always wait for the command passed to finish before exiting, so if you start a long running process and end your terminal session, your process will stop running on the compute nodes and your job will end'''. To run a non-interactive submission that will remain running after you logout, you will need to wrap your <code>srun</code> commands in a batch script and submit it with [[#sbatch | sbatch]]
===Common srun arguments===
* <code>--mem=1gb</code> ''if no unit is given MB is assumed''
* <code>--nodes=2</code> ''if passed to srun, the given command will be run concurrently on each node''
* <code>--qos=dpart</code>
* <code>--time=hh:mm:ss</code> ''time needed to run your job''
* <code>--job-name=helloWorld</code>
* <code>--output filename</code> ''file to redirect stdout to''
* <code>--error filename</code> ''file to redirect stderr''
* <code>--partition $PNAME</code> ''request job run in the $PNAME partition''
* <code>--ntasks 2</code> ''request 2 "tasks" which map to cores on a CPU, if passed to srun the given command will be run concurrently on each core''

===Interactive Shell Sessions===
An interactive shell session on a compute node can be useful for debugging or developing code that isn't ready to be run as a batch job. To get an interactive shell on a node, use <code>srun</code> to invoke a shell:
<pre>
tgray26@opensub01:srun --pty --mem 1gb --time=01:00:00 bash
tgray26@openlab06:
</pre>
'''Please do not leave interactive shells running for long periods of time when you are not working. This blocks resources from being used by everyone else.'''

==salloc==
The salloc command can also be used to request resources be allocated without needing a batch script. Running salloc with a list of resources will allocate the resources you requested, create a job, and drop you into a subshell with the environment variables necessary to run commands in the newly created job allocation. When your time is up or you exit the subshell, your job allocation will be relinquished.
<pre>
tgray26@opensub00:salloc -N 1 --mem=2gb --time=01:00:00
salloc: Granted job allocation 159
tgray26@opensub00:srun /usr/bin/hostname
openlab00.umiacs.umd.edu
tgray26@opensub00:exit
exit
salloc: Relinquishing job allocation 159
</pre>
'''Please note that any commands not invoked with srun will be run locally on the submit node. Please be careful when using salloc.'''

==sbatch==
The sbatch command allows you to write a batch script to be submitted and run non-interactively on the compute nodes. To run a simple Hello World command on the compute nodes you could write a file, helloWorld.sh with the following contents:
<pre>
#!/bin/bash

srun bash -c 'echo Hello World from `hostname`'
</pre>
Then you need to submit the script with sbatch and request resources:
<pre>tgray26@opensub00:sbatch --mem=1gb --time=1:00:00 helloWorld.sh
Submitted batch job 121
</pre>
SLURM will return a job number that you can use to check the status of your job with squeue:
<pre>
tgray26@opensub00:squeue
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
121 dpart helloWor tgray26 R 0:01 2 openlab[00-01]
</pre>
====Advanced Batch Scripts====
You can also write a batch script with all of your resources/options defined in the script itself. This is useful for jobs that need to be run 10s/100s/1000s of times. You can then handle any necessary environment setup and run commands on the resources you requested by invoking commands with srun. The srun commands can also be more complex and be told to only use portions of your entire job allocation, each of these distinct srun commands makes up one "job step". The batch script will be run on the first node allocated as part of your job allocation and each job step will be run on whatever resources you tell them to. In the following example I have a batch job that will request 2 nodes in the cluster, then I load a specific version of Python into my environment and submit two job steps, each one using one node. Since srun is blocks until the command finishes, I use the '&' operator to background the process so that both job steps can run at once; however, this means that I then need to use the wait command to block processing until all background processes have finished.
<pre>
#!/bin/bash

# Lines that begin with #SBATCH specify commands to be used by SLURM for scheduling

#SBATCH --job-name=helloWorld # sets the job name
#SBATCH --output helloWorld.out.%j # indicates a file to redirect STDOUT to; %j is the jobid
#SBATCH --error helloWorld.out.%j # indicates a file to redirect STDERR to; %j is the jobid
#SBATCH --time=00:05:00 # how long you think your job will take to complete; format=hh:mm:ss
#SBATCH --qos=dpart # set QOS, this will determine what resources can be requested
#SBATCH --nodes=2 # number of nodes to allocate for your job
#SBATCH --ntasks=4 # request 4 cpu cores be reserved for your node total
#SBATCH --ntasks-per-node=2 # request 2 cpu cores be reserved per node
#SBATCH --mem 1gb # memory required by job; if unit is not specified MB will be assumed

module load Python/2.7.9 # run any commands necessary to setup your environment

srun -N 1 --mem=512mb bash -c "hostname; python --version" & # use srun to invoke commands within your job; using an '&'
srun -N 1 --mem=512mb bash -c "hostname; python --version" & # will background the process allowing them to run concurrently
wait # wait for any background processes to complete

# once the end of the batch script is reached your job allocation will be revoked
</pre>

Another useful thing to know is that you can pass additional arguments into your sbatch scripts on the command line and reference them as <code>${1}</code> for the first argument and so on.

====More Examples====

* [[SLURM/ArrayJobs]]

===scancel===
The scancel command can be used to cancel job allocations or job steps that are no longer needed. It can be passed individual job IDs or an option to delete all of your jobs or jobs that meet certain criteria.
*<code>scancel 255</code> ''cancel job 255''
*<code>scancel 255.3</code> ''cancel job step 3 of job 255''
*<code>scancel --user tgray26 --partition dpart</code> ''cancel all jobs for tgray26 in the dpart partition''

=Identifying Resources and Features=
The sinfo can show you additional features of nodes in the cluster but you need to ask it to show some non-default options using a command like this
<code>sinfo -o "%15N %10c %10m %25f %10G"</code>.

<pre>
$ sinfo -o "%40N %8c %8m %20f %25G"
NODELIST CPUS MEMORY AVAIL_FEATURES GRES
openlab[30-33] 64 257759 Opteron,6274 (null)
openlab[00-07] 8 7822 Opteron,2354 (null)
openlab[10-11,13-18,20-23,25,27-29] 16 23939 Xeon,x5560 (null)
openlab08 32 128720 Xeon,E5-2690 gpu:k20:2
openlab09 32 128722 Xeon,E5-2690 gpu:m40:1,gpu:k20:2
</pre>

You can also identify further specific information about a node using [https://wiki.umiacs.umd.edu/umiacs/index.php/SLURM/ClusterStatus#scontrol scontrol].

=Requesting GPUs=
If you need to do processing on a GPU, you will need to request that your job have access to GPUs just as you need to request processors or cpu cores. You will also need to make sure that you submit your job to the correct partition since nodes with GPUs are often put into their own partition to prevent the nodes from being tied up by jobs that don't utilize GPUs. In SLURM, GPUs are considered "generic resources" also known as GRES. To request some number of GPUs be reserved/available for your job you can use the flag <code>--gres:gpu:2</code> or if there are multiple types of GPUs available in the cluster and you need a specific type, you can provide the type option to the gres flag <code>--gres:k20:1</code>
<pre>
tgray26@opensub01:srun --pty --partition gpu --qos=gpu --gres=gpu:2 nvidia-smi
Wed Jul 13 15:33:18 2016
+------------------------------------------------------+
| NVIDIA-SMI 361.28 Driver Version: 361.28 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K20c Off | 0000:03:00.0 Off | 0 |
| 30% 24C P0 48W / 225W | 11MiB / 4799MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 Tesla K20c Off | 0000:84:00.0 Off | 0 |
| 30% 23C P0 52W / 225W | 11MiB / 4799MiB | 93% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
</pre>
Please note that your job will only be able to see/access the GPUs you requested. If you only need 1 GPU, please request only 1 GPU and the other one will be left available for other users:
<pre>
tgray26@opensub01:srun --pty --partition gpu --qos=gpu --gres=gpu:k20:1 nvidia-smi
Wed Jul 13 15:31:29 2016
+------------------------------------------------------+
| NVIDIA-SMI 361.28 Driver Version: 361.28 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K20c Off | 0000:03:00.0 Off | 0 |
| 30% 24C P0 50W / 225W | 11MiB / 4799MiB | 92% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
</pre>
The <code>--gres</code> flag may also be passed to [[#sbatch | sbatch]] and [[#salloc | salloc]] rather than directly to [[#srun | srun]]

=MPI example=
<pre>
#!/usr/bin/bash
#SBATCH --job-name=mpi_test # Job name
#SBATCH --nodes=4 # Number of nodes
#SBATCH --ntasks=8 # Number of MPI ranks
#SBATCH --ntasks-per-node=2 # Number of MPI ranks per node
#SBATCH --ntasks-per-socket=1 # Number of tasks per processor socket on the node
#SBATCH --time=00:30:00 # Time limit hrs:min:sec

module load mpi

srun --mpi=openmpi /nfshomes/derek/testing/mpi/a.out
</pre>

WebSpace

2019-06-25T14:44:34Z

Dkontyko: /* Personal Web Space */

UMIACS provides web space hosting for research/lab pages and user pages.

== '''Hosting websites in UMIACS Object Store ''(preferred method)''''' ==
Please refer to the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store Help Page] for details on hosting a website in the UMIACS Object Store. This is currently our most updated and reliable method for hosting websites.

==Main Website and Lab Pages==

<pre>http://www.umiacs.umd.edu</pre>

Users can access the main website and lab sites for editing in two ways:
* From Unix as /fs/www - and can be remotely accessed by [[SFTP]] to a supported Unix host (eg. [[OpenLAB]])
* From Windows as \\fluidfs.ad.umiacs.umd.edu\www-umiacs - and remotely accessed by the same file share over the [[VPN]]

Faculty members and authorized users can modify their own public profiles on the main UMIACS homepage. For instructions, see [[ContentManagement]].

==Personal Web Space==

Your personal website URL at UMIACS is

<pre>http://www.umiacs.umd.edu/~username</pre>

where '''username''' is your UMIACS username. Users can set this page to redirect to any page of their choice by setting the '''Home Page''' attribute in their UMIACS [https://intranet.umiacs.umd.edu/directory/info/ directory entry].

In general, large datasets related to a lab's research should go into the specific lab's web tree, not the individual users. Remember that a user's webpage is not permanently maintained once the user leaves UMIACS.

UMIACS currently supports two ways of hosting a personal website within our network, the Object Store and the OPENLab file space.

===UMIACS Object Store===

This is the preferred method of hosting a personal website at UMIACS. Please see the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store (OBJ) Help Page] for more information on creating a website within OBJ. Once you create your website in OBJ, you will need to set your directory '''Home Page''' to the bucket's URL (the URL that ends in <code>umiacs.io</code>).

===OPENLab File Space===

This is primarily a legacy method for users who already have their websites configured this way. If you believe that your circumstances require your personal website to be hosted on this file space, please contact the [[HelpDesk | Help Desk]]. (This does not affect existing users who already have websites hosted on the OPENLab file space.)

You will need set your directory '''Home Page''' attribute to <code>http://users.umiacs.umd.edu/~username</code>, where '''username''' is your UMIACS username (similar to your personal URL above). You can access your website for editing in two ways:

* From Unix as /fs/www-users/username - and can be remotely accessed via [[SFTP]] to a supported UNIX host (eg. [[OpenLAB]]).
* From Windows as \\fluidfs.ad.umiacs.umd.edu\www-users\username - and remotely accessed by the same file share over the [[VPN]].

==Adding A Password Protected Folder To Your Web Space==

1) Create the directory you want to password protect or <tt>cd</tt> into the directory you want to password protect

2) Create a file called ''.htaccess'' (<tt> vi .htaccess</tt>) in the directory you wish to password protect.

3) In the file you just created type the following lines

<pre>
AuthUserFile "/your/directory/here/".htpasswd
AuthName "Secure Document"
AuthType Basic
require user username
</pre>

For example, if you were going to protect the <tt>/fs/www-users/username/private</tt> directory and you want the required name to be <tt>class239</tt>, then your file would look like this:
<pre>
AuthUserFile /fs/www-users/username/private/.htpasswd
AuthName "Secure Document"
AuthType Basic
require user class239
</pre>

4) Create a file called ''.htpasswd'' in the same directory as ''.htaccess''. You create this file by typing in <tt>htpasswd -c .htpasswd ''username''</tt> in the directory area to be protected.

In the example above, the username is <tt>class239</tt> so you would type <tt>htpasswd -c .htpasswd class239</tt>

You will be prompted to enter the password you want. The ''.htpasswd'' file will be created in the current directory and will contain an encrypted version of the password.

To later change the username, edit the ''.htaccess'' file and change the username. If you want to later change the password, just retype the above line in step 4 and enter the new password at the prompt.

==Restricting Content based on IP address==
It is possible to have pages on your webspace only accessible to clients connecting from certain IP addresses. In order to accomplish this, cd in to the directory you wish to restrict, and edit your ''.htaccess'' or ''httpd.conf'' file. The example below shows how to make content only viewable to clients connecting from the UMD wifi in Apache 2.2.

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">SetEnvIF X-Forwarded-For "^128\.8\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^129\.2\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^192\.168\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^206\.196\.(?:1[6-9][0-9]|2[0-5][0-9])\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^10\.\d+\.\d+\.\d+$" UMD_NETWORK
Order Deny,Allow
Deny from all
Allow from env=UMD_NETWORK
</pre>

The SetEnvIF directive will modify one's environment if the specified attribute matches the provided regular expression. In this example, IP addresses that are forwarded from an IP within UMD's IP space are tagged with UMD_NETWORK. Then, all traffic to the example directory is blocked unless it has the UMD_NETWORK tag. See the following pages for a more in depth explanation of the commands used.

[https://httpd.apache.org/docs/2.2/howto/htaccess.html .htaccess], [https://httpd.apache.org/docs/2.2/mod/mod_setenvif.html#setenvif SetEnvIf], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#order Order], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#deny Deny], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#allow Allow]

WebSpace

2019-06-25T14:42:08Z

Dkontyko: /* Personal Web Space */ substantial rewrite to this section, clarifying our preferred method of hosting a personal website and more clearly describing the process for setting one's home directory

UMIACS provides web space hosting for research/lab pages and user pages.

== '''Hosting websites in UMIACS Object Store ''(preferred method)''''' ==
Please refer to the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store Help Page] for details on hosting a website in the UMIACS Object Store. This is currently our most updated and reliable method for hosting websites.

==Main Website and Lab Pages==

<pre>http://www.umiacs.umd.edu</pre>

Users can access the main website and lab sites for editing in two ways:
* From Unix as /fs/www - and can be remotely accessed by [[SFTP]] to a supported Unix host (eg. [[OpenLAB]])
* From Windows as \\fluidfs.ad.umiacs.umd.edu\www-umiacs - and remotely accessed by the same file share over the [[VPN]]

Faculty members and authorized users can modify their own public profiles on the main UMIACS homepage. For instructions, see [[ContentManagement]].

==Personal Web Space==

Your personal website URL at UMIACS is

<pre>http://www.umiacs.umd.edu/~username</pre>

where '''username''' is your UMIACS username. Users can set this page to redirect to any page of their choice by setting the '''Home Page''' attribute in their UMIACS [https://intranet.umiacs.umd.edu/directory/info/ directory entry].

In general, large datasets related to a lab's research should go into the specific lab's web tree, not the individual users. Remember that users' webpage is not permanently maintained once the user leaves UMIACS.

UMIACS currently supports two ways of hosting a personal website within our network, the Object Store and the OPENLab file space.

===UMIACS Object Store===

This is the preferred method of hosting a personal website at UMIACS. Please see the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store (OBJ) Help Page] for more information on creating a website within OBJ. Once you create your website in OBJ, you will need to set your directory '''Home Page''' to the bucket's URL (the URL that ends in <code>umiacs.io</code>).

===OPENLab File Space===

This is primarily a legacy method for users who already have their websites configured this way. If you believe that your circumstances require your personal website to be hosted on this file space, please contact the [[HelpDesk | Help Desk]]. (This does not affect existing users who already have websites hosted on the OPENLab file space.)

You will need set your directory '''Home Page''' attribute to <code>http://users.umiacs.umd.edu/~username</code>, where '''username''' is your UMIACS username (similar to your personal URL above). You can access your website for editing in two ways:

* From Unix as /fs/www-users/username - and can be remotely accessed via [[SFTP]] to a supported UNIX host (eg. [[OpenLAB]]).
* From Windows as \\fluidfs.ad.umiacs.umd.edu\www-users\username - and remotely accessed by the same file share over the [[VPN]].

==Adding A Password Protected Folder To Your Web Space==

1) Create the directory you want to password protect or <tt>cd</tt> into the directory you want to password protect

2) Create a file called ''.htaccess'' (<tt> vi .htaccess</tt>) in the directory you wish to password protect.

3) In the file you just created type the following lines

<pre>
AuthUserFile "/your/directory/here/".htpasswd
AuthName "Secure Document"
AuthType Basic
require user username
</pre>

For example, if you were going to protect the <tt>/fs/www-users/username/private</tt> directory and you want the required name to be <tt>class239</tt>, then your file would look like this:
<pre>
AuthUserFile /fs/www-users/username/private/.htpasswd
AuthName "Secure Document"
AuthType Basic
require user class239
</pre>

4) Create a file called ''.htpasswd'' in the same directory as ''.htaccess''. You create this file by typing in <tt>htpasswd -c .htpasswd ''username''</tt> in the directory area to be protected.

In the example above, the username is <tt>class239</tt> so you would type <tt>htpasswd -c .htpasswd class239</tt>

You will be prompted to enter the password you want. The ''.htpasswd'' file will be created in the current directory and will contain an encrypted version of the password.

To later change the username, edit the ''.htaccess'' file and change the username. If you want to later change the password, just retype the above line in step 4 and enter the new password at the prompt.

==Restricting Content based on IP address==
It is possible to have pages on your webspace only accessible to clients connecting from certain IP addresses. In order to accomplish this, cd in to the directory you wish to restrict, and edit your ''.htaccess'' or ''httpd.conf'' file. The example below shows how to make content only viewable to clients connecting from the UMD wifi in Apache 2.2.

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">SetEnvIF X-Forwarded-For "^128\.8\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^129\.2\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^192\.168\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^206\.196\.(?:1[6-9][0-9]|2[0-5][0-9])\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^10\.\d+\.\d+\.\d+$" UMD_NETWORK
Order Deny,Allow
Deny from all
Allow from env=UMD_NETWORK
</pre>

The SetEnvIF directive will modify one's environment if the specified attribute matches the provided regular expression. In this example, IP addresses that are forwarded from an IP within UMD's IP space are tagged with UMD_NETWORK. Then, all traffic to the example directory is blocked unless it has the UMD_NETWORK tag. See the following pages for a more in depth explanation of the commands used.

[https://httpd.apache.org/docs/2.2/howto/htaccess.html .htaccess], [https://httpd.apache.org/docs/2.2/mod/mod_setenvif.html#setenvif SetEnvIf], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#order Order], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#deny Deny], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#allow Allow]

WebSpace

2019-06-25T14:33:07Z

Dkontyko: /* Personal Web Space */

UMIACS provides web space hosting for research/lab pages and user pages.

== '''Hosting websites in UMIACS Object Store ''(preferred method)''''' ==
Please refer to the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store Help Page] for details on hosting a website in the UMIACS Object Store. This is currently our most updated and reliable method for hosting websites.

==Main Website and Lab Pages==

<pre>http://www.umiacs.umd.edu</pre>

Users can access the main website and lab sites for editing in two ways:
* From Unix as /fs/www - and can be remotely accessed by [[SFTP]] to a supported Unix host (eg. [[OpenLAB]])
* From Windows as \\fluidfs.ad.umiacs.umd.edu\www-umiacs - and remotely accessed by the same file share over the [[VPN]]

Faculty members and authorized users can modify their own public profiles on the main UMIACS homepage. For instructions, see [[ContentManagement]].

==Personal Web Space==

Your personal website URL at UMIACS is

<pre>http://www.umiacs.umd.edu/~username</pre>

where '''username''' is your UMIACS username.

Users can set this page to redirect to any page of their choice by setting the '''Home Page''' attribute in their UMIACS [https://intranet.umiacs.umd.edu/directory/info/ directory entry].

UMIACS currently supports multiple ways of hosting a personal website.

===UMIACS Object Store===

This is the preferred method of hosting a personal website at UMIACS. Please see the [https://obj.umiacs.umd.edu/obj/help UMIACS Object Store (OBJ) Help Page] for more information on creating a website within OBJ. Once you create your website in OBJ, you will need to set your directory '''Home Page''' to the bucket's URL (the URL that ends in <code>umiacs.io</code>).

===OPENLab File Space===

This is primarily a legacy method for users who already have their websites configured this way. If you believe that your circumstances require your personal website to be hosted on this file space, please contact the [[HelpDesk | Help Desk]].

Users can access their website for editing two ways:

* From Unix as /fs/www-users/username - and can be remotely accessed via [[SFTP]] to a supported UNIX host (eg. [[OpenLAB]])
* From Windows as \\fluidfs.ad.umiacs.umd.edu\www-users\username - and remotely accessed by the same file share over the [[VPN]]

In general, large datasets related to a Labs research should go into the specific lab's web tree, not the individual users. Remember that users' webpage is not permanently maintained once the user leaves UMIACS.

==Adding A Password Protected Folder To Your Web Space==

1) Create the directory you want to password protect or <tt>cd</tt> into the directory you want to password protect

2) Create a file called ''.htaccess'' (<tt> vi .htaccess</tt>) in the directory you wish to password protect.

3) In the file you just created type the following lines

<pre>
AuthUserFile "/your/directory/here/".htpasswd
AuthName "Secure Document"
AuthType Basic
require user username
</pre>

For example, if you were going to protect the <tt>/fs/www-users/username/private</tt> directory and you want the required name to be <tt>class239</tt>, then your file would look like this:
<pre>
AuthUserFile /fs/www-users/username/private/.htpasswd
AuthName "Secure Document"
AuthType Basic
require user class239
</pre>

4) Create a file called ''.htpasswd'' in the same directory as ''.htaccess''. You create this file by typing in <tt>htpasswd -c .htpasswd ''username''</tt> in the directory area to be protected.

In the example above, the username is <tt>class239</tt> so you would type <tt>htpasswd -c .htpasswd class239</tt>

You will be prompted to enter the password you want. The ''.htpasswd'' file will be created in the current directory and will contain an encrypted version of the password.

To later change the username, edit the ''.htaccess'' file and change the username. If you want to later change the password, just retype the above line in step 4 and enter the new password at the prompt.

==Restricting Content based on IP address==
It is possible to have pages on your webspace only accessible to clients connecting from certain IP addresses. In order to accomplish this, cd in to the directory you wish to restrict, and edit your ''.htaccess'' or ''httpd.conf'' file. The example below shows how to make content only viewable to clients connecting from the UMD wifi in Apache 2.2.

<pre style="white-space: pre-wrap;
white-space: -moz-pre-wrap;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;">SetEnvIF X-Forwarded-For "^128\.8\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^129\.2\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^192\.168\.\d+\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^206\.196\.(?:1[6-9][0-9]|2[0-5][0-9])\.\d+$" UMD_NETWORK
SetEnvIF X-Forwarded-For "^10\.\d+\.\d+\.\d+$" UMD_NETWORK
Order Deny,Allow
Deny from all
Allow from env=UMD_NETWORK
</pre>

The SetEnvIF directive will modify one's environment if the specified attribute matches the provided regular expression. In this example, IP addresses that are forwarded from an IP within UMD's IP space are tagged with UMD_NETWORK. Then, all traffic to the example directory is blocked unless it has the UMD_NETWORK tag. See the following pages for a more in depth explanation of the commands used.

[https://httpd.apache.org/docs/2.2/howto/htaccess.html .htaccess], [https://httpd.apache.org/docs/2.2/mod/mod_setenvif.html#setenvif SetEnvIf], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#order Order], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#deny Deny], [https://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#allow Allow]

NASUsers

2019-06-25T14:12:53Z

Dkontyko: /* Web Pages */ Redirecting this section to the WebSpace page to remove incorrect and redundant information

===Web Pages===

Please see [[WebSpace#Personal%20Web%20Space | Personal Web Space]].

===Personal FTP Sites for Distributing Data===

Your ftp site is online at

ftp://ftp.umiacs.umd.edu/pub/username

On any supported UNIX workstation, you can access your ftp site as

/fs/ftp-umiacs/pub/username

Windows users can map it as a network drive from

\\fluidfs.ad.umiacs.umd.edu\ftp-umiacs\pub

You can also upload files using [[FTP]], [[SFTP]], and [[SCP]] through openlab.umiacs.umd.edu.

Please note that anyone with an internet connection can log in and download these files so please to do not use your ftp site to store confidential data.

This file system has regular backups with our [[TSM]] service and has [[Snapshots]] for easy user restores.

===Usage Guidelines===

Personal NAS is configured to be highly available and modest in both size and usage. Please store large or heavily accessed data sets in a dedicated project storage directory that is tuned for your application.

Please avoid storing shared project data in personal storage allocations. Separating project data from personal data will simplify administration and data management for both researchers and staff.

CUPS

2019-06-25T00:47:18Z

Dkontyko:

UMIACS has replaced our LPRng printing system on our UNIX systems in favor of the industry standard [http://www.cups.org CUPS] ('''Common Unix Print System'''). This provides us with better support for printers and their specific options.

Printing is only available from UMIACS Networks or when attached to the UMIACS VPN.

A list of printers can be found using,
* http://print.umiacs.umd.edu/printers (available only on UMIACS Networks)
* '''lpstat -p'''

You can see documentation about printing from the command line on our RHEL/Ubuntu hosts at
* http://www.cups.org/documentation.php/doc-1.4/options.html

==Changes from the old LPRng print system==
===Duplexing===
To duplex you need to specify <code>-o sides=two-sided-long-edge</code> instead of <code>-Zduplex</code> when submitting your job. <code>-o sides=two-sided-short-edge</code> can be used for the other duplex orientation. You can specify defaults or create a personal instance of a print queue please see this [http://www.cups.org/documentation.php/doc-1.4/options.html#WITHOPTIONS documentation].

===Commands===
Using the '''lpr''' command is available, although the '''lp''' command is the main supported command and has more functionality. We encourage you to use the '''lp''' (SysV) commands.
===Discrete Queues===
Since now the [[CUPS]] and [[WindowsPrinting]] systems are now discrete. There can be jobs in the queue from the other print system that if they misbehave can cause the other print system queue to stall. Please contact staff for assistance if you run into a problem.
===Banner Pages===
The queues are all no banner queues by default. If you would like to change this behavior for your account you can for each PRINTER_QUEUE by running

<code>lpoptions -o job-sheets=standard PRINTER_QUEUE</code>

CUPS

2019-06-25T00:47:09Z

Dkontyko:

RHEL

2019-06-25T00:32:28Z

Dkontyko: Redirect until this page is properly built out

#REDIRECT [[RHEL7]]

ConferenceRooms

2019-06-21T21:29:57Z

Dkontyko: Redirected page to Iribe/ConferenceRooms

#REDIRECT [[Iribe/ConferenceRooms]]

Ubuntu

2019-06-12T17:33:56Z

Dkontyko:

To provide a more up to date desktop linux experience UMIACS provides support for Ubuntu LTS (long-term support) releases. We will only be supporting LTS releases as there is a large amount of work that goes into testing the auto installer and supporting the more cutting edge Ubuntu releases.

Previous Ubuntu LTS (Long Term Support) releases are supported for 3 years on the desktop and 5 years on the server. Starting with Ubuntu 12.04 LTS, LTS releases will be supported for 5 years on both the desktop and the server.
* Ubuntu 14.04 LTS (Trusty)- End of life date: April 2019

===Features===
* Users have the ability to install software from blessed repositories without staff intervention.
* More bleeding edge desktop software experience than our RHEL offerings.

===Software===
Besides being able to install software from the blessed Ubuntu universes we provide only our binary software common tree to Ubuntu LTS releases. This is analogous to the traditional software found in <tt>/opt</tt> but is now found in <tt>/opt/common</tt>. The software includes Matlab, Mathematica, Compilers, etc. Starting with Ubuntu 14.04 LTS, you can use [[Modules]] to easily load these paths and relevant libraries into your environment. Also, please see [[Ubuntu/SoftwareCenter | Ubuntu Software Center]].

===Data Storage===
Please see the UNIX section in our [[LocalDataStorage]] article.