Nexus/Vulcan - Revision history

Mbaney at 19:57, 27 April 2026

2026-04-27T19:57:24Z

← Older revision		Revision as of 19:57, 27 April 2026
Line 62:		Line 62:
	* '''vulcan-ampere''' - This partition contains nodes with GPUs from NVIDIA's [https://www.nvidia.com/en-us/data-center/ampere-architecture/ Ampere architecture] or a newer architecture. Job allocations are guaranteed. Please be aware of the following restrictions on this partition:		* '''vulcan-ampere''' - This partition contains nodes with GPUs from NVIDIA's [https://www.nvidia.com/en-us/data-center/ampere-architecture/ Ampere architecture] or a newer architecture. Job allocations are guaranteed. Please be aware of the following restrictions on this partition:
	*: ''Time limit'': there is a 4 hour time limit on interactive jobs in this partition. If you need to run longer jobs, you will need to modify your workflow into a job that can be submitted as a batch script.		*: ''Time limit'': there is a 4 hour time limit on interactive jobs in this partition. If you need to run longer jobs, you will need to modify your workflow into a job that can be submitted as a batch script.
	*: ''CPU/memory per GPU limit'': there is a limit of 4 CPUs and 48G memory maximum per GPU requested by a job. If you need to run jobs with more CPUs/memory, you will either need to request more GPUs in the job or use a different partition.		*: ''CPU/memory per GPU limit'': there is a limit of 4 CPUs and 48G memory maximum per non-H200 GPU requested by a job, and 16 CPUs and 256G memory maximum per H200 GPU requested by a job. If you need to run jobs with more CPUs/memory, you will either need to request more GPUs in the job or use a different partition.

	: Submission is restricted to the Slurm [[#Accounts \| accounts]] of the faculty who invested in these nodes:		: Submission is restricted to the Slurm [[#Accounts \| accounts]] of the faculty who invested in these nodes:

Mbaney: /* Accounts */

2026-04-27T13:42:27Z

Accounts

← Older revision		Revision as of 13:42, 27 April 2026
Line 90:		Line 90:
	* vulcan-rama		* vulcan-rama
	* vulcan-ramani		* vulcan-ramani
			* vulcan-ruoshi
	* vulcan-yaser		* vulcan-yaser
	* vulcan-zwicker		* vulcan-zwicker
Line 105:		Line 106:
	vulcan-rama vulcan - rama chellappa vulcan		vulcan-rama vulcan - rama chellappa vulcan
	vulcan-ramani vulcan - ramani duraiswami vulcan		vulcan-ramani vulcan - ramani duraiswami vulcan
			vulcan-ruoshi vulcan - ruoshi liu vulcan
	vulcan-yaser vulcan - yaser yacoob vulcan		vulcan-yaser vulcan - yaser yacoob vulcan
	vulcan-zwicker vulcan - matthias zwicker vulcan		vulcan-zwicker vulcan - matthias zwicker vulcan

Mbaney: /* Partitions */

2026-04-27T13:09:24Z

Partitions

← Older revision		Revision as of 13:09, 27 April 2026
Line 60:		Line 60:
	There are a few additional partitions available to subsets of Vulcan users based on specific requirements.		There are a few additional partitions available to subsets of Vulcan users based on specific requirements.

	* '''vulcan-ampere''' - This partition contains nodes with GPUs from NVIDIA's [https://www.nvidia.com/en-us/data-center/ampere-architecture/ Ampere architecture]. Job allocations are guaranteed. Please be aware of the following restrictions on this partition:		* '''vulcan-ampere''' - This partition contains nodes with GPUs from NVIDIA's [https://www.nvidia.com/en-us/data-center/ampere-architecture/ Ampere architecture] or a newer architecture. Job allocations are guaranteed. Please be aware of the following restrictions on this partition:
	*: ''Time limit'': there is a 4 hour time limit on interactive jobs in this partition. If you need to run longer jobs, you will need to modify your workflow into a job that can be submitted as a batch script.		*: ''Time limit'': there is a 4 hour time limit on interactive jobs in this partition. If you need to run longer jobs, you will need to modify your workflow into a job that can be submitted as a batch script.
	*: ''CPU/memory per GPU limit'': there is a limit of 4 CPUs and 48G memory maximum per GPU requested by a job. If you need to run jobs with more CPUs/memory, you will either need to request more GPUs in the job or use a different partition.		*: ''CPU/memory per GPU limit'': there is a limit of 4 CPUs and 48G memory maximum per GPU requested by a job. If you need to run jobs with more CPUs/memory, you will either need to request more GPUs in the job or use a different partition.
Line 68:		Line 68:
	:* Jia-Bin Huang (vulcan-jbhuang)		:* Jia-Bin Huang (vulcan-jbhuang)
	:* Christopher Metzler (vulcan-metzler)		:* Christopher Metzler (vulcan-metzler)
			:* Ruoshi Liu (vulcan-ruoshi)
	:* Matthias Zwicker (vulcan-zwicker)		:* Matthias Zwicker (vulcan-zwicker)

Mbaney: /* QoS */

2025-11-13T17:26:43Z

QoS

← Older revision		Revision as of 17:26, 13 November 2025
Line 134:		Line 134:

	==QoS==		==QoS==
	~~CML~~ currently has 3 QoS for the '''vulcan-dpart''' partition, 1 QoS for the '''vulcan-scavenger''' partition, and 1 QoS for the '''vulcan-cpu''' partition. If you do not specify a QoS when submitting your job using the <code>--qos</code> parameter, you will receive the <code>vulcan-default</code> QoS assuming you are using a Vulcan account.		Vulcan currently has 3 QoS for the '''vulcan-dpart''' partition, 1 QoS for the '''vulcan-scavenger''' partition, and 1 QoS for the '''vulcan-cpu''' partition. If you do not specify a QoS when submitting your job using the <code>--qos</code> parameter, you will receive the <code>vulcan-default</code> QoS assuming you are using a Vulcan account.

	The important part here is that in different QoS you can have a shorter/longer maximum wall time, a different total number of jobs running at once, and a different maximum number of track-able resources (TRES) for the job. In the cml-scavenger QoS, one more constraint that you are restricted by is the total number of TRES per user (over multiple jobs).		The important part here is that in different QoS you can have a shorter/longer maximum wall time, a different total number of jobs running at once, and a different maximum number of track-able resources (TRES) for the job. In the cml-scavenger QoS, one more constraint that you are restricted by is the total number of TRES per user (over multiple jobs).

Mbaney: /* QoS */

2025-11-13T15:52:00Z

QoS

← Older revision		Revision as of 15:52, 13 November 2025
Line 134:		Line 134:

	==QoS==		==QoS==
	~~You need to decide~~ the ~~QOS to submit with which will set a certain number of restrictions to your job~~. If you do not specify a QoS when submitting your job using the <code>--qos</code> parameter, you will receive the ~~'''~~vulcan-default~~'''~~ QoS assuming you are using a Vulcan account.		CML currently has 3 QoS for the '''vulcan-dpart''' partition, 1 QoS for the '''vulcan-scavenger''' partition, and 1 QoS for the '''vulcan-cpu''' partition. If you do not specify a QoS when submitting your job using the <code>--qos</code> parameter, you will receive the <code>vulcan-default</code> QoS assuming you are using a Vulcan account.

	The ~~following <code>sacctmgr<~~/~~code> command will list~~ the ~~current QOS~~. ~~Either~~ the ~~<code>vulcan~~-~~default</code>~~, ~~<code>vulcan-medium</code>, or <code>vulcan-high</code> QOS~~ is ~~required for~~ the ~~vulcan-dpart partition. Please note that only faculty accounts~~ (~~see above~~) ~~have access to the <code>vulcan-high</code> QoS~~.		The important part here is that in different QoS you can have a shorter/longer maximum wall time, a different total number of jobs running at once, and a different maximum number of track-able resources (TRES) for the job. In the cml-scavenger QoS, one more constraint that you are restricted by is the total number of TRES per user (over multiple jobs).

	~~The following example will show you the current limits that the QOS have. The output is truncated to show only relevant Vulcan QoS.~~
	<pre>		<pre>
	$ show_qos		$ show_qos --all \| grep vulcan
	Name MaxWall MaxTRES MaxJobsPU MaxTRESPU		Name MaxWall MaxTRES MaxJobsPU MaxTRESPU
	------------------------- ----------- ------------------------------ --------- ------------------------------		------------------------- ----------- ------------------------------ --------- ------------------------------
Line 154:		Line 153:
	vulcan-scavenger-multi 3-00:00:00 cpu=288,gres/gpu=72,mem=1152G		vulcan-scavenger-multi 3-00:00:00 cpu=288,gres/gpu=72,mem=1152G
	...		...
			</pre>

	$ show_partition_qos		<pre>
			$ show_partition_qos --all \| grep vulcan
	Name MaxSubmitPU MaxTRESPU GrpTRES		Name MaxSubmitPU MaxTRESPU GrpTRES
	------------------------- ----------- ------------------------------ --------------------		------------------------- ----------- ------------------------------ --------------------

Mbaney: /* Partitions */

2025-10-24T18:33:39Z

Partitions

← Older revision		Revision as of 18:33, 24 October 2025
Line 70:		Line 70:
	:* Matthias Zwicker (vulcan-zwicker)		:* Matthias Zwicker (vulcan-zwicker)

	* '''vulcan-scavenger-multi''' - This partition allows multi-node jobs (up to 9 total nodes per job) and allows jobs more resources than the vulcan-scavenger partition, but only contains nodes with GTX 1080 Ti and RTX 2080 Ti GPUs in them. As with vulcan-scavenger, it is preemptable when jobs in other non-scavenged-named <code>vulcan-</code> partitions are ready to be scheduled.		* '''vulcan-scavenger-multi''' - This partition allows multi-node jobs (up to 9 total nodes per job) and allows jobs more resources than the vulcan-scavenger partition, but only contains nodes with GTX 1080 Ti, TITAN Xp, and/or RTX 2080 Ti GPUs in them. As with vulcan-scavenger, it is preemptable when jobs in other non-scavenged-named <code>vulcan-</code> partitions are ready to be scheduled.
	*: Access to this partition is on a per-use basis. Please contact Abhinav Shrivastava if you would like to be granted access to this partition.		*: Access to this partition is on a per-use basis. Please contact Abhinav Shrivastava if you would like to be granted access to this partition.

Mbaney: /* Network */

2025-10-09T20:03:51Z

Network

← Older revision		Revision as of 20:03, 9 October 2025
Line 43:		Line 43:
	# One pair of network switches connected to each other via dual 100GbE links for redundancy, serving the following compute nodes:		# One pair of network switches connected to each other via dual 100GbE links for redundancy, serving the following compute nodes:
	#* brigid[16-17],vulcan[29-45]: Two 100GbE links per node, one to each switch in the pair (redundancy).		#* brigid[16-17],vulcan[29-45]: Two 100GbE links per node, one to each switch in the pair (redundancy).
			#* vulcan46: Four 100GbE links, two to each switch in the pair (redundancy and increased bandwidth).
	# One pair of network switches connected to the above pair of switches through several sets of intermediary switches, and to each other via dual 10GbE links for redundancy. The immediate connection to these sets of intermediary switches is via two 40GbE links to a pair of them, one between the first two switches in each pair and one between the second two switches in each pair for redundancy. This pair serves the following compute nodes:		# One pair of network switches connected to the above pair of switches through several sets of intermediary switches, and to each other via dual 10GbE links for redundancy. The immediate connection to these sets of intermediary switches is via two 40GbE links to a pair of them, one between the first two switches in each pair and one between the second two switches in each pair for redundancy. This pair serves the following compute nodes:
	#* brigid[18-19],vulcan[00-28]: Two 10GbE links per node, one to each switch in the pair (redundancy).		#* brigid[18-19],vulcan[00-28]: Two 10GbE links per node, one to each switch in the pair (redundancy).

Mbaney: /* Project Storage */

2025-09-24T15:59:34Z

Project Storage

← Older revision		Revision as of 15:59, 24 September 2025
Line 198:		Line 198:

	===Project Storage===		===Project Storage===
	Users within the Vulcan compute infrastructure can request project based allocations for up to 10TB for up to 180 days by [[HelpDesk \| contacting staff]] with approval from the Vulcan faculty manager (Dr. Shrivastava). These allocations will be available from <code>/fs/vulcan~~-projects</code> and <code>/fs/cfar~~-projects</code> under a name that you provide when you request the allocation. Near the end of the allocation period, staff will contact you and ask if you would like to renew the allocation for up to another 180 days (requires re-approval from Dr. Shrivastava).		Users within the Vulcan compute infrastructure can request project based allocations for up to 10TB for up to 180 days by [[HelpDesk \| contacting staff]] with approval from the Vulcan faculty manager (Dr. Shrivastava). These allocations will be available from <code>/fs/vulcan-projects</code> under a name that you provide when you request the allocation. Near the end of the allocation period, staff will contact you and ask if you would like to renew the allocation for up to another 180 days (requires re-approval from Dr. Shrivastava).
	* If you are no longer in need of the storage allocation, you will need to relocate all desired data within two weeks of the end of the allocation period. Staff will then remove the allocation.		* If you are no longer in need of the storage allocation, you will need to relocate all desired data within two weeks of the end of the allocation period. Staff will then remove the allocation.
	* If you do not respond to staff's request by the end of the allocation period, staff will make the allocation temporarily inaccessible.		* If you do not respond to staff's request by the end of the allocation period, staff will make the allocation temporarily inaccessible.

Mbaney: /* Partitions */

2025-05-15T15:07:07Z

Partitions

← Older revision		Revision as of 15:07, 15 May 2025
Line 71:		Line 71:
	* '''vulcan-scavenger-multi''' - This partition allows multi-node jobs (up to 9 total nodes per job) and allows jobs more resources than the vulcan-scavenger partition, but only contains nodes with GTX 1080 Ti and RTX 2080 Ti GPUs in them. As with vulcan-scavenger, it is preemptable when jobs in other non-scavenged-named <code>vulcan-</code> partitions are ready to be scheduled.		* '''vulcan-scavenger-multi''' - This partition allows multi-node jobs (up to 9 total nodes per job) and allows jobs more resources than the vulcan-scavenger partition, but only contains nodes with GTX 1080 Ti and RTX 2080 Ti GPUs in them. As with vulcan-scavenger, it is preemptable when jobs in other non-scavenged-named <code>vulcan-</code> partitions are ready to be scheduled.
	*: Access to this partition is on a per-use basis. Please contact Abhinav Shrivastava if you would like to be granted access to this partition.		*: Access to this partition is on a per-use basis. Please contact Abhinav Shrivastava if you would like to be granted access to this partition.

			There is one additional partition available solely to Dr. Ramani Duraiswami's sponsored accounts.

			* '''vulcan-ramani''' - This partition is for exclusive priority access to Dr. Duraiswami's purchased GPU nodes. Job allocations are guaranteed.

	==Accounts==		==Accounts==

Mbaney: /* QoS */

2025-05-15T15:06:14Z

QoS

← Older revision		Revision as of 15:06, 15 May 2025
Line 155:		Line 155:
	...		...
	vulcan 500 cpu=1760,mem=15824G		vulcan 500 cpu=1760,mem=15824G
			vulcan-ampere 500
			vulcan-cpu 500
			vulcan-ramani 500
	vulcan-scavenger 500		vulcan-scavenger 500
	vulcan-scavenger-multi 500		vulcan-scavenger-multi 500