SLURM/Priority - Revision history

Mbaney at 16:40, 30 April 2026

2026-04-30T16:40:32Z

← Older revision		Revision as of 16:40, 30 April 2026
Line 11:		Line 11:

	==Pending Jobs==		==Pending Jobs==
	If the partition that you submit your job to cannot start your job instantly due to no compute node(s) in the partition having the resources free to run it, your job will remain in the Pending state with the listed reason <tt>(Resources)</tt>. If there is another job already pending with this reason in a partition, you submit a job to the same partition, and your job gets assigned a lower priority value than that pending job, your job will instead remain in the Pending state with reason <tt>(Priority)</tt>. If there are multiple jobs pending and your job is not the highest priority job pending, the scheduler will only start execution of your job if ~~starting your job~~ would not push the start times for any higher priority jobs in the same partition further back.		If the partition that you submit your job to cannot start your job instantly due to no compute node(s) in the partition having the resources free to run it, your job will remain in the Pending state with the listed reason <tt>(Resources)</tt>. If there is another job already pending with this reason in a partition, you submit a job to the same partition, and your job gets assigned a lower priority value than that pending job, your job will instead remain in the Pending state with reason <tt>(Priority)</tt>. If there are multiple jobs pending and your job is not the highest priority job pending, the scheduler will only start execution of your job if doing so would not push the start times for any higher priority jobs in the same partition further back.

	Lowering some combination of the resources you are requesting and/or the time limit may allow submitted jobs to start sooner (or instantly) during times where a partition is under resource pressure. The command <code>squeue -j <jobid> --start</code> can be used to provide a time estimate for when your job will start, where <jobid> is the job ID you receive from either srun or sbatch. This time is subject to change depending on if other users' jobs end sooner or more jobs get submitted.		Lowering some combination of the resources you are requesting and/or the time limit may allow submitted jobs to start sooner (or instantly) during times where a partition is under resource pressure. The command <code>squeue -j <jobid> --start</code> can be used to provide a time estimate for when your job will start, where <jobid> is the job ID you receive from either srun or sbatch. This time is subject to change depending on if other users' jobs end sooner or more jobs get submitted.

Mbaney: /* Pending Jobs */

2026-04-29T20:53:06Z

Pending Jobs

← Older revision		Revision as of 20:53, 29 April 2026
Line 11:		Line 11:

	==Pending Jobs==		==Pending Jobs==
	If the partition that you submit your job to cannot ~~begin~~ your job instantly due to no compute node(s) in the partition having the resources free to run it, your job will remain in the Pending state with the listed reason <tt>(Resources)</tt>. If there is another job already pending with this reason in a partition, you submit a job to the same partition, and your job gets assigned a lower priority value than that pending job, your job will instead remain in the Pending state with reason <tt>(Priority)</tt>. If there are multiple jobs pending and your job is not the highest priority job pending, the scheduler will only ~~begin~~ execution of your job if starting your job would not push the ~~begin~~ times for any higher priority jobs in the same partition further back.		If the partition that you submit your job to cannot start your job instantly due to no compute node(s) in the partition having the resources free to run it, your job will remain in the Pending state with the listed reason <tt>(Resources)</tt>. If there is another job already pending with this reason in a partition, you submit a job to the same partition, and your job gets assigned a lower priority value than that pending job, your job will instead remain in the Pending state with reason <tt>(Priority)</tt>. If there are multiple jobs pending and your job is not the highest priority job pending, the scheduler will only start execution of your job if starting your job would not push the start times for any higher priority jobs in the same partition further back.

	Lowering some combination of the resources you are requesting and/or the time limit may allow submitted jobs to start sooner (or instantly) during times where a partition is under resource pressure. The command <code>squeue -j <jobid> --start</code> can be used to provide a time estimate for when your job will start, where <jobid> is the job ID you receive from either srun or sbatch. This time is subject to change depending on if other users' jobs end sooner or more jobs get submitted.		Lowering some combination of the resources you are requesting and/or the time limit may allow submitted jobs to start sooner (or instantly) during times where a partition is under resource pressure. The command <code>squeue -j <jobid> --start</code> can be used to provide a time estimate for when your job will start, where <jobid> is the job ID you receive from either srun or sbatch. This time is subject to change depending on if other users' jobs end sooner or more jobs get submitted.

	You can use the command alias <code>[[SLURM/JobSubmission#show_available_nodes \| show_available_nodes]]</code> with a variety of different submission arguments to get a better idea of what jobs may be able to ~~begin~~ sooner, but the output of this command alias is not definitive, for reasons mentioned in the footnotes on the page linked to.		You can use the command alias <code>[[SLURM/JobSubmission#show_available_nodes \| show_available_nodes]]</code> with a variety of different submission arguments to get a better idea of what jobs may be able to start sooner, but the output of this command alias is not definitive, for reasons mentioned in the footnotes on the page linked to.

	==Priority Factors==		==Priority Factors==

Mbaney: /* Age */

2026-04-29T20:50:55Z

Age

← Older revision		Revision as of 20:50, 29 April 2026
Line 86:		Line 86:

	===Age===		===Age===
	The longer a job is eligible to run but cannot due to resources being unavailable or having a lower priority value than one or more other jobs, the higher the job's priority becomes as it continues to wait in the queue. This is the only priority modifier that can change a job's priority value once it has been submitted, and the priority modifier for this factor reaches its limit after 7 days.		The longer a job is eligible to run but cannot due to resources being unavailable or it having a lower priority value than one or more other jobs, the higher the job's priority becomes as it continues to wait in the queue. This is the only priority modifier that can change a job's priority value once it has been submitted, and the priority modifier for this factor reaches its limit after 7 days.

	Jobs' age priority factors on our clusters are recalculated every 5 minutes.		Jobs' age priority factors on our clusters are recalculated every 5 minutes.

Mbaney: /* Fair-share */

2026-04-29T20:49:45Z

Fair-share

← Older revision		Revision as of 20:49, 29 April 2026
Line 47:		Line 47:

	===Fair-share===		===Fair-share===
	The more resources your jobs have already consumed within an account, the lower priority factor your future jobs will have when compared to other users' jobs in the same account who have used fewer resources (so as to "fair-share" with other users). Additionally, if there are multiple accounts that can submit to a partition, and the sum of resources of all users' jobs within account A is greater than the sum of resources of all users' jobs within account B, ~~the lower priority factor~~ all future jobs from users in account A will have when compared to all future jobs from users in account B. (In other words, fair-share is hierarchical.)		The more resources your jobs have already consumed within an account, the lower priority factor your future jobs will have when compared to other users' jobs in the same account who have used fewer resources (so as to "fair-share" with other users). Additionally, if there are multiple accounts that can submit to a partition, and the sum of resources used by all users' jobs within account A is greater than the sum of resources used by all users' jobs within account B, all future jobs from users in account A will have a lower priority factor when compared to all future jobs from users in account B. (In other words, fair-share is hierarchical.)

	You can view the various fair-share statistics with the command <code>sshare -l</code>. It will show your specific FairShare values (always between 0.0 and 1.0) within accounts that you have access to. You can also view other accounts' Level Fairshare (LevelFS).		You can view the various fair-share statistics with the command <code>sshare -l</code>. It will show your specific FairShare values (always between 0.0 and 1.0) within accounts that you have access to. You can also view other accounts' Level Fairshare (LevelFS).

Mbaney: /* Age */

2026-04-28T20:51:14Z

Age

← Older revision		Revision as of 20:51, 28 April 2026
Line 86:		Line 86:

	===Age===		===Age===
	The longer a job is eligible to run but cannot due to resources being unavailable or having a lower priority value than one or more other jobs, the higher the job's priority becomes as it ~~continue~~ to wait in the queue. This is the only priority modifier that can change a job's priority value once it has been submitted, and the priority modifier for this factor reaches its limit after 7 days.		The longer a job is eligible to run but cannot due to resources being unavailable or having a lower priority value than one or more other jobs, the higher the job's priority becomes as it continues to wait in the queue. This is the only priority modifier that can change a job's priority value once it has been submitted, and the priority modifier for this factor reaches its limit after 7 days.

	Jobs' age priority factors on our clusters are recalculated every 5 minutes.		Jobs' age priority factors on our clusters are recalculated every 5 minutes.

Mbaney: /* Fair-share */

2026-04-15T14:28:59Z

Fair-share

← Older revision		Revision as of 14:28, 15 April 2026
Line 54:		Line 54:
	-------------------- ---------- ---------- ----------- ----------- ----------- ------------- ---------- ---------- ------------------------------ ------------------------------		-------------------- ---------- ---------- ----------- ----------- ----------- ------------- ---------- ---------- ------------------------------ ------------------------------
	root 0.000000 68444174744 1.000000 cpu=4797787,mem=70530109515,e+		root 0.000000 68444174744 1.000000 cpu=4797787,mem=70530109515,e+
	cbcb-heng 1 0.028571 2041564795 0.029831 0.029831 0.957779 cpu=107606,mem=2890646050,ene+
	cbcb 1 0.028571 4454658377 0.065046 0.065046 0.439246 cpu=452139,mem=22276633804,en+		cbcb 1 0.028571 4454658377 0.065046 0.065046 0.439246 cpu=452139,mem=22276633804,en+
	class 1 0.028571 255617290 0.003733 0.003733 7.652841 cpu=7021,mem=74554606,energy=+		class 1 0.028571 255617290 0.003733 0.003733 7.652841 cpu=7021,mem=74554606,energy=+
	clip 1 0.028571 3057933838 0.044674 0.044674 0.639549 cpu=33214,mem=2744443460,ener+		clip 1 0.028571 3057933838 0.044674 0.044674 0.639549 cpu=33214,mem=2744443460,ener+
	cml-abhinav 1 0.028571 39726437 0.000580 0.000580 49.220844 cpu=0,mem=0,energy=0,node=0,b+
	cml-director 1 0.028571 672692955 0.009829 0.009829 2.906778 cpu=65977,mem=1080976998,ener+
	cml-furongh 1 0.028571 765366431 0.011183 0.011183 2.554814 cpu=78802,mem=1513011200,ener+
	cml-hajiagha 1 0.028571 12397205 0.000181 0.000181 157.726572 cpu=0,mem=0,energy=0,node=0,b+
	cml-ramani 1 0.028571 0 0.000000 0.000000 1.3575e+11 cpu=0,mem=0,energy=0,node=0,b+
	cml-scavenger 1 0.028571 2564733604 0.037475 0.037475 0.762406 cpu=370867,mem=3568986794,ene+
	cml-sfeizi 1 0.028571 40715178 0.000595 0.000595 48.025548 cpu=0,mem=0,energy=0,node=0,b+
	cml-tokekar 1 0.028571 26458249 0.000387 0.000387 73.903937 cpu=23240,mem=178488320,energ+
	cml-tomg 1 0.028571 82800 0.000001 0.000001 2.3615e+04 cpu=0,mem=0,energy=0,node=0,b+
	cml-wriva-high 1 0.028571 0 0.000000 0.000000 inf cpu=0,mem=0,energy=0,node=0,b+
	cml-wriva 1 0.028571 1047428702 0.015305 0.015305 1.866828 cpu=5366,mem=137386666,energy+
	cml 1 0.028571 66866114 0.000975 0.000975 29.299389 cpu=1796,mem=29426756,energy=+		cml 1 0.028571 66866114 0.000975 0.000975 29.299389 cpu=1796,mem=29426756,energy=+
	csd-sarahwie 1 0.028571 710055035 0.007034 0.007034 4.061956 cpu=0,mem=0,energy=0,node=0,b+
	gamma 1 0.028571 2609474948 0.038129 0.038129 0.749334 cpu=34089,mem=360373862,energ+		gamma 1 0.028571 2609474948 0.038129 0.038129 0.749334 cpu=34089,mem=360373862,energ+
	mbrc 1 0.028571 73411964 0.001073 0.001073 26.635560 cpu=1195,mem=4896358,energy=0+		mbrc 1 0.028571 73411964 0.001073 0.001073 26.635560 cpu=1195,mem=4896358,energy=0+
Line 80:		Line 67:
	scavenger 1 0.028571 40888195964 0.597419 0.597419 0.047825 cpu=3142204,mem=29902903931,e+		scavenger 1 0.028571 40888195964 0.597419 0.597419 0.047825 cpu=3142204,mem=29902903931,e+
	scavenger username 1 0.000835 171 0.000000 0.000000 0.033975 9.8885e+04 cpu=0,mem=0,energy=0,node=0,b+		scavenger username 1 0.000835 171 0.000000 0.000000 0.033975 9.8885e+04 cpu=0,mem=0,energy=0,node=0,b+
	vulcan-abhinav 1 0.028571 1742123078 0.025452 0.025452 1.122544 cpu=26166,mem=781842841,energ+
	vulcan-djacobs 1 0.028571 912637 0.000013 0.000013 2.1425e+03 cpu=0,mem=0,energy=0,node=0,b+
	vulcan-jbhuang 1 0.028571 283430750 0.004141 0.004141 6.898930 cpu=172,mem=5661764,energy=0,+
	vulcan-metzler 1 0.028571 262070179 0.003829 0.003829 7.461241 cpu=0,mem=0,energy=0,node=0,b+
	vulcan-rama 1 0.028571 0 0.000000 0.000000 1.6942e+14 cpu=0,mem=0,energy=0,node=0,b+
	vulcan-ramani 1 0.028571 681333557 0.009955 0.009955 2.869914 cpu=22188,mem=568033280,energ+
	vulcan-yaser 1 0.028571 397309 0.000006 0.000006 4.9215e+03 cpu=0,mem=0,energy=0,node=0,b+
	vulcan-zwicker 1 0.028571 133204932 0.001946 0.001946 14.679402 cpu=0,mem=0,energy=0,node=0,b+
	vulcan 1 0.028571 1247236491 0.018224 0.018224 1.567761 cpu=147273,mem=1161243818,ene+		vulcan 1 0.028571 1247236491 0.018224 0.018224 1.567761 cpu=147273,mem=1161243818,ene+
	</pre>		</pre>

Mbaney at 13:10, 15 April 2026

2026-04-15T13:10:43Z

← Older revision		Revision as of 13:10, 15 April 2026
Line 13:		Line 13:
	If the partition that you submit your job to cannot begin your job instantly due to no compute node(s) in the partition having the resources free to run it, your job will remain in the Pending state with the listed reason <tt>(Resources)</tt>. If there is another job already pending with this reason in a partition, you submit a job to the same partition, and your job gets assigned a lower priority value than that pending job, your job will instead remain in the Pending state with reason <tt>(Priority)</tt>. If there are multiple jobs pending and your job is not the highest priority job pending, the scheduler will only begin execution of your job if starting your job would not push the begin times for any higher priority jobs in the same partition further back.		If the partition that you submit your job to cannot begin your job instantly due to no compute node(s) in the partition having the resources free to run it, your job will remain in the Pending state with the listed reason <tt>(Resources)</tt>. If there is another job already pending with this reason in a partition, you submit a job to the same partition, and your job gets assigned a lower priority value than that pending job, your job will instead remain in the Pending state with reason <tt>(Priority)</tt>. If there are multiple jobs pending and your job is not the highest priority job pending, the scheduler will only begin execution of your job if starting your job would not push the begin times for any higher priority jobs in the same partition further back.

	Lowering some combination of the resources you are requesting and/or the time limit may allow submitted jobs to ~~run more quickly~~ or instantly during times where a partition is under resource pressure. The command <code>squeue -j <jobid> --start</code> can be used to provide a time estimate for when your job will start, where <jobid> is the job ID you receive from either srun or sbatch. This time is subject to change depending on if other users' jobs end sooner or more jobs get submitted.		Lowering some combination of the resources you are requesting and/or the time limit may allow submitted jobs to start sooner (or instantly) during times where a partition is under resource pressure. The command <code>squeue -j <jobid> --start</code> can be used to provide a time estimate for when your job will start, where <jobid> is the job ID you receive from either srun or sbatch. This time is subject to change depending on if other users' jobs end sooner or more jobs get submitted.

	You can use the command alias <code>[[SLURM/JobSubmission#show_available_nodes \| show_available_nodes]]</code> with a variety of different submission arguments to get a better idea of what jobs may be able to begin sooner, but the output of this command alias is not definitive, for reasons mentioned in the footnotes on the page linked to.		You can use the command alias <code>[[SLURM/JobSubmission#show_available_nodes \| show_available_nodes]]</code> with a variety of different submission arguments to get a better idea of what jobs may be able to begin sooner, but the output of this command alias is not definitive, for reasons mentioned in the footnotes on the page linked to.

Mbaney at 20:57, 14 April 2026

2026-04-14T20:57:33Z

← Older revision		Revision as of 20:57, 14 April 2026
Line 112:		Line 112:

	===Association===		===Association===
	Some lab/center-specific SLURM accounts ~~may~~ have priority values directly attached to them. Jobs run under these accounts gain this many extra points of priority.		Some lab/center-specific SLURM accounts have priority values directly attached to them. Jobs run under these accounts gain this many extra points of priority.

	===Nice value===		===Nice value===

Mbaney: /* Pending Jobs */

2026-04-08T13:55:44Z

Pending Jobs

← Older revision		Revision as of 13:55, 8 April 2026
Line 11:		Line 11:

	==Pending Jobs==		==Pending Jobs==
	If the partition that you submit your job to cannot begin your job instantly due to no compute node(s) in the partition having the resources free to run it, your job will remain in the Pending state with the listed reason <tt>(Resources)</tt>. If there is another job already pending with this reason, you submit a job to the same partition, and your job gets assigned a lower priority value than that pending job, your job will instead remain in the Pending state with reason <tt>(Priority)</tt>. If there are multiple jobs pending and your job is not the highest priority job pending, the scheduler will only begin execution of your job if starting your job would not push the begin times for any higher priority jobs in the same partition further back.		If the partition that you submit your job to cannot begin your job instantly due to no compute node(s) in the partition having the resources free to run it, your job will remain in the Pending state with the listed reason <tt>(Resources)</tt>. If there is another job already pending with this reason in a partition, you submit a job to the same partition, and your job gets assigned a lower priority value than that pending job, your job will instead remain in the Pending state with reason <tt>(Priority)</tt>. If there are multiple jobs pending and your job is not the highest priority job pending, the scheduler will only begin execution of your job if starting your job would not push the begin times for any higher priority jobs in the same partition further back.

	Lowering some combination of the resources you are requesting and/or the time limit may allow submitted jobs to run more quickly or instantly during times where a partition is under resource pressure. The command <code>squeue -j <jobid> --start</code> can be used to provide a time estimate for when your job will start, where <jobid> is the job ID you receive from either srun or sbatch. This time is subject to change depending on if other users' jobs end sooner or more jobs get submitted.		Lowering some combination of the resources you are requesting and/or the time limit may allow submitted jobs to run more quickly or instantly during times where a partition is under resource pressure. The command <code>squeue -j <jobid> --start</code> can be used to provide a time estimate for when your job will start, where <jobid> is the job ID you receive from either srun or sbatch. This time is subject to change depending on if other users' jobs end sooner or more jobs get submitted.

Mbaney at 13:54, 8 April 2026

2026-04-08T13:54:57Z

← Older revision		Revision as of 13:54, 8 April 2026
Line 123:		Line 123:
	assuming all three jobs were submitted at the same time. You cannot use negative values for this argument.		assuming all three jobs were submitted at the same time. You cannot use negative values for this argument.

	Because this value is absolute, if you want to use it, we would recommend using small numbers ~~only~~ - one or two digits. Larger numbers may impact your job's ability to run at all as a result of the other factors at play.		Because this value is absolute, if you want to use it, we would recommend using small numbers - one or two digits - only. Larger numbers may impact your job's ability to run at all as a result of the other factors at play.