SLURM/ArrayJobs - Revision history

Mbaney at 20:30, 28 March 2024

2024-03-28T20:30:24Z

← Older revision		Revision as of 20:30, 28 March 2024
Line 15:		Line 15:
	</pre>		</pre>

	==~~Sbatch submission~~ script==		==Submission script==
	Save this to a file called <code>array.sh</code> and you should be able to submit the job as <code>sbatch array.sh</code>.		Save this to a file called <code>array.sh</code> and you should be able to submit the job as <code>sbatch array.sh</code>.
	<pre>		<pre>
Line 25:		Line 25:

	#SBATCH --job-name=example		#SBATCH --job-name=example
			#SBATCH --array=1-16 # run 16 jobs at the same time
			#SBATCH --time=0-00:05:00 # run for 5 minutes (d-hh:mm:ss)
			#SBATCH --mem-per-cpu=500MB # use 500MB per core

	# ~~16 jobs will run in this array at the same time~~		# all bash commands must be after all SBATCH directives
	~~#SBATCH --array=1-16~~

	~~# run for five minutes~~
	~~# d-hh:mm:ss~~
	~~#SBATCH --time=0-00:05:00~~

	~~# default qos should do it~~
	~~#SBATCH --qos=dpart~~

	~~# 500MB memory per core~~
	~~# this is a hard limit~~
	~~#SBATCH --mem-per-cpu=500MB~~

	~~# you may not place~~ bash commands ~~before the last~~ SBATCH ~~directive~~

	# define and create a unique scratch directory		# define and create a unique scratch directory

Mbaney at 19:37, 15 December 2020

2020-12-15T19:37:42Z

← Older revision		Revision as of 19:37, 15 December 2020
Line 33:		Line 33:
	#SBATCH --time=0-00:05:00		#SBATCH --time=0-00:05:00

	# default ~~partition~~ should do it		# default qos should do it
	#SBATCH --~~partition default~~		#SBATCH --qos=dpart

	# 500MB memory per core		# 500MB memory per core

Derek at 17:00, 30 June 2017

2017-06-30T17:00:29Z

← Older revision		Revision as of 17:00, 30 June 2017
Line 1:		Line 1:
			Here is an example to get you started using array jobs in [[SLURM]].

	==Array computation example job==		==Array computation example job==
	Save this code to a file called <code>test.py</code>.		Save this code to a file called <code>test.py</code>.

Derek at 16:59, 30 June 2017

2017-06-30T16:59:20Z

← Older revision		Revision as of 16:59, 30 June 2017
Line 1:		Line 1:
	~~<b>~~Array computation example job~~</b>~~		==Array computation example job==
	Save this code to a file called <code>test.py</code>.		Save this code to a file called <code>test.py</code>.

Line 13:		Line 13:
	</pre>		</pre>

	~~<b>~~Sbatch submission script~~</b>~~		==Sbatch submission script==
	Save this to a file called <code>array.sh</code> and you should be able to submit the job as <code>sbatch array.sh</code>.		Save this to a file called <code>array.sh</code> and you should be able to submit the job as <code>sbatch array.sh</code>.
	<pre>		<pre>

Derek at 16:59, 30 June 2017

2017-06-30T16:59:03Z

← Older revision		Revision as of 16:59, 30 June 2017
Line 1:		Line 1:
	<b>Array computation example job</b>		<b>Array computation example job</b>
			Save this code to a file called <code>test.py</code>.

	<pre>		<pre>
	import time		import time
Line 12:		Line 14:

	<b>Sbatch submission script</b>		<b>Sbatch submission script</b>
			Save this to a file called <code>array.sh</code> and you should be able to submit the job as <code>sbatch array.sh</code>.
	<pre>		<pre>
	#!/bin/bash		#!/bin/bash

Derek at 19:10, 29 June 2017

2017-06-29T19:10:51Z

← Older revision		Revision as of 19:10, 29 June 2017
Line 1:		Line 1:
			<b>Array computation example job</b>
			<pre>
			import time

			print('start at ' + time.strftime('%H:%M:%S'))

			print('sleep for 10 seconds ...')
			time.sleep(10)

			print('stop at ' + time.strftime('%H:%M:%S'))
			</pre>

			<b>Sbatch submission script</b>
	<pre>		<pre>
	#!/bin/bash		#!/bin/bash

Derek at 19:07, 29 June 2017

2017-06-29T19:07:52Z

← Older revision		Revision as of 19:07, 29 June 2017
Line 15:		Line 15:
	#SBATCH --time=0-00:05:00		#SBATCH --time=0-00:05:00

	# ~~short~~ partition should do it		# default partition should do it
	#SBATCH --partition ~~short~~		#SBATCH --partition default

	# 500MB memory per core		# 500MB memory per core

Derek at 19:06, 29 June 2017

2017-06-29T19:06:59Z

← Older revision		Revision as of 19:06, 29 June 2017
Line 25:		Line 25:

	# define and create a unique scratch directory		# define and create a unique scratch directory
	SCRATCH_DIRECTORY=/~~global/work~~/${USER}/job-array-example/${SLURM_JOBID}		SCRATCH_DIRECTORY=/scratch0/${USER}/job-array-example/${SLURM_JOBID}
	mkdir -p ${SCRATCH_DIRECTORY}		mkdir -p ${SCRATCH_DIRECTORY}
	cd ${SCRATCH_DIRECTORY}		cd ${SCRATCH_DIRECTORY}

Derek: Created page with "

 #!/bin/bash  ##################### # job-array example # #####################  #SBATCH --job-name=example  # 16 jobs will run in this array at the same time #SBATCH --a..."

2017-06-29T19:06:33Z

Created page with "<pre> #!/bin/bash ##################### # job-array example # ##################### #SBATCH --job-name=example # 16 jobs will run in this array at the same time #SBATCH --a..."

New page

<pre>
#!/bin/bash

#####################
# job-array example #
#####################

#SBATCH --job-name=example

# 16 jobs will run in this array at the same time
#SBATCH --array=1-16

# run for five minutes
# d-hh:mm:ss
#SBATCH --time=0-00:05:00

# short partition should do it
#SBATCH --partition short

# 500MB memory per core
# this is a hard limit
#SBATCH --mem-per-cpu=500MB

# you may not place bash commands before the last SBATCH directive

# define and create a unique scratch directory
SCRATCH_DIRECTORY=/global/work/${USER}/job-array-example/${SLURM_JOBID}
mkdir -p ${SCRATCH_DIRECTORY}
cd ${SCRATCH_DIRECTORY}

cp ${SLURM_SUBMIT_DIR}/test.py ${SCRATCH_DIRECTORY}

# each job will see a different ${SLURM_ARRAY_TASK_ID}
echo "now processing task id:: " ${SLURM_ARRAY_TASK_ID}
python test.py > output_${SLURM_ARRAY_TASK_ID}.txt

# after the job is done we copy our output back to $SLURM_SUBMIT_DIR
cp output_${SLURM_ARRAY_TASK_ID}.txt ${SLURM_SUBMIT_DIR}

# we step out of the scratch directory and remove it
cd ${SLURM_SUBMIT_DIR}
rm -rf ${SCRATCH_DIRECTORY}

# happy end
exit 0
</pre>