Singularity: Difference between revisions

From UMIACS
Jump to navigation Jump to search
No edit summary
(Redirected page to Apptainer)
Tag: New redirect
 
Line 1: Line 1:
[https://sylabs.io/docs/ Singularity] is a container platform that doesn't elevate the privileges of a user running the container.  This is important as UMIACS runs many multi-tenant hosts and doesn't provide administrative control to users on them.
#REDIRECT [[Apptainer]]
 
You can find out what the current version is that we provide by running the '''singularity --version''' command.  If this instead says <code>singularity: command not found</code> please contact staff and we will ensure that the software is available on the host you are looking for it on.
 
<pre>
$ singularity --version
singularity version 3.4.2-1.1.el7
</pre>
 
Singularity can run a variety of images including its own format and [https://sylabs.io/guides/3.0/user-guide/singularity_and_docker.html Docker images].  To create images, you need to have administrative rights. Therefore, you will need to do this on a host that you have administrative access to (laptop or personal desktop) rather than a UMIACS-supported host.
 
If you are going to pull large images, you may run out of space in your home directory. We suggest you run the following commands to setup a alternate cache directory.
<pre>
export WORKDIR=/scratch0/username
export SINGULARITY_CACHEDIR=${WORKDIR}/.cache
mkdir -p $SINGULARITY_CACHEDIR
</pre>
 
We do suggest you pull images down into an intermediate file ('''SIF''' file) as you then do not have to worry about re-caching the image.
<pre>
$ singularity pull cuda10.2.sif docker://nvidia/cuda:10.2-devel
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob sha256:7ddbc47eeb70dc7f08e410a6667948b87ff3883024eb41478b44ef9a81bf400c
25.45 MiB / 25.45 MiB [====================================================] 0s
Copying blob sha256:c1bbdc448b7263673926b8fe2e88491e5083a8b4b06ddfabf311f2fc5f27e2ff
34.53 KiB / 34.53 KiB [====================================================] 0s
Copying blob sha256:8c3b70e3904492c753652606df4726430426f42ea56e06ea924d6fea7ae162a1
845 B / 845 B [============================================================] 0s
Copying blob sha256:45d437916d5781043432f2d72608049dcf74ddbd27daa01a25fa63c8f1b9adc4
162 B / 162 B [============================================================] 0s
Copying blob sha256:d8f1569ddae616589c5a2dabf668fadd250ee9d89253ef16f0cb0c8a9459b322
6.88 MiB / 6.88 MiB [======================================================] 0s
Copying blob sha256:902fc5ce82298797e09dc1b28ed942da3892a354321725fad6955ee651557e98
9.00 MiB / 9.00 MiB [======================================================] 0s
Copying blob sha256:ae1bb79c5cfc1c10948684c3a2964ec7444b19cae7edc487aa474804336b52c1
184 B / 184 B [============================================================] 0s
Copying blob sha256:fa6605c8fe7a3779c56b4768ac46738f160f005067f22c03e3cf5e87ad5d19d0
674.37 MiB / 674.37 MiB [==================================================] 8s
Copying blob sha256:0508a679d3397b66fdf6cd56db9a99e2b7af5a8519e9f9534d2dff07779f2482
783.58 MiB / 783.58 MiB [=================================================] 10s
Copying config sha256:44b5f1390c8715653c6488b3cb4fbee875c30cacadc827b2b4372b18318b2fa0
7.28 KiB / 7.28 KiB [======================================================] 0s
Writing manifest to image destination
Storing signatures
2020/04/14 11:52:55  info unpack layer: sha256:7ddbc47eeb70dc7f08e410a6667948b87ff3883024eb41478b44ef9a81bf400c
2020/04/14 11:52:56  info unpack layer: sha256:c1bbdc448b7263673926b8fe2e88491e5083a8b4b06ddfabf311f2fc5f27e2ff
2020/04/14 11:52:56  info unpack layer: sha256:8c3b70e3904492c753652606df4726430426f42ea56e06ea924d6fea7ae162a1
2020/04/14 11:52:56  info unpack layer: sha256:45d437916d5781043432f2d72608049dcf74ddbd27daa01a25fa63c8f1b9adc4
2020/04/14 11:52:56  info unpack layer: sha256:d8f1569ddae616589c5a2dabf668fadd250ee9d89253ef16f0cb0c8a9459b322
2020/04/14 11:52:57  info unpack layer: sha256:902fc5ce82298797e09dc1b28ed942da3892a354321725fad6955ee651557e98
2020/04/14 11:52:57  info unpack layer: sha256:ae1bb79c5cfc1c10948684c3a2964ec7444b19cae7edc487aa474804336b52c1
2020/04/14 11:52:57  info unpack layer: sha256:fa6605c8fe7a3779c56b4768ac46738f160f005067f22c03e3cf5e87ad5d19d0
2020/04/14 11:53:14  info unpack layer: sha256:0508a679d3397b66fdf6cd56db9a99e2b7af5a8519e9f9534d2dff07779f2482
INFO:    Creating SIF file...
INFO:    Build complete: cuda10.2.sif
</pre>
 
Now you can run the local image with the '''run''' command or start a shell with the '''shell''' command.  Please note that if you are in an environment with GPUs and you want to access them inside the container you need to specify the '''--nv''' flag.
 
<pre>
$ singularity run --nv cuda10.2.sif nvidia-smi -L
GPU 0: GeForce GTX 1080 Ti (UUID: GPU-9ee980c3-8746-08dd-8e14-82fbaf88367e)
</pre>
 
==Example==
We have a [https://gitlab.umiacs.umd.edu/derek/gpudocker gpudocker] example workflow using our [[GitLab]] as a Docker registry.  You can clone the repository and further customize this to your needs. The workflow is:
# Run Docker on a laptop or personal desktop on to create the image.
# Tag the image and and push it to the repository.
# Pull the image down onto one of our workstations/clusters and run it with your data.
 
<pre>
$ singularity pull gpudocker.sif docker://registry.umiacs.umd.edu/derek/gpudocker
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob sha256:7ddbc47eeb70dc7f08e410a6667948b87ff3883024eb41478b44ef9a81bf400c
25.45 MiB / 25.45 MiB [====================================================] 2s
Copying blob sha256:c1bbdc448b7263673926b8fe2e88491e5083a8b4b06ddfabf311f2fc5f27e2ff
34.53 KiB / 34.53 KiB [====================================================] 0s
Copying blob sha256:8c3b70e3904492c753652606df4726430426f42ea56e06ea924d6fea7ae162a1
845 B / 845 B [============================================================] 0s
Copying blob sha256:45d437916d5781043432f2d72608049dcf74ddbd27daa01a25fa63c8f1b9adc4
162 B / 162 B [============================================================] 0s
Copying blob sha256:d8f1569ddae616589c5a2dabf668fadd250ee9d89253ef16f0cb0c8a9459b322
6.88 MiB / 6.88 MiB [======================================================] 0s
Copying blob sha256:85386706b02069c58ffaea9de66c360f9d59890e56f58485d05c1a532ca30db1
8.05 MiB / 8.05 MiB [======================================================] 1s
Copying blob sha256:ee9b457b77d047ff322858e2de025e266ff5908aec569560e77e2e4451fc23f4
184 B / 184 B [============================================================] 0s
Copying blob sha256:be4f3343ecd31ebf7ec8809f61b1d36c2c2f98fc4e63582401d9108575bc443a
656.83 MiB / 656.83 MiB [=================================================] 28s
Copying blob sha256:30b4effda4fdab95ec4eba8873f86e7574c2edddf4dc5df8212e3eda1545aafa
782.81 MiB / 782.81 MiB [=================================================] 37s
Copying blob sha256:b6f46848806c8750a68edc4463bf146ed6c3c4af18f5d3f23281dcdfb1c65055
100.58 MiB / 100.58 MiB [==================================================] 5s
Copying blob sha256:44845dc671f759820baac0376198141ca683f554bb16a177a3cfe262c9e368ff
1.47 GiB / 1.47 GiB [====================================================] 1m4s
Copying config sha256:1b2e5b7b99af9d797ef6fbd091a6a2c6a30e519e31a74f5e9cacb4c8c462d6ed
7.56 KiB / 7.56 KiB [======================================================] 0s
Writing manifest to image destination
Storing signatures
2020/04/14 12:21:17  info unpack layer: sha256:7ddbc47eeb70dc7f08e410a6667948b87ff3883024eb41478b44ef9a81bf400c
2020/04/14 12:21:18  info unpack layer: sha256:c1bbdc448b7263673926b8fe2e88491e5083a8b4b06ddfabf311f2fc5f27e2ff
2020/04/14 12:21:18  info unpack layer: sha256:8c3b70e3904492c753652606df4726430426f42ea56e06ea924d6fea7ae162a1
2020/04/14 12:21:18  info unpack layer: sha256:45d437916d5781043432f2d72608049dcf74ddbd27daa01a25fa63c8f1b9adc4
2020/04/14 12:21:18  info unpack layer: sha256:d8f1569ddae616589c5a2dabf668fadd250ee9d89253ef16f0cb0c8a9459b322
2020/04/14 12:21:18  info unpack layer: sha256:85386706b02069c58ffaea9de66c360f9d59890e56f58485d05c1a532ca30db1
2020/04/14 12:21:18  info unpack layer: sha256:ee9b457b77d047ff322858e2de025e266ff5908aec569560e77e2e4451fc23f4
2020/04/14 12:21:18  info unpack layer: sha256:be4f3343ecd31ebf7ec8809f61b1d36c2c2f98fc4e63582401d9108575bc443a
2020/04/14 12:21:35  info unpack layer: sha256:30b4effda4fdab95ec4eba8873f86e7574c2edddf4dc5df8212e3eda1545aafa
2020/04/14 12:21:55  info unpack layer: sha256:b6f46848806c8750a68edc4463bf146ed6c3c4af18f5d3f23281dcdfb1c65055
2020/04/14 12:21:58  info unpack layer: sha256:44845dc671f759820baac0376198141ca683f554bb16a177a3cfe262c9e368ff
INFO:    Creating SIF file...
INFO:    Build complete: gpudocker.sif
</pre>
 
<pre>
$ singularity run --nv gpudocker.sif python3 -c 'from __future__ import print_function; import torch; print(torch.cuda.current_device()); x = torch.rand(5, 3); print(x)'
0
tensor([[0.5299, 0.9827, 0.7858],
        [0.2044, 0.6783, 0.2606],
        [0.0538, 0.4272, 0.9361],
        [0.1980, 0.2654, 0.4160],
        [0.1680, 0.8407, 0.0509]])
</pre>

Latest revision as of 16:57, 14 October 2022

Redirect to:

Retrieved from "https://wiki.umiacs.umd.edu/umiacs/index.php?title=Singularity&oldid=10691"