|
|
| Line 1: |
Line 1: |
| [https://sylabs.io/docs/ Singularity] is a container platform that doesn't elevate the privileges of a user running the container. This is important as UMIACS runs many multi-tenant hosts and doesn't provide administrative control to users on them.
| | #REDIRECT [[Apptainer]] |
| | |
| You can find out what the current version is that we provide by running the '''singularity --version''' command. If this instead says <code>singularity: command not found</code> please contact staff and we will ensure that the software is available on the host you are looking for it on.
| |
| | |
| <pre>
| |
| $ singularity --version
| |
| singularity version 3.4.2-1.1.el7
| |
| </pre>
| |
| | |
| Singularity can run a variety of images including its own format and [https://sylabs.io/guides/3.0/user-guide/singularity_and_docker.html Docker images]. To create images, you need to have administrative rights. Therefore, you will need to do this on a host that you have administrative access to (laptop or personal desktop) rather than a UMIACS-supported host.
| |
| | |
| If you are going to pull large images, you may run out of space in your home directory. We suggest you run the following commands to setup a alternate cache directory.
| |
| <pre>
| |
| export WORKDIR=/scratch0/username
| |
| export SINGULARITY_CACHEDIR=${WORKDIR}/.cache
| |
| mkdir -p $SINGULARITY_CACHEDIR
| |
| </pre>
| |
| | |
| We do suggest you pull images down into an intermediate file ('''SIF''' file) as you then do not have to worry about re-caching the image.
| |
| <pre>
| |
| $ singularity pull cuda10.2.sif docker://nvidia/cuda:10.2-devel
| |
| INFO: Converting OCI blobs to SIF format
| |
| INFO: Starting build...
| |
| Getting image source signatures
| |
| Copying blob sha256:7ddbc47eeb70dc7f08e410a6667948b87ff3883024eb41478b44ef9a81bf400c
| |
| 25.45 MiB / 25.45 MiB [====================================================] 0s
| |
| Copying blob sha256:c1bbdc448b7263673926b8fe2e88491e5083a8b4b06ddfabf311f2fc5f27e2ff
| |
| 34.53 KiB / 34.53 KiB [====================================================] 0s
| |
| Copying blob sha256:8c3b70e3904492c753652606df4726430426f42ea56e06ea924d6fea7ae162a1
| |
| 845 B / 845 B [============================================================] 0s
| |
| Copying blob sha256:45d437916d5781043432f2d72608049dcf74ddbd27daa01a25fa63c8f1b9adc4
| |
| 162 B / 162 B [============================================================] 0s
| |
| Copying blob sha256:d8f1569ddae616589c5a2dabf668fadd250ee9d89253ef16f0cb0c8a9459b322
| |
| 6.88 MiB / 6.88 MiB [======================================================] 0s
| |
| Copying blob sha256:902fc5ce82298797e09dc1b28ed942da3892a354321725fad6955ee651557e98
| |
| 9.00 MiB / 9.00 MiB [======================================================] 0s
| |
| Copying blob sha256:ae1bb79c5cfc1c10948684c3a2964ec7444b19cae7edc487aa474804336b52c1
| |
| 184 B / 184 B [============================================================] 0s
| |
| Copying blob sha256:fa6605c8fe7a3779c56b4768ac46738f160f005067f22c03e3cf5e87ad5d19d0
| |
| 674.37 MiB / 674.37 MiB [==================================================] 8s
| |
| Copying blob sha256:0508a679d3397b66fdf6cd56db9a99e2b7af5a8519e9f9534d2dff07779f2482
| |
| 783.58 MiB / 783.58 MiB [=================================================] 10s
| |
| Copying config sha256:44b5f1390c8715653c6488b3cb4fbee875c30cacadc827b2b4372b18318b2fa0
| |
| 7.28 KiB / 7.28 KiB [======================================================] 0s
| |
| Writing manifest to image destination
| |
| Storing signatures
| |
| 2020/04/14 11:52:55 info unpack layer: sha256:7ddbc47eeb70dc7f08e410a6667948b87ff3883024eb41478b44ef9a81bf400c
| |
| 2020/04/14 11:52:56 info unpack layer: sha256:c1bbdc448b7263673926b8fe2e88491e5083a8b4b06ddfabf311f2fc5f27e2ff
| |
| 2020/04/14 11:52:56 info unpack layer: sha256:8c3b70e3904492c753652606df4726430426f42ea56e06ea924d6fea7ae162a1
| |
| 2020/04/14 11:52:56 info unpack layer: sha256:45d437916d5781043432f2d72608049dcf74ddbd27daa01a25fa63c8f1b9adc4
| |
| 2020/04/14 11:52:56 info unpack layer: sha256:d8f1569ddae616589c5a2dabf668fadd250ee9d89253ef16f0cb0c8a9459b322
| |
| 2020/04/14 11:52:57 info unpack layer: sha256:902fc5ce82298797e09dc1b28ed942da3892a354321725fad6955ee651557e98
| |
| 2020/04/14 11:52:57 info unpack layer: sha256:ae1bb79c5cfc1c10948684c3a2964ec7444b19cae7edc487aa474804336b52c1
| |
| 2020/04/14 11:52:57 info unpack layer: sha256:fa6605c8fe7a3779c56b4768ac46738f160f005067f22c03e3cf5e87ad5d19d0
| |
| 2020/04/14 11:53:14 info unpack layer: sha256:0508a679d3397b66fdf6cd56db9a99e2b7af5a8519e9f9534d2dff07779f2482
| |
| INFO: Creating SIF file...
| |
| INFO: Build complete: cuda10.2.sif
| |
| </pre>
| |
| | |
| Now you can run the local image with the '''run''' command or start a shell with the '''shell''' command. Please note that if you are in an environment with GPUs and you want to access them inside the container you need to specify the '''--nv''' flag.
| |
| | |
| <pre>
| |
| $ singularity run --nv cuda10.2.sif nvidia-smi -L
| |
| GPU 0: GeForce GTX 1080 Ti (UUID: GPU-9ee980c3-8746-08dd-8e14-82fbaf88367e)
| |
| </pre>
| |
| | |
| ==Example==
| |
| We have a [https://gitlab.umiacs.umd.edu/derek/gpudocker gpudocker] example workflow using our [[GitLab]] as a Docker registry. You can clone the repository and further customize this to your needs. The workflow is:
| |
| # Run Docker on a laptop or personal desktop on to create the image.
| |
| # Tag the image and and push it to the repository.
| |
| # Pull the image down onto one of our workstations/clusters and run it with your data. | |
| | |
| <pre>
| |
| $ singularity pull gpudocker.sif docker://registry.umiacs.umd.edu/derek/gpudocker
| |
| INFO: Converting OCI blobs to SIF format
| |
| INFO: Starting build...
| |
| Getting image source signatures
| |
| Copying blob sha256:7ddbc47eeb70dc7f08e410a6667948b87ff3883024eb41478b44ef9a81bf400c
| |
| 25.45 MiB / 25.45 MiB [====================================================] 2s
| |
| Copying blob sha256:c1bbdc448b7263673926b8fe2e88491e5083a8b4b06ddfabf311f2fc5f27e2ff
| |
| 34.53 KiB / 34.53 KiB [====================================================] 0s
| |
| Copying blob sha256:8c3b70e3904492c753652606df4726430426f42ea56e06ea924d6fea7ae162a1
| |
| 845 B / 845 B [============================================================] 0s
| |
| Copying blob sha256:45d437916d5781043432f2d72608049dcf74ddbd27daa01a25fa63c8f1b9adc4
| |
| 162 B / 162 B [============================================================] 0s
| |
| Copying blob sha256:d8f1569ddae616589c5a2dabf668fadd250ee9d89253ef16f0cb0c8a9459b322
| |
| 6.88 MiB / 6.88 MiB [======================================================] 0s
| |
| Copying blob sha256:85386706b02069c58ffaea9de66c360f9d59890e56f58485d05c1a532ca30db1
| |
| 8.05 MiB / 8.05 MiB [======================================================] 1s
| |
| Copying blob sha256:ee9b457b77d047ff322858e2de025e266ff5908aec569560e77e2e4451fc23f4
| |
| 184 B / 184 B [============================================================] 0s
| |
| Copying blob sha256:be4f3343ecd31ebf7ec8809f61b1d36c2c2f98fc4e63582401d9108575bc443a
| |
| 656.83 MiB / 656.83 MiB [=================================================] 28s
| |
| Copying blob sha256:30b4effda4fdab95ec4eba8873f86e7574c2edddf4dc5df8212e3eda1545aafa
| |
| 782.81 MiB / 782.81 MiB [=================================================] 37s
| |
| Copying blob sha256:b6f46848806c8750a68edc4463bf146ed6c3c4af18f5d3f23281dcdfb1c65055
| |
| 100.58 MiB / 100.58 MiB [==================================================] 5s
| |
| Copying blob sha256:44845dc671f759820baac0376198141ca683f554bb16a177a3cfe262c9e368ff
| |
| 1.47 GiB / 1.47 GiB [====================================================] 1m4s
| |
| Copying config sha256:1b2e5b7b99af9d797ef6fbd091a6a2c6a30e519e31a74f5e9cacb4c8c462d6ed
| |
| 7.56 KiB / 7.56 KiB [======================================================] 0s
| |
| Writing manifest to image destination
| |
| Storing signatures
| |
| 2020/04/14 12:21:17 info unpack layer: sha256:7ddbc47eeb70dc7f08e410a6667948b87ff3883024eb41478b44ef9a81bf400c
| |
| 2020/04/14 12:21:18 info unpack layer: sha256:c1bbdc448b7263673926b8fe2e88491e5083a8b4b06ddfabf311f2fc5f27e2ff
| |
| 2020/04/14 12:21:18 info unpack layer: sha256:8c3b70e3904492c753652606df4726430426f42ea56e06ea924d6fea7ae162a1
| |
| 2020/04/14 12:21:18 info unpack layer: sha256:45d437916d5781043432f2d72608049dcf74ddbd27daa01a25fa63c8f1b9adc4
| |
| 2020/04/14 12:21:18 info unpack layer: sha256:d8f1569ddae616589c5a2dabf668fadd250ee9d89253ef16f0cb0c8a9459b322
| |
| 2020/04/14 12:21:18 info unpack layer: sha256:85386706b02069c58ffaea9de66c360f9d59890e56f58485d05c1a532ca30db1
| |
| 2020/04/14 12:21:18 info unpack layer: sha256:ee9b457b77d047ff322858e2de025e266ff5908aec569560e77e2e4451fc23f4
| |
| 2020/04/14 12:21:18 info unpack layer: sha256:be4f3343ecd31ebf7ec8809f61b1d36c2c2f98fc4e63582401d9108575bc443a
| |
| 2020/04/14 12:21:35 info unpack layer: sha256:30b4effda4fdab95ec4eba8873f86e7574c2edddf4dc5df8212e3eda1545aafa
| |
| 2020/04/14 12:21:55 info unpack layer: sha256:b6f46848806c8750a68edc4463bf146ed6c3c4af18f5d3f23281dcdfb1c65055
| |
| 2020/04/14 12:21:58 info unpack layer: sha256:44845dc671f759820baac0376198141ca683f554bb16a177a3cfe262c9e368ff
| |
| INFO: Creating SIF file...
| |
| INFO: Build complete: gpudocker.sif
| |
| </pre>
| |
| | |
| <pre>
| |
| $ singularity run --nv gpudocker.sif python3 -c 'from __future__ import print_function; import torch; print(torch.cuda.current_device()); x = torch.rand(5, 3); print(x)'
| |
| 0
| |
| tensor([[0.5299, 0.9827, 0.7858],
| |
| [0.2044, 0.6783, 0.2606],
| |
| [0.0538, 0.4272, 0.9361],
| |
| [0.1980, 0.2654, 0.4160],
| |
| [0.1680, 0.8407, 0.0509]])
| |
| </pre>
| |