Data Storage: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| Line 14: | Line 14: | ||
==How can I transfer my data?== | ==How can I transfer my data?== | ||
* '''Local data storage''' is typically best transferred using commands or programs available in the operating system on which it is stored or most commonly accessed from. A number of these commands are covered [[LocalDataTransfer | here]]. | * '''[[LocalDataStorage | Local data storage]]''' is typically best transferred using commands or programs available in the operating system on which it is stored or most commonly accessed from. A number of these commands are covered [[LocalDataTransfer | here]]. | ||
* '''[[OBJ | UMIACS' Object Store]]''' has a [https://obj.umiacs.umd.edu built-in web interface] for transferring data in and out of it, which is usually the best option for individual or relatively small amounts of files/folders. For bulk transfers, one of many [[S3Clients | compatible clients]] can be used. | * '''[[OBJ | UMIACS' Object Store]]''' has a [https://obj.umiacs.umd.edu built-in web interface] for transferring data in and out of it, which is usually the best option for individual or relatively small amounts of files/folders. For bulk transfers, one of many [[S3Clients | compatible clients]] can be used. | ||
* '''Cloud data storage''' transfer methods vary based on the service provider. Most major providers will provide simple upload/download functionality, which works well for individual or relatively small amounts of files/folders. For bulk transfers, [[Rclone]] is one program that [[HelpDesk | UMIACS staff]] often uses, as it has compatibility with many major cloud storage service providers. | * '''[[CloudDataStorage | Cloud data storage]]''' transfer methods vary based on the service provider. Most major providers will provide simple upload/download functionality, which works well for individual or relatively small amounts of files/folders. For bulk transfers, [[Rclone]] is one program that [[HelpDesk | UMIACS staff]] often uses, as it has compatibility with many major cloud storage service providers. | ||
==How is my data retained?== | ==How is my data retained?== | ||
Revision as of 16:51, 25 October 2024
This is a landing page for all topics related to data storage that are available at UMIACS. It is under active development.
Where can I store my data?
Before choosing where to store your data, consider how you may need to interact with that data in the short-term and in the long-term, and who you might need to share the data with. You can copy data between the different types of data storage listed below, but it may be unnecessarily cumbersome if you don't choose the right place for the "master copy" of your data.
- Local data storage is best suited for data that you are actively working on. Some examples are code that you are developing or using (perhaps to run computational jobs in SLURM with), results from computational jobs that have already run but have not yet been published in a paper, or files that need to be processed by another desktop-based or server-based application.
- Data on UMIACS' local storage can be moved between different UMIACS-supported hosts and shared with other UMIACS account holders using common methods such as cp, File Explorer, Finder, and more.
- UMIACS' Object Store is best suited for data that is going to remain static, but needs to remain accessible for very long periods of time, such as data referenced by published papers. It is also suitable for transferring specific versions of data in and out of UMIACS via large singular files, such as archive (tar/zip/etc.) files.
- Data in UMIACS' Object Store can be moved in and out of it, to local storage (whether UMIACS' or otherwise), via its built-in web interface or one of many compatible clients. Data can be shared publicly via simple download links or static websites that can visualize the data in a more accessible way. UMIACS account holders can sponsor Collaborator accounts for external collaborators that may need to upload and download new versions of data without needing direct access to UMIACS-supported hosts.
- Cloud data storage is best suited for collaborative data, such as simple or rich text documents, PDF forms, spreadsheets, presentations, pictures, videos, and more. Many cloud storage service providers also provide web-based apps attached to their storage service that can be used to edit these types of data without ever having to download it to a specific device.
- The methods that can be used to move data in and out of cloud storage to local storage vary based on the specific service provider, but often involve web-based or mobile app-based upload and download. Data can be shared with others across the web via accounts associated with that service provider, and often publicly via simple download links.
How can I transfer my data?
- Local data storage is typically best transferred using commands or programs available in the operating system on which it is stored or most commonly accessed from. A number of these commands are covered here.
- UMIACS' Object Store has a built-in web interface for transferring data in and out of it, which is usually the best option for individual or relatively small amounts of files/folders. For bulk transfers, one of many compatible clients can be used.
- Cloud data storage transfer methods vary based on the service provider. Most major providers will provide simple upload/download functionality, which works well for individual or relatively small amounts of files/folders. For bulk transfers, Rclone is one program that UMIACS staff often uses, as it has compatibility with many major cloud storage service providers.