UMobj: Difference between revisions

From UMIACS
Jump to navigation Jump to search
No edit summary
No edit summary
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The UMobj suite of utilities provide command-line access to UMIACS Object Stores.  UMobj is preinstalled on all UMIACS-supported linux machines.
The UMobj suite of utilities provide command-line access to the [https://obj.umiacs.umd.edu/obj UMIACS Object Store].  UMobj is preinstalled on all UMIACS-supported RHEL8 machines. For other operating systems or non UMIACS-supported hosts, we encourage use of one of the many [[S3Clients#Command_Line_Clients | third-party command line clients]] that exist.


==When to use UMobj==
==When to use UMobj==
Line 5: Line 5:
* you have a large number of files to upload (hundreds or thousands of files)
* you have a large number of files to upload (hundreds or thousands of files)
* you are uploading large files (files greater than 4GB in size)
* you are uploading large files (files greater than 4GB in size)
==Supported object storage clusters==
* https://obj.umiacs.umd.edu/obj


==Setup==
==Setup==
Line 18: Line 15:


<pre>
<pre>
export OBJ_ACCESS_KEY_ID="31sdfadDFAHFDN+344qOEIS"
export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>"
export OBJ_SECRET_ACCESS_KEY="NDSMK3233adfahadflkkPDSH092DSJKDKDJKFDLSFLNK"
export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>"
export OBJ_SERVER="obj.umiacs.umd.edu"
export OBJ_SERVER="obj.umiacs.umd.edu"
</pre>
</pre>


Or in tcsh you can do the following or add it into your <tt>.tcshrc</tt>.
Or, in tcsh, you can do the following or add it into your <tt>.tcshrc</tt>.


<pre>
<pre>
setenv OBJ_ACCESS_KEY_ID "31sdfadDFAHFDN+344qOEIS"
setenv OBJ_ACCESS_KEY_ID "<ACCESS_KEY>"
setenv OBJ_SECRET_ACCESS_KEY "NDSMK3233adfahadflkkPDSH092DSJKDKDJKFDLSFLNK"
setenv OBJ_SECRET_ACCESS_KEY "<SECRET_KEY>"
setenv OBJ_SERVER "obj.umiacs.umd.edu"
setenv OBJ_SERVER "obj.umiacs.umd.edu"
</pre>
</pre>
(substituting in your actual <ACCESS_KEY> and <SECRET_KEY> for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]).


==Detailed Usage==
==Detailed Usage==
 
For an example of how to use UMobj, please see [[UMobj/Example]].
For an example of how to use UMobj, please see [[ UMobj/Example ]].


For complete usage information, please see the [[GitLab]] page for [https://gitlab.umiacs.umd.edu/staff/umobj/blob/master/README.md#umobj umobj].
For complete usage information, please see the [[GitLab]] page for [https://gitlab.umiacs.umd.edu/staff/umobj/blob/master/README.md#umobj umobj].

Latest revision as of 18:49, 8 July 2024

The UMobj suite of utilities provide command-line access to the UMIACS Object Store. UMobj is preinstalled on all UMIACS-supported RHEL8 machines. For other operating systems or non UMIACS-supported hosts, we encourage use of one of the many third-party command line clients that exist.

When to use UMobj

Use umobj if:

  • you have a large number of files to upload (hundreds or thousands of files)
  • you are uploading large files (files greater than 4GB in size)

Setup

We highly recommend setting a few environmental variables containing your credentials for convenience. When logged into the Object Store web interface (see list above), you can find these credentials on the user page. E.g. https://obj.umiacs.umd.edu/obj/user/

For example, if you use the bash shell, you can add something like the following to your .bashrc or .bash_profile.

export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>"
export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>"
export OBJ_SERVER="obj.umiacs.umd.edu"

Or, in tcsh, you can do the following or add it into your .tcshrc.

setenv OBJ_ACCESS_KEY_ID "<ACCESS_KEY>"
setenv OBJ_SECRET_ACCESS_KEY "<SECRET_KEY>"
setenv OBJ_SERVER "obj.umiacs.umd.edu"

(substituting in your actual <ACCESS_KEY> and <SECRET_KEY> for your personal account or LabGroup in the Object Store).

Detailed Usage

For an example of how to use UMobj, please see UMobj/Example.

For complete usage information, please see the GitLab page for umobj.