UMobj/Example: Difference between revisions

From UMIACS
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]).
In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]).
<pre>
<pre>
export OBJ_ACCESS_KEY_ID=<ACCESS_KEY>
export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>"
export OBJ_SECRET_ACCESS_KEY=<SECRET_KEY>
export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>"
export OBJ_SERVER="obj.umiacs.umd.edu"
export OBJ_SERVER="obj.umiacs.umd.edu"
</pre>
</pre>

Revision as of 15:58, 10 May 2024

In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or LabGroup in the Object Store).

export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>"
export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>"
export OBJ_SERVER="obj.umiacs.umd.edu"

We will be uploading this 4.4GB file.

# stat CASIA-WebFace.tar.gz
  File: `CASIA-WebFace.tar.gz'
  Size: 4018119666      Blocks: 7847896    IO Block: 262144 regular file
Device: 13h/19d Inode: 87161       Links: 1
Access: (0644/-rw-r--r--)  Uid: (12372/  carlos)   Gid: (18149/   janus)
Access: 2015-11-24 19:34:05.715695000 -0500
Modify: 2015-11-23 14:23:35.872087000 -0500
Change: 2015-11-23 16:07:10.062306000 -0500

If the bucket does not exist yet you need to create the bucket first.

# mkobj janus_datasets:
Created bucket janus_datasets.

Now you can copy the file to the bucket.

# cpobj CASIA-WebFace.tar.gz janus_datasets:
100% |#########################################################################################################################################|

To verify the object store has the data expected we can compare the checksums. First you can calculate the md5sum locally and then use cmpobj to stream the data from the object store and create a md5sum. They should match.

# md5sum CASIA-WebFace.tar.gz
004c2475e5ed66771e8873be67a93105  CASIA-WebFace.tar.gz
# cmpobj janus_datasets:CASIA-WebFace.tar.gz
004c2475e5ed66771e8873be67a93105