UMobj/Example: Difference between revisions

From UMIACS
Jump to navigation Jump to search
(Created page with " In your shell export the credentials if not already done. Make sure if you are using a shared lab/group bucket that you are using the correct keys. <pre> export OBJ_ACCESS_K...")
 
No edit summary
Line 29: Line 29:
# cpobj CASIA-WebFace.tar.gz janus_datasets:
# cpobj CASIA-WebFace.tar.gz janus_datasets:
100% |#########################################################################################################################################|
100% |#########################################################################################################################################|
</pre>
To verify the object store has the data expected we can compare the checksums.  First you can calculate the md5sum locally and then use cmpobj to stream the data from the object store and create a md5sum.  They should match.
<pre>
# md5sum CASIA-WebFace.tar.gz
004c2475e5ed66771e8873be67a93105  CASIA-WebFace.tar.gz
# cmpobj janus_datasets:CASIA-WebFace.tar.gz
004c2475e5ed66771e8873be67a93105
</pre>
</pre>

Revision as of 14:17, 25 November 2015

In your shell export the credentials if not already done. Make sure if you are using a shared lab/group bucket that you are using the correct keys.

export OBJ_ACCESS_KEY_ID="REDACTED"
export OBJ_SECRET_ACCESS_KEY="REDACTED"
export OBJ_SERVER="obj.umiacs.umd.edu"

We will be uploading this 4.4GB file.

# stat CASIA-WebFace.tar.gz
  File: `CASIA-WebFace.tar.gz'
  Size: 4018119666      Blocks: 7847896    IO Block: 262144 regular file
Device: 13h/19d Inode: 87161       Links: 1
Access: (0644/-rw-r--r--)  Uid: (12372/  carlos)   Gid: (18149/   janus)
Access: 2015-11-24 19:34:05.715695000 -0500
Modify: 2015-11-23 14:23:35.872087000 -0500
Change: 2015-11-23 16:07:10.062306000 -0500

If the bucket does not exist yet you need to create the bucket first.

# mkobj janus_datasets:
Created bucket janus_datasets.

Now you can copy the file to the bucket.

# cpobj CASIA-WebFace.tar.gz janus_datasets:
100% |#########################################################################################################################################|

To verify the object store has the data expected we can compare the checksums. First you can calculate the md5sum locally and then use cmpobj to stream the data from the object store and create a md5sum. They should match.

# md5sum CASIA-WebFace.tar.gz
004c2475e5ed66771e8873be67a93105  CASIA-WebFace.tar.gz
# cmpobj janus_datasets:CASIA-WebFace.tar.gz
004c2475e5ed66771e8873be67a93105