UMobj/Example: Difference between revisions
Jump to navigation
Jump to search
(Created page with " In your shell export the credentials if not already done. Make sure if you are using a shared lab/group bucket that you are using the correct keys. <pre> export OBJ_ACCESS_K...") |
No edit summary |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]). | |||
In your shell export the credentials if not already done. | |||
<pre> | <pre> | ||
export OBJ_ACCESS_KEY_ID=" | export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>" | ||
export OBJ_SECRET_ACCESS_KEY=" | export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>" | ||
export OBJ_SERVER="obj.umiacs.umd.edu" | export OBJ_SERVER="obj.umiacs.umd.edu" | ||
</pre> | </pre> | ||
| Line 13: | Line 12: | ||
Size: 4018119666 Blocks: 7847896 IO Block: 262144 regular file | Size: 4018119666 Blocks: 7847896 IO Block: 262144 regular file | ||
Device: 13h/19d Inode: 87161 Links: 1 | Device: 13h/19d Inode: 87161 Links: 1 | ||
Access: (0644/-rw-r--r--) Uid: ( | Access: (0644/-rw-r--r--) Uid: (10001/username) Gid: (18001/groupname) | ||
Access: 2015-11-24 19:34:05.715695000 -0500 | Access: 2015-11-24 19:34:05.715695000 -0500 | ||
Modify: 2015-11-23 14:23:35.872087000 -0500 | Modify: 2015-11-23 14:23:35.872087000 -0500 | ||
| Line 29: | Line 28: | ||
# cpobj CASIA-WebFace.tar.gz janus_datasets: | # cpobj CASIA-WebFace.tar.gz janus_datasets: | ||
100% |#########################################################################################################################################| | 100% |#########################################################################################################################################| | ||
</pre> | |||
To verify the object store has the data expected we can compare the checksums. First you can calculate the md5sum locally and then use cmpobj to stream the data from the object store and create a md5sum. They should match. | |||
<pre> | |||
# md5sum CASIA-WebFace.tar.gz | |||
004c2475e5ed66771e8873be67a93105 CASIA-WebFace.tar.gz | |||
# cmpobj janus_datasets:CASIA-WebFace.tar.gz | |||
004c2475e5ed66771e8873be67a93105 | |||
</pre> | </pre> | ||
Latest revision as of 15:59, 10 May 2024
In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or LabGroup in the Object Store).
export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>" export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>" export OBJ_SERVER="obj.umiacs.umd.edu"
We will be uploading this 4.4GB file.
# stat CASIA-WebFace.tar.gz File: `CASIA-WebFace.tar.gz' Size: 4018119666 Blocks: 7847896 IO Block: 262144 regular file Device: 13h/19d Inode: 87161 Links: 1 Access: (0644/-rw-r--r--) Uid: (10001/username) Gid: (18001/groupname) Access: 2015-11-24 19:34:05.715695000 -0500 Modify: 2015-11-23 14:23:35.872087000 -0500 Change: 2015-11-23 16:07:10.062306000 -0500
If the bucket does not exist yet you need to create the bucket first.
# mkobj janus_datasets: Created bucket janus_datasets.
Now you can copy the file to the bucket.
# cpobj CASIA-WebFace.tar.gz janus_datasets: 100% |#########################################################################################################################################|
To verify the object store has the data expected we can compare the checksums. First you can calculate the md5sum locally and then use cmpobj to stream the data from the object store and create a md5sum. They should match.
# md5sum CASIA-WebFace.tar.gz 004c2475e5ed66771e8873be67a93105 CASIA-WebFace.tar.gz # cmpobj janus_datasets:CASIA-WebFace.tar.gz 004c2475e5ed66771e8873be67a93105