UMobj/Example: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]). | In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or [[OBJ#LabGroups | LabGroup]] in the [https://obj.umiacs.umd.edu/obj/user/ Object Store]). | ||
<pre> | <pre> | ||
export OBJ_ACCESS_KEY_ID=<ACCESS_KEY> | export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>" | ||
export OBJ_SECRET_ACCESS_KEY=<SECRET_KEY> | export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>" | ||
export OBJ_SERVER="obj.umiacs.umd.edu" | export OBJ_SERVER="obj.umiacs.umd.edu" | ||
</pre> | </pre> | ||
Revision as of 15:58, 10 May 2024
In your shell, export the credentials if not already done (substituting in your actual ACCESS_KEY and SECRET_KEY for your personal account or LabGroup in the Object Store).
export OBJ_ACCESS_KEY_ID="<ACCESS_KEY>" export OBJ_SECRET_ACCESS_KEY="<SECRET_KEY>" export OBJ_SERVER="obj.umiacs.umd.edu"
We will be uploading this 4.4GB file.
# stat CASIA-WebFace.tar.gz File: `CASIA-WebFace.tar.gz' Size: 4018119666 Blocks: 7847896 IO Block: 262144 regular file Device: 13h/19d Inode: 87161 Links: 1 Access: (0644/-rw-r--r--) Uid: (12372/ carlos) Gid: (18149/ janus) Access: 2015-11-24 19:34:05.715695000 -0500 Modify: 2015-11-23 14:23:35.872087000 -0500 Change: 2015-11-23 16:07:10.062306000 -0500
If the bucket does not exist yet you need to create the bucket first.
# mkobj janus_datasets: Created bucket janus_datasets.
Now you can copy the file to the bucket.
# cpobj CASIA-WebFace.tar.gz janus_datasets: 100% |#########################################################################################################################################|
To verify the object store has the data expected we can compare the checksums. First you can calculate the md5sum locally and then use cmpobj to stream the data from the object store and create a md5sum. They should match.
# md5sum CASIA-WebFace.tar.gz 004c2475e5ed66771e8873be67a93105 CASIA-WebFace.tar.gz # cmpobj janus_datasets:CASIA-WebFace.tar.gz 004c2475e5ed66771e8873be67a93105