Difference between revisions of "S3Clients"

From UMIACS
Jump to navigation Jump to search
 
(15 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
Many popular S3 desktop clients can be used to access the UMIACS Object Store.  These tools complement the [[UMobj]] command line utilities and the built-in web interface by providing integration with the native file explorer on your desktop machine.
 
Many popular S3 desktop clients can be used to access the UMIACS Object Store.  These tools complement the [[UMobj]] command line utilities and the built-in web interface by providing integration with the native file explorer on your desktop machine.
  
'''Note''': Many of these clients have features that are just not supported by our Object Stores in UMIACS.  This includes permissions, which we suggest you manage permissions from the built-in web application for the Object Store.
+
'''Note''': Many of these clients have features that are not supported by our Object Store in UMIACS.  One prominent example of this is permissions. We suggest you instead manage permissions from the [https://obj.umiacs.umd.edu/obj built-in web application] for the Object Store.
  
==Dragon Disk==
+
=Graphical Clients=
* http://www.dragondisk.com/
 
 
 
This is a free, cross platform S3 client that supports Windows, Mac and Linux.  To configure Dragon Disk, select the "Other S3 compatible service" dropdown menu choice in the add account dialog.  The following fields are required.
 
 
 
* '''Service Endpoint''' - This should be your object store, <code>obj.umiacs.umd.edu</code> or <code>gembox.cbcb.umd.edu</code>
 
* '''Access Key''' - This is your access key as provided to you in the object store
 
* '''Secret Key''' - This is your secret key as provided to you in the object store
 
* '''Connect using SSL/HTTPS''' - Please toggle the checkbox.
 
  
 
==Cyberduck==
 
==Cyberduck==
Line 18: Line 10:
 
This is a free Windows and Mac S3 browser (it is however nagware that asks for a donation).  It supports our S3 Object Stores using the "S3 (Amazon Simple Storage Service)"  drop down menu choice in the add bookmark dialog.  The following fields are required.
 
This is a free Windows and Mac S3 browser (it is however nagware that asks for a donation).  It supports our S3 Object Stores using the "S3 (Amazon Simple Storage Service)"  drop down menu choice in the add bookmark dialog.  The following fields are required.
  
* '''Server''' - This should be your object store, <code>obj.umiacs.umd.edu</code> or <code>gembox.cbcb.umd.edu</code>
+
[[Image:Cyberduck.png|400px]]
* '''Username''' - This is your access key as provided to you in the object store
+
 
 +
* '''Server''' - This should be your object store (<code>obj.umiacs.umd.edu</code>)
 +
* '''Access Key ID''' - This is your access key as provided to you in the object store
 +
* '''Password''' - This is your secret key as provided to you in the object store
  
 
You will be prompted for your secret key when you connect and may choose to save the password.
 
You will be prompted for your secret key when you connect and may choose to save the password.
Line 27: Line 22:
 
This is a paid file transfer application for Mac.  It supports our S3 Object Stores using the "S3" menu choice after clicking the plus sign to add a favorite.  The following fields are required:
 
This is a paid file transfer application for Mac.  It supports our S3 Object Stores using the "S3" menu choice after clicking the plus sign to add a favorite.  The following fields are required:
  
* '''Server''' - This should be your object store: <code>obj.umiacs.umd.edu</code> or <code>gembox.cbcb.umd.edu</code>
+
[[Image:Transmit.png|400px]]
 +
 
 +
* '''Server''' - This should be your object store (<code>obj.umiacs.umd.edu</code>)
 
* '''Access Key ID''' - This is your access key as provided to you in the object store
 
* '''Access Key ID''' - This is your access key as provided to you in the object store
 
* '''Secret''' - This is your secret key as provided to you in the object store
 
* '''Secret''' - This is your secret key as provided to you in the object store
Line 33: Line 30:
 
These settings can be saved as a favorite for easy access.  Transmit also allows you to mount your Obj buckets as local disks, which will support easy drag-and-drop of files.
 
These settings can be saved as a favorite for easy access.  Transmit also allows you to mount your Obj buckets as local disks, which will support easy drag-and-drop of files.
  
 +
=Command Line Clients=
 
==s3cmd==
 
==s3cmd==
Command line client for accessing S3 like services.
+
Command line client for accessing S3-like services.
  
 
* http://s3tools.org/s3cmd
 
* http://s3tools.org/s3cmd
Line 47: Line 45:
 
secret_key = SECRET_KEY
 
secret_key = SECRET_KEY
 
use_https = True
 
use_https = True
 +
</pre>
 +
 +
==mc==
 +
The Minio Client is a comprehensive single binary (Go) command line client for cloud based storage services.
 +
 +
* https://min.io/download
 +
 +
Users in UMIACS can run this client on supported systems through adding a software [[Modules|modules]] for <code>mc</code>.
 +
 +
<pre>
 +
module add mc
 +
</pre>
 +
 +
You will need to setup a cloud provider for Obj and you should first retrieve the ACCESS_KEY and SECRET_KEY for your personal or LabAccount in the [https://obj.umiacs.umd.edu/obj/user/ Object Store ].
 +
 +
<pre>
 +
mc config host add obj http://obj.umiacs.umd.edu ACCESS_KEY SECRET_KEY
 +
</pre>
 +
 +
You can then use the normal <code>mc</code> commands like the following to list the contents of a bucket.
 +
 +
<pre>
 +
$ mc ls obj/iso
 +
[2017-02-10 16:45:04 EST] 3.5GiB rhel-server-7.3-x86_64-dvd.iso
 +
[2017-02-13 12:21:33 EST] 4.0GiB rhel-workstation-7.3-x86_64-dvd.iso
 +
</pre>
 +
 +
There is also the ability to search for file globs of specific files using the <code>find</code> sub-command for <code>mc</code>.
 +
 +
<pre>
 +
$ mc find derek/derek_support --name "*.log"
 +
derek/derek_support/mds_20170918/ceph-mds.objmds01.log
 +
derek/derek_support/satellite.log
 +
derek/derek_support/umiacs-49168.log
 
</pre>
 
</pre>

Latest revision as of 13:41, 14 May 2021

Many popular S3 desktop clients can be used to access the UMIACS Object Store. These tools complement the UMobj command line utilities and the built-in web interface by providing integration with the native file explorer on your desktop machine.

Note: Many of these clients have features that are not supported by our Object Store in UMIACS. One prominent example of this is permissions. We suggest you instead manage permissions from the built-in web application for the Object Store.

Graphical Clients

Cyberduck

This is a free Windows and Mac S3 browser (it is however nagware that asks for a donation). It supports our S3 Object Stores using the "S3 (Amazon Simple Storage Service)" drop down menu choice in the add bookmark dialog. The following fields are required.

Cyberduck.png

  • Server - This should be your object store (obj.umiacs.umd.edu)
  • Access Key ID - This is your access key as provided to you in the object store
  • Password - This is your secret key as provided to you in the object store

You will be prompted for your secret key when you connect and may choose to save the password.

Transmit

This is a paid file transfer application for Mac. It supports our S3 Object Stores using the "S3" menu choice after clicking the plus sign to add a favorite. The following fields are required:

Transmit.png

  • Server - This should be your object store (obj.umiacs.umd.edu)
  • Access Key ID - This is your access key as provided to you in the object store
  • Secret - This is your secret key as provided to you in the object store

These settings can be saved as a favorite for easy access. Transmit also allows you to mount your Obj buckets as local disks, which will support easy drag-and-drop of files.

Command Line Clients

s3cmd

Command line client for accessing S3-like services.

You need to configure a file like ~/.s3cmd that looks like the following with your ACCESS_KEY and SECRET_KEY substituted.

[default]
access_key = ACCESS_KEY
host_base = obj.umiacs.umd.edu
host_bucket = %(bucket)s.obj.umiacs.umd.edu
secret_key = SECRET_KEY
use_https = True

mc

The Minio Client is a comprehensive single binary (Go) command line client for cloud based storage services.

Users in UMIACS can run this client on supported systems through adding a software modules for mc.

module add mc

You will need to setup a cloud provider for Obj and you should first retrieve the ACCESS_KEY and SECRET_KEY for your personal or LabAccount in the Object Store .

mc config host add obj http://obj.umiacs.umd.edu ACCESS_KEY SECRET_KEY

You can then use the normal mc commands like the following to list the contents of a bucket.

$ mc ls obj/iso
[2017-02-10 16:45:04 EST] 3.5GiB rhel-server-7.3-x86_64-dvd.iso
[2017-02-13 12:21:33 EST] 4.0GiB rhel-workstation-7.3-x86_64-dvd.iso

There is also the ability to search for file globs of specific files using the find sub-command for mc.

$ mc find derek/derek_support --name "*.log"
derek/derek_support/mds_20170918/ceph-mds.objmds01.log
derek/derek_support/satellite.log
derek/derek_support/umiacs-49168.log