OBJ

UMIACS Object Store

An object store is a web-based storage solution focused on reliability, scalability, and security. It is best suited for public content storage/distribution, archiving data, or secure data sharing between users. Our Object Store can be used through the web interface, the command line UMobj utilities, third-party graphical clients, and even programmatically using many popular programming languages. We support a subset of the Amazon Simple Storage Services (S3) API, built around a technology called Ceph.

All data in our Object Store is stored on hard drives located in a datacenter managed by UMIACS Staff.

Terminology

S3-like storage thinks in terms of buckets and keys. Keys are analogous to files. A bucket is simply a container to a set of keys. There is no actual hierarchy inside of a bucket, but the standard UNIX path separator, a forward slash (/) at the end of a Key name, is interpreted by many clients (including this web site and our UMobj utilities) as being a directory delimiter. This allows you to copy data from your local filesystems to your buckets through umobj or third-party clients. You may specify who has what types of access to your buckets via Access Control Lists (ACLs) at the bucket level or the individual key level.

Your data is protected from individual machine failure via replication within the cluster. All data is checksummed in accordance with the Amazon S3 protocol to ensure that data in transit is valid before it is accepted by the cluster. However, there are no backups or snapshots of this data in the cluster, so if a user deletes a key or bucket in the Object Store, there is no way to restore that information.

Getting Started

UMIACS users are allocated 50GB of storage. Faculty are allocated 10TB. To get started, log in and you will be redirected to the initial help page. You can also find the link from our https://intranet.umiacs.umd.edu site as "OBJbox Object Store".

Buckets

You can create and browse your buckets (containers that hold data) by visiting your buckets page. You can also set bucket-level Access Control Lists (ACLs) from this page. Bucket-level ACLs get implicitly inherited to all the keys within the bucket. However, individual keys can have additional specific ACLs applied for more granular control.

Bucket names must be unique. When you create a bucket it will notify you if the name is already taken.

Keys (files)

After selecting a bucket, you will be able to create folders and upload files within that bucket. Listed files can be downloaded, deleted, or assigned a specific ACL by the key owner/creator.

Please note: Local file system ownership and permissions, and special files (such as symlinks) can not be represented in the Object Store. We highly suggest that if you are securing data into the Object Store and need these to be faithfully maintained that you use a local archive tool (tar, zip, etc..) to collect the data and then upload the resulting archive file(s).

Hosting a Website in your Bucket

Please visit OBJ/WebHosting for more information.

Deleting Keys (files)

Within the web interface you can delete files one-by-one. If you want to remove a bunch of files, you will need to use a different client as described below.

This is dangerous as there are no backups of files in the Object Store. Be careful to only delete the data you intend to delete.

Clients

There are several clients that can be used (sometimes with a limited set of features) on your desktop to gain access to the Object Store. All supported UMIACS systems running RHEL8 have a copy of our UMobj utilities preinstalled which provide command line access to the Object Store. We also have an article in our wiki on third party clients that lists and explains the details. These clients need to be configured with your Access and Secret Keys as described below.

Access Key and Secret Key

Each user has one or more pairs of Access Keys and Secret Keys that are used as a credential to not expose your password when using the Object Store. These can be obtained by clicking on your username in the upper right-hand corner. You'll use these to identify and authenticate yourself to the Object Store.

Please treat your Secret Key(s) with the same confidentiality as a password and do not plaintext it over email or otherwise or share it with any other individuals.

When using the umobj utilities, you will need to make sure you have added these credentials in your local shell initialization files. There are links to files that have these automatically generated for the 3 most popular UNIX shell families (bash/sh, csh/tcsh, and zsh). Please make sure that whatever file(s) you copy these credentials into can not be read by other users (eg. chmod 600 filename).

Note: Each Access Key and Secret Key are specific to a particular object store, so if you are accessing multiple object stores you may want to write the credentials for each to separate files and then source each file when you want to use the associated object store. Please contact staff if you have any questions.

LabGroups

LabGroups allow a group of users to share data while avoiding the need for complex ACLs by maintaining group ownership. Designated LabGroup managers can grant granular access (read, write, full control, manager) to buckets owned by the LabGroup. All objects owned by a LabGroup count against the group quota. LabGroups can be navigated using the menu with username in the top right corner of the page. Note: this will only appear if you are a member of at least one LabGroup. At this point, you can browse the Object Store as the LabGroup and obtain your unique Access Key and Secret Key pair using the instructions above. To switch to another LabGroup or back to your own buckets, click the menu again and select another user or group.

Managing LabGroups

LabGroups have many different levels of membership: Managers, FULL_CONTROL, READ/WRITE, and READ. Managers can add or remove LabGroup Members while every other access level cannot. If you hold the Manager role in a LabGroup, you can add and remove users using the Manage LabGroups page, which is available under the Manage menu at the top of the page. After selecting a LabGroup, you can add users by typing their username into the search field and selecting a membership role.

Requesting a LabGroup

To request a LabGroup for your project, please contact staff.