Rclone

Rclone is a command line program useful for syncing files and directories. Its functionality is similar to rsync, but has additional capabilities that support cloud storage services such as Dropbox, Open Drive, Amazon S3, and many more.

Below are directions on how to remote setup rclone on a headless machine. If you would like to set up specific Google Drive or Dropbox remotes, you can go directly to the section labeled "Google Drive" or "Dropbox".

Remote Setup on A Headless System

To remote setup rclone, you will first have to SSH into one of UMIACS' hosts and load the rclone module by typing module load rclone. If this does not work, check that the rclone module is available by running the command module avail rclone. If the module is not available, SSH to a host where it is.

If you are instead installing rclone on a system that you have administrative privileges on, downloads are available at https://rclone.org/downloads/.

Once you have loaded the module or installed the software, there are two ways to remote configure rclone. The first is by copying the rclone config file, and the second is by using rclone authorize.

Rclone configuration file

To copy the rclone config file, first configure rclone on your desktop machine. This can be achieved by running the command rclone config to set up the file and then finding the location of the configuration file using the command rclone config file, which should yield output similar to what is below:

$ rclone config file
Configuration file is stored at:
/home/user/.rclone.conf

This file can then be transferred to the remote box using scp, copy/paste, ftp, sftp, etc. The command rclone config file can also be run on the remote box to find the correct location to which the file should be moved.

Rclone authorize

To configure using rclone authorize instead, run the command Remote config on your headless machine, and then select the option for working on a headless machine by answering the first prompt with an n. The resulting output should look like this:

Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> n

For this to work, you will need rclone available on a machine that has a web browser available.
Execute the following on your machine:
	rclone authorize "amazon cloud drive"
Then paste the result below:
result>

Next, on your main desktop machine, run the command rclone authorize "amazon cloud drive". Follow the link that the output directs you to; you should now have received a secret token on your desktop machine console, in a message similar to the one below:

rclone authorize "amazon cloud drive"
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
Paste the following into your remote machine --->
SECRET_TOKEN
<---End paste

Go back to your headless machine and paste the secret message on the console where it says result>. Finally, approve the token by answering the prompts that follow:

result> SECRET_TOKEN
--------------------
[acd12]
client_id = 
client_secret = 
token = SECRET_TOKEN
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d>

If there are any issues with the remote setup, visit https://rclone.org/remote_setup/ for more information.

Google Drive

These instructions assume that you have SSHd into one of UMIACS' hosts and have successfully been able to load the rclone module using the command module load rclone, or installed rclone yourself on a system that you have administrative privileges on. If this is not the case, please do so before proceeding with the Google Drive remote setup.

First, make a new remote by running the command rclone config. This should produce output similar to what is below:

n) New remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
n/r/c/s/q> 

Choose to make a new remote, and give it a name. The setup will then prompt you to choose a type of storage to configure. You can either choose the number from the options that corresponds to "Google Drive", or simply type in drive. When prompted for your Google Application Client Id and Secret, it is okay to leave those blank.

Rclone will then prompt for a scope that it should use when requesting access from your drive, which you can provide by selecting one of the numbers on the screen. You can leave the root folder id blank, but enter remote config for the service account file. You will then be prompted to use auto config:

Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine or Y didn't work
y) Yes
n) No
y/n>

Choosing the y option is probably best here unless you already have rclone setup on a headless machine. The console should then redirect you to a browser page that will prompt you to login and authorize rclone for access. Once you have followed the instructions on the webpage, the console will display your credentials and ask for approval to confirm the remote.

Dropbox

These instructions assume that you have SSHd into one of UMIACS' hosts and have successfully been able to load the rclone module using the command module load rclone, or installed rclone yourself on a system that you have administrative privileges on. If this is not the case, please do so before proceeding with the Dropbox remote setup.

First, make a new remote by running the command rclone config. This should produce output similar to what is below:

n) New remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
n/r/c/s/q> 

Choose to make a new remote, and give it a name. The setup will then prompt you to choose a type of storage to configure. You can either choose the number from the options that corresponds to "Dropbox", or simply type in dropbox. When prompted for your Dropbox App Key and Secret, it is okay to leave those blank.

The console will then redirect you to a browser page that will prompt you to enter a code displayed on the console and authorize rclone for access. Once you have followed the instructions on the webpage, the console will display your credentials and ask for approval to confirm the remote.