Rclone - Revision history

Mbaney at 22:12, 11 November 2024

2024-11-11T22:12:06Z

← Older revision		Revision as of 22:12, 11 November 2024
Line 195:		Line 195:
	</pre>		</pre>

	To copy a local directory to a drive directory called backup		To copy a local filesystem directory to a drive directory called backup
	<pre>		<pre>
	rclone copy /home/source <REMOTENAME>:<NEWBACKUPNAME>		rclone copy /home/source <REMOTENAME>:<NEWBACKUPNAME>

Mbaney at 16:54, 25 October 2024

2024-10-25T16:54:42Z

← Older revision		Revision as of 16:54, 25 October 2024
Line 4:		Line 4:

	==Remote Setup on A Headless System==		==Remote Setup on A Headless System==
	To remote setup rclone, you will first have to [[SSH]] into one of UMIACS~~' hosts~~ and load the rclone module by typing <tt>module load rclone</tt>. If this does not work, check that the rclone module is available by running the command <tt>module avail rclone</tt>. If the module is not available, SSH to a host where it is.		To remote setup rclone, you will first have to [[SSH]] into one a UMIACS-supported host and load the rclone module by typing <tt>module load rclone</tt>. If this does not work, check that the rclone module is available by running the command <tt>module avail rclone</tt>. If the module is not available, SSH to a host where it is.

	If you are instead installing rclone on a system that you have administrative privileges on, downloads are available at https://rclone.org/downloads/.		If you are instead installing rclone on a system that you have administrative privileges on, downloads are available at https://rclone.org/downloads/.

Mbaney at 16:54, 25 October 2024

2024-10-25T16:54:11Z

← Older revision		Revision as of 16:54, 25 October 2024
Line 1:		Line 1:
	[https://rclone.org Rclone] is a command line program useful for syncing files and directories. Its functionality is similar to rsync, but has additional capabilities that support cloud storage services such as Google Drive, Dropbox, Box, OneDrive, Open Drive, Amazon S3, and many more.		[https://rclone.org Rclone] is a command line program useful for syncing files and directories. Its functionality is similar to [[LocalDataTransfer#Verifying_transfer \| rsync]], but has additional capabilities that support cloud storage services such as Google Drive, Dropbox, Box, OneDrive, Open Drive, Amazon S3, and many more.

	Below are directions on how to remote setup rclone on a headless machine. If you would like to set up specific Google Drive or Dropbox remotes, you can go directly to the section labeled "Google Drive" or "Dropbox".		Below are directions on how to remote setup rclone on a headless machine. If you would like to set up specific Google Drive or Dropbox remotes, you can go directly to the section labeled "Google Drive" or "Dropbox".

Mbaney at 16:37, 25 October 2024

2024-10-25T16:37:56Z

← Older revision		Revision as of 16:37, 25 October 2024
Line 1:		Line 1:
	Rclone is a command line program useful for syncing files and directories. Its functionality is similar to rsync, but has additional capabilities that support cloud storage services such as Google Drive, Dropbox, Box, OneDrive, Open Drive, Amazon S3, and many more.		[https://rclone.org Rclone] is a command line program useful for syncing files and directories. Its functionality is similar to rsync, but has additional capabilities that support cloud storage services such as Google Drive, Dropbox, Box, OneDrive, Open Drive, Amazon S3, and many more.

	Below are directions on how to remote setup rclone on a headless machine. If you would like to set up specific Google Drive or Dropbox remotes, you can go directly to the section labeled "Google Drive" or "Dropbox".		Below are directions on how to remote setup rclone on a headless machine. If you would like to set up specific Google Drive or Dropbox remotes, you can go directly to the section labeled "Google Drive" or "Dropbox".

Mbaney at 16:37, 25 October 2024

2024-10-25T16:37:39Z

← Older revision		Revision as of 16:37, 25 October 2024
Line 1:		Line 1:
	Rclone is a command line program useful for syncing files and directories. Its functionality is similar to rsync, but has additional capabilities that support cloud storage services such as Dropbox, Open Drive, Amazon S3, and many more.		Rclone is a command line program useful for syncing files and directories. Its functionality is similar to rsync, but has additional capabilities that support cloud storage services such as Google Drive, Dropbox, Box, OneDrive, Open Drive, Amazon S3, and many more.

	Below are directions on how to remote setup rclone on a headless machine. If you would like to set up specific Google Drive or Dropbox remotes, you can go directly to the section labeled "Google Drive" or "Dropbox".		Below are directions on how to remote setup rclone on a headless machine. If you would like to set up specific Google Drive or Dropbox remotes, you can go directly to the section labeled "Google Drive" or "Dropbox".

Mbaney at 13:49, 12 March 2024

2024-03-12T13:49:19Z

← Older revision		Revision as of 13:49, 12 March 2024
Line 163:		Line 163:
	</pre>		</pre>

	The window will then prompt for sign in with a Google account. Sign into your UMD account using your ~~$USERNAME~~@umd.edu email. You will then be brought to the UMD CAS Dual Factor Authentication Screen. Sign in with your normal credentials and dual factor authenticate. Allow rclone to access your Google Account.		The window will then prompt for sign in with a Google account. Sign into your UMD account using your <DIRECTORYID>@umd.edu email. You will then be brought to the UMD CAS Dual Factor Authentication Screen. Sign in with your normal credentials and dual factor authenticate. Allow rclone to access your Google Account.

	You will then be prompted to Configure this as a team drive. Enter the default value unless the application requires further customization.		You will then be prompted to Configure this as a team drive. Enter the default value unless the application requires further customization.
Line 187:		Line 187:
	List directories in top level of your drive		List directories in top level of your drive
	<pre>		<pre>
	rclone lsd $REMOTENAME$:		rclone lsd <REMOTENAME>:
	</pre>		</pre>

	List all the files in your drive		List all the files in your drive
	<pre>		<pre>
	rclone ls $REMOTENAME$:		rclone ls <REMOTENAME>:
	</pre>		</pre>

	To copy a local directory to a drive directory called backup		To copy a local directory to a drive directory called backup
	<pre>		<pre>
	rclone copy /home/source $REMOTENAME$:$NEWBACKUPNAME$		rclone copy /home/source <REMOTENAME>:<NEWBACKUPNAME>
	</pre>		</pre>

Line 211:		Line 211:
	Then use <tt>crontab -e</tt> to edit your crontab. Once in the crontab, you can input your Rclone command using the command path found previously. An example of an Rclone cronjob that would occur every hour on the hour would be:		Then use <tt>crontab -e</tt> to edit your crontab. Once in the crontab, you can input your Rclone command using the command path found previously. An example of an Rclone cronjob that would occur every hour on the hour would be:
	<pre>		<pre>
	0 * * * * /opt/common/rclone-v1.53.4-linux-amd64/rclone copy /$PATHTOBEBACKEDUP$/ $REMOTENAME$:$BACKUPNAME$		0 * * * * /opt/common/rclone-v1.53.4-linux-amd64/rclone copy /<PATHTOBEBACKEDUP>/ <REMOTENAME>:<BACKUPNAME>
	</pre>		</pre>

	==Dropbox==		==Dropbox==

	These instructions assume that you have SSH'd into one of UMIACS' hosts and have successfully been able to load the rclone module using the command <tt>module load rclone</tt>, or installed rclone yourself on a system that you have administrative privileges on. If this is not the case, please do so before proceeding with the Dropbox remote setup.		These instructions assume that you have SSH'd into one of UMIACS' hosts and have successfully been able to load the rclone module using the command <tt>module load rclone</tt>, or installed rclone yourself on a system that you have administrative privileges on. If this is not the case, please do so before proceeding with the Dropbox remote setup.

Mbaney at 13:41, 5 September 2023

2023-09-05T13:41:36Z

← Older revision		Revision as of 13:41, 5 September 2023
Line 4:		Line 4:

	==Remote Setup on A Headless System==		==Remote Setup on A Headless System==

	To remote setup rclone, you will first have to [[SSH]] into one of UMIACS' hosts and load the rclone module by typing <tt>module load rclone</tt>. If this does not work, check that the rclone module is available by running the command <tt>module avail rclone</tt>. If the module is not available, SSH to a host where it is.		To remote setup rclone, you will first have to [[SSH]] into one of UMIACS' hosts and load the rclone module by typing <tt>module load rclone</tt>. If this does not work, check that the rclone module is available by running the command <tt>module avail rclone</tt>. If the module is not available, SSH to a host where it is.

Line 12:		Line 11:

	===Rclone configuration file===		===Rclone configuration file===

	To copy the rclone config file, first configure rclone on your desktop machine. This can be achieved by running the command <tt>rclone config</tt> to set up the file and then finding the location of the configuration file using the command <tt>rclone config file</tt>, which should yield output similar to what is below:		To copy the rclone config file, first configure rclone on your desktop machine. This can be achieved by running the command <tt>rclone config</tt> to set up the file and then finding the location of the configuration file using the command <tt>rclone config file</tt>, which should yield output similar to what is below:

Line 24:		Line 22:

	===Rclone authorize===		===Rclone authorize===

	To configure using rclone authorize instead, run the command <tt>Remote config</tt> on your headless machine, and then select the option for working on a headless machine by answering the first prompt with an <tt>n</tt>. The resulting output should look like this:		To configure using rclone authorize instead, run the command <tt>Remote config</tt> on your headless machine, and then select the option for working on a headless machine by answering the first prompt with an <tt>n</tt>. The resulting output should look like this:

Line 75:		Line 72:

	==Google Drive==		==Google Drive==
			In order to sync files to Google Drive on Linux systems, use of the command line program Rclone is necessary. The first step is to SSH into one of UMIACS' hosts with X11 forwarding enabled (use the <code>-XY</code> flag) or (if the host is a workstation) log in over GUI and launch the terminal. If Rclone is already in the module tree, then load the Rclone module using:
	In order to sync files to Google Drive on Linux systems, use of the command line program Rclone is necessary.
	The first step is to SSH into one of UMIACS' hosts with X11 forwarding enabled (~~Use~~ the -XY flag) or ~~gain physical access to~~ the ~~command line~~.
	If Rclone is already in the module tree then load ~~into~~ the Rclone module using:
	<pre>		<pre>
	module load rclone		module load rclone
Line 116:		Line 110:
	Storage> drive		Storage> drive
	</pre>		</pre>
	When prompted for your Google Application Client Id and Secret, it is okay to leave those blank. Rclone will then prompt for a scope that it should use when requesting access from your drive, which you can provide by selecting one of the numbers on the screen. The suggested value for full operation is <tt>1</tt> .
			When prompted for your Google Application Client Id and Secret, it is okay to leave those blank. Rclone will then prompt for a scope that it should use when requesting access from your drive, which you can provide by selecting one of the numbers on the screen. The suggested value for full operation is <tt>1</tt>.

	<pre>		<pre>
	Google Application Client Id		Google Application Client Id
Line 147:		Line 143:
	scope> 1		scope> 1
	</pre>		</pre>

	You can leave the <tt>root_folder_id</tt> and the <tt>service_account_file</tt> blank. You will then be prompted to edit the advanced config. Enter the default value unless the application requires further customization.		You can leave the <tt>root_folder_id</tt> and the <tt>service_account_file</tt> blank. You will then be prompted to edit the advanced config. Enter the default value unless the application requires further customization.

	<pre>		<pre>
	Edit advanced config? (y/n)		Edit advanced config? (y/n)
Line 164:		Line 162:
	y/n>		y/n>
	</pre>		</pre>

	The window will then prompt for sign in with a Google account. Sign into your UMD account using your $USERNAME@umd.edu email. You will then be brought to the UMD CAS Dual Factor Authentication Screen. Sign in with your normal credentials and dual factor authenticate. Allow rclone to access your Google Account.		The window will then prompt for sign in with a Google account. Sign into your UMD account using your $USERNAME@umd.edu email. You will then be brought to the UMD CAS Dual Factor Authentication Screen. Sign in with your normal credentials and dual factor authenticate. Allow rclone to access your Google Account.

	You will then be prompted to Configure this as a team drive. Enter the default value unless the application requires further customization.		You will then be prompted to Configure this as a team drive. Enter the default value unless the application requires further customization.

	<pre>		<pre>
	Configure this as a team drive?		Configure this as a team drive?
Line 174:		Line 174:
	</pre>		</pre>

	Finally, confirm your selections if they are correct, edit them if they are incorrect.		Finally, confirm your selections if they are correct, edit them if they are incorrect.

	<pre>		<pre>
	--------------------		--------------------
Line 188:		Line 189:
	rclone lsd $REMOTENAME$:		rclone lsd $REMOTENAME$:
	</pre>		</pre>

	List all the files in your drive		List all the files in your drive
	<pre>		<pre>
	rclone ls $REMOTENAME$:		rclone ls $REMOTENAME$:
	</pre>		</pre>

	To copy a local directory to a drive directory called backup		To copy a local directory to a drive directory called backup
	<pre>		<pre>
Line 205:		Line 208:
	/opt/common/rclone-v1.53.4-linux-amd64/rclone		/opt/common/rclone-v1.53.4-linux-amd64/rclone
	</pre>		</pre>
	Then use <tt>crontab -e</tt> to edit your crontab. Once in the crontab you can input your Rclone command using the command path found previously. An example of an Rclone cronjob that would occur every hour on the hour would be:
			Then use <tt>crontab -e</tt> to edit your crontab. Once in the crontab, you can input your Rclone command using the command path found previously. An example of an Rclone cronjob that would occur every hour on the hour would be:
	<pre>		<pre>
	0 * * * * /opt/common/rclone-v1.53.4-linux-amd64/rclone copy /$PATHTOBEBACKEDUP$/ $REMOTENAME$:$BACKUPNAME$		0 * * * * /opt/common/rclone-v1.53.4-linux-amd64/rclone copy /$PATHTOBEBACKEDUP$/ $REMOTENAME$:$BACKUPNAME$
Line 212:		Line 216:
	==Dropbox==		==Dropbox==

	These instructions assume that you have ~~SSHd~~ into one of UMIACS' hosts and have successfully been able to load the rclone module using the command <tt>module load rclone</tt>, or installed rclone yourself on a system that you have administrative privileges on. If this is not the case, please do so before proceeding with the Dropbox remote setup.		These instructions assume that you have SSH'd into one of UMIACS' hosts and have successfully been able to load the rclone module using the command <tt>module load rclone</tt>, or installed rclone yourself on a system that you have administrative privileges on. If this is not the case, please do so before proceeding with the Dropbox remote setup.

	First, make a new remote by running the command <tt>rclone config</tt>. This should produce output similar to what is below:		First, make a new remote by running the command <tt>rclone config</tt>. This should produce output similar to what is below:

Mbaney: /* Creating a CronJob */

2021-12-07T17:09:50Z

Creating a CronJob

← Older revision		Revision as of 17:09, 7 December 2021
Line 199:		Line 199:
	More commands and usage can be found at: https://rclone.org/drive/		More commands and usage can be found at: https://rclone.org/drive/

	===Creating a ~~CronJob~~===		===Creating a cronjob===
	A ~~CronJob~~ will allow you to automate backups. To create a ~~CronJob~~ for an Rclone command, first you must find the path of Rclone. This can be done using the <tt>which</tt> command.		A cronjob will allow you to automate backups. To create a cronjob for an Rclone command, first you must find the path of Rclone. This can be done using the <tt>which</tt> command.
	<pre>		<pre>
	[user@workstation ~]$ which rclone		[user@workstation ~]$ which rclone
	/opt/common/rclone-v1.53.4-linux-amd64/rclone		/opt/common/rclone-v1.53.4-linux-amd64/rclone
	</pre>		</pre>
	Then use <tt>crontab -e</tt> to edit your crontab. Once in the crontab you can input your Rclone command using the command path found previously. An example of an Rclone that would occur every hour on the hour would be:		Then use <tt>crontab -e</tt> to edit your crontab. Once in the crontab you can input your Rclone command using the command path found previously. An example of an Rclone cronjob that would occur every hour on the hour would be:
	<pre>		<pre>
	0 * * * * /opt/common/rclone-v1.53.4-linux-amd64/rclone copy /$PATHTOBEBACKEDUP$/ $REMOTENAME$:$BACKUPNAME$		0 * * * * /opt/common/rclone-v1.53.4-linux-amd64/rclone copy /$PATHTOBEBACKEDUP$/ $REMOTENAME$:$BACKUPNAME$

Kfunka at 20:09, 24 February 2021

2021-02-24T20:09:43Z

@@ Line 181: / Line 181: @@
 d) Delete this remote
 y/e/d> y
 </pre>
@@ Line 209: / Line 198: @@
 More commands and usage can be found at: https://rclone.org/drive/
 ==Dropbox==

Kfunka at 20:08, 24 February 2021

2021-02-24T20:08:52Z

← Older revision		Revision as of 20:08, 24 February 2021
Line 191:		Line 191:
	Then use <tt>crontab -e</tt> to edit your crontab. Once in the crontab you can input your Rclone command using the command path found previously. An example of an Rclone that would occur every hour on the hour would be:		Then use <tt>crontab -e</tt> to edit your crontab. Once in the crontab you can input your Rclone command using the command path found previously. An example of an Rclone that would occur every hour on the hour would be:
	<pre>		<pre>
	0 * * * * /opt/common/rclone-v1.53.4-linux-amd64/rclone copy ~/$PATHTOBEBACKEDUP$ $REMOTENAME$:$BACKUPNAME$		0 * * * * /opt/common/rclone-v1.53.4-linux-amd64/rclone copy /$PATHTOBEBACKEDUP$/ $REMOTENAME$:$BACKUPNAME$
	</pre>		</pre>