Some of the additional features of rsync are:
itemize(
- it() support for copying links, devices, owners, groups and permissions
+ it() support for copying links, devices, owners, groups, and permissions
it() exclude and exclude-from options similar to GNU tar
it() a CVS exclude mode for ignoring the same files that CVS would ignore
it() can use any transparent remote shell, including ssh or rsh
--backup-dir make backups into this directory
--suffix=SUFFIX backup suffix (default ~ w/o --backup-dir)
-u, --update update only (don't overwrite newer files)
+ --inplace update the destination files inplace
+ -K, --keep-dirlinks treat symlinked dir on receiver as dir
-l, --links copy symlinks as symlinks
-L, --copy-links copy the referent of all symlinks
--copy-unsafe-links copy the referent of "unsafe" symlinks
--log-format=FORMAT log file transfers using specified format
--password-file=FILE get password from FILE
--bwlimit=KBPS limit I/O bandwidth, KBytes per second
- --write-batch=PREFIX write batch fileset starting with PREFIX
- --read-batch=PREFIX read batch fileset starting with PREFIX
+ --write-batch=FILE write a batch to FILE
+ --read-batch=FILE read a batch from FILE
--checksum-seed=NUM set block/file checksum seed
-4 --ipv4 prefer IPv4
-6 --ipv6 prefer IPv6
regardless of the timestamps. This might change in the future (feel
free to comment on this on the mailing list if you have an opinion).
+dit(bf(-K, --keep-dirlinks)) On the receiving side, if a symlink is
+pointing to a directory, it will be treated as matching a directory
+from the sender.
+
+dit(bf(--inplace)) This causes rsync not to create a new copy of the file
+and then move it into place. Instead rsync will overwrite the existing
+file, meaning that the rsync algorithm can't extract the full amount of
+network reduction it might otherwise.
+
+This option is useful for transfer of large files with block-based change
+or appended data, and also on systems that are disk bound not network bound.
+
+WARNING: The file's data will be in an inconsistent state during the
+transfer (and possibly afterward if the transfer gets interrupted), so you
+should not use this option to update files that are in use. Also note that
+rsync will be unable to update a file inplace that is not writable by the
+receiving user.
+
dit(bf(-l, --links)) When symlinks are encountered, recreate the
symlink on the destination.
dit(bf(-W, --whole-file)) With this option the incremental rsync algorithm
is not used and the whole file is sent as-is instead. The transfer may be
faster if this option is used when the bandwidth between the source and
-target machines is higher than the bandwidth to disk (especially when the
+destination machines is higher than the bandwidth to disk (especially when the
"disk" is actually a networked filesystem). This is the default when both
-the source and target are on the local machine.
+the source and destination are specified as local paths.
dit(bf(--no-whole-file)) Turn off --whole-file, for use when it is the
default.
dit(bf(--include-from=FILE)) This specifies a list of include patterns
from a file.
-If em(FILE) is bf(-) the list will be read from standard input.
+If em(FILE) is "-" the list will be read from standard input.
dit(bf(--files-from=FILE)) Using this option allows you to specify the
exact list of files to transfer (as read from the specified FILE or "-"
-for stdin). It also tweaks the default behavior of rsync to make
+for standard input). It also tweaks the default behavior of rsync to make
transferring just the specified files and directories easier. For
instance, the --relative option is enabled by default when this option
is used (use --no-relative if you want to turn that off), all
result is an average transfer rate equaling the specified limit. A value
of zero specifies no limit.
-dit(bf(--write-batch=PREFIX)) Generate a set of files that can be
-transferred as a batch update. Each filename in the set starts with
-PREFIX. See the "BATCH MODE" section for details.
+dit(bf(--write-batch=FILE)) Record a file that can later be applied to
+another identical destination with --read-batch. See the "BATCH MODE"
+section for details.
-dit(bf(--read-batch=PREFIX)) Apply a previously generated change batch,
-using the fileset whose filenames start with PREFIX. See the "BATCH
-MODE" section for details.
+dit(bf(--read-batch=FILE)) Apply all of the changes stored in FILE, a
+file previously generated by --write-batch.
+If em(FILE) is "-" the batch data will be read from standard input.
+See the "BATCH MODE" section for details.
dit(bf(-4, --ipv4) or bf(-6, --ipv6)) Tells rsync to prefer IPv4/IPv6
when creating sockets. This only affects sockets that rsync has direct
dit(bf(--checksum-seed=NUM)) Set the MD4 checksum seed to the integer
NUM. This 4 byte checksum seed is included in each block and file
MD4 checksum calculation. By default the checksum seed is generated
-by the server and defaults to the current time(), or 32761 if
-bf(--write-batch) or bf(--read-batch) are specified. This option
+by the server and defaults to the current time(). This option
is used to set a specific checksum seed, which is useful for
applications that want repeatable block and file checksums, or
in the case where the user wants a more random checksum seed.
Note that setting NUM to 0 causes rsync to use the default of time()
-for checksum seed. Note also that bf(--write-batch) and bf(--read-batch)
-set the checksum seed to 32761, so bf(--checksum-seed=NUM) needs to
-follow these options if you want to specify a different checksum
-seed in batch mode.
+for checksum seed.
enddit()
hosts. In order to do this using batch mode, rsync is run with the
write-batch option to apply the changes made to the source tree to one
of the destination trees. The write-batch option causes the rsync
-client to store the information needed to repeat this operation against
-other destination trees in a batch update fileset (see below). The
-filename of each file in the fileset starts with a prefix specified by
-the user as an argument to the write-batch option. This fileset is
-then copied to each remote host, where rsync is run with the read-batch
-option, again specifying the same prefix, and the destination tree.
-Rsync updates the destination tree using the information stored in the
-batch update fileset.
+client to store in a "batch file" all the information needed to repeat
+this operation against other, identical destination trees.
+
+To apply the recorded changes to another destination tree, run rsync
+with the read-batch option, specifying the name of the same batch
+file, and the destination tree. Rsync updates the destination tree
+using the information stored in the batch file.
+
+For convenience, one additional file is creating when the write-batch
+option is used. This file's name is created by appending
+".sh" to the batch filename. The .sh file contains
+a command-line suitable for updating a destination tree using that
+batch file. It can be executed using a Bourne(-like) shell, optionally
+passing in an alternate destination tree pathname which is then used
+instead of the original path. This is useful when the destination tree
+path differs from the original destination tree path.
+
+Generating the batch file once saves having to perform the file
+status, checksum, and data block generation more than once when
+updating multiple destination trees. Multicast transport protocols can
+be used to transfer the batch update files in parallel to many hosts
+at once, instead of sending the same data to every host individually.
-The fileset consists of 4 files:
+Examples:
-itemize(
-it() bf(<prefix>.rsync_argvs) command-line arguments
-it() bf(<prefix>.rsync_flist) rsync internal file metadata
-it() bf(<prefix>.rsync_csums) rsync checksums
-it() bf(<prefix>.rsync_delta) data blocks for file update & change
+verb(
+ $ rsync --write-batch=foo -a host:/source/dir/ /adest/dir/
+ $ scp foo* remote:
+ $ ssh remote ./foo.sh /bdest/dir/
)
-The .rsync_argvs file contains a command-line suitable for updating a
-destination tree using that batch update fileset. It can be executed
-using a Bourne(-like) shell, optionally passing in an alternate
-destination tree pathname which is then used instead of the original
-path. This is useful when the destination tree path differs from the
-original destination tree path.
+verb(
+ $ rsync --write-batch=foo -a /source/dir/ /adest/dir/
+ $ ssh remote rsync --read-batch=- -a /bdest/dir/ <foo
+)
-Generating the batch update fileset once saves having to perform the
-file status, checksum and data block generation more than once when
-updating multiple destination trees. Multicast transport protocols can
-be used to transfer the batch update files in parallel to many hosts at
-once, instead of sending the same data to every host individually.
+In these examples, rsync is used to update /adest/dir/ from /source/dir/
+and the information to repeat this operation is stored in "foo" and
+"foo.sh". The host "remote" is then updated with the batched data going
+into the directory /bdest/dir. The differences between the two examples
+reveals some of the flexibility you have in how you deal with batches:
-Example:
+itemize(
-verb(
- $ rsync --write-batch=pfx -a /source/dir/ /adest/dir/
- $ rcp pfx.rsync_* remote:
- $ ssh remote rsync --read-batch=pfx -a /bdest/dir/
- # or alternatively
- $ ssh remote ./pfx.rsync_argvs /bdest/dir/
-)
+ it() The first example shows that the initial copy doesn't have to be
+ local -- you can push or pull data to/from a remote host using either the
+ remote-shell syntax or rsync daemon syntax, as desired.
+
+ it() The first example uses the created "foo.sh" file to get the right
+ rsync options when running the read-batch command on the remote host.
-In this example, rsync is used to update /adest/dir/ with /source/dir/
-and the information to repeat this operation is stored in the files
-pfx.rsync_*. These files are then copied to the machine named "remote".
-Rsync is then invoked on "remote" to update /bdest/dir/ the same way as
-/adest/dir/. The last line shows the rsync_argvs file being used to
-invoke rsync.
+ it() The second example reads the batch data via standard input so that
+ the batch file doesn't need to be copied to the remote machine first.
+ This example avoids the foo.sh script because it needed to use a modified
+ --read-batch option, but you could edit the script file if you wished to
+ make use of it (just be sure that no other option is trying to use
+ standard input, such as the "--exclude-from=-" option).
+
+)
Caveats:
-The read-batch option expects the destination tree it is meant to update
+The read-batch option expects the destination tree that it is updating
to be identical to the destination tree that was used to create the
batch update fileset. When a difference between the destination trees
-is encountered the update will fail at that point, leaving the
+is encountered the update might fail at that point, leaving the
destination tree in a partially updated state. In that case, rsync can
be used in its regular (non-batch) mode of operation to fix up the
destination tree.
-The rsync version used on all destinations should be identical to the
-one used on the original destination.
+The rsync version used on all destinations must be at least as new as the
+one used to generate the batch file.
-The -z/--compress option does not work in batch mode and yields a usage
-error. A separate compression tool can be used instead to reduce the
-size of the batch update files for transport to the destination.
-
-The -n/--dryrun option does not work in batch mode and yields a runtime
+The --dry-run (-n) option does not work in batch mode and yields a runtime
error.
+You should use an equivalent set of options when reading a batch file that
+you used when generating it with a few exceptions. For instance
+--write-batch changes to --read-batch, --files-from is dropped, and the
+--include/--exclude options are not needed unless --delete is specified
+without --delete-excluded. Other options that affect how the update
+happens should generally remain the same as it is possible to confuse rsync
+into expecting a different data stream than the one that is contained in
+the batch file. For example, it would not work to change the setting of
+the -H or -c option, but it would work to add or remove the --delete
+option.
+
+The code that creates the BATCH.sh file transforms any include/exclude
+options into a single list that is appended as a "here" document to the
+shell script file. An advanced user can use this to modify the exclude
+list if a change in what gets deleted by --delete is desired. A normal
+user can ignore this detail and just use the shell script as an easy way
+to run the appropriate --read-batch command for the batched data.
+
See bf(http://www.ils.unc.edu/i2dsi/unc_rsync+.html) for papers and technical
reports.
I've probably missed some people, my apologies if I have.
Especial thanks also to: David Dykstra, Jos Backus, Sebastian Krahmer,
-Martin Pool, Wayne Davison.
+Martin Pool, Wayne Davison, J.W. Schultz.
manpageauthor()
Matt McCutchen's Web Site / rsync / rsync.git / blobdiff