Some of the additional features of rsync are:
itemize(
- it() support for copying links, devices, owners, groups and permissions
+ it() support for copying links, devices, owners, groups, and permissions
it() exclude and exclude-from options similar to GNU tar
it() a CVS exclude mode for ignoring the same files that CVS would ignore
it() can use any transparent remote shell, including ssh or rsh
--backup-dir make backups into this directory
--suffix=SUFFIX backup suffix (default ~ w/o --backup-dir)
-u, --update update only (don't overwrite newer files)
+ --inplace update the destination files inplace
+ -K, --keep-dirlinks treat symlinked dir on receiver as dir
-l, --links copy symlinks as symlinks
-L, --copy-links copy the referent of all symlinks
--copy-unsafe-links copy the referent of "unsafe" symlinks
-W, --whole-file copy whole files, no incremental checks
--no-whole-file turn off --whole-file
-x, --one-file-system don't cross filesystem boundaries
- -B, --block-size=SIZE checksum blocking size (default 700)
+ -B, --block-size=SIZE force a fixed checksum block-size
-e, --rsh=COMMAND specify the remote shell
--rsync-path=PATH specify path to rsync on the remote machine
--existing only update files that already exist
--ignore-errors delete even if there are I/O errors
--max-delete=NUM don't delete more than NUM files
--partial keep partially transferred files
+ --partial-dir=DIR put a partially transferred file into DIR
--force force deletion of dirs even if not empty
--numeric-ids don't map uid/gid values by user/group name
--timeout=TIME set I/O timeout in seconds
--log-format=FORMAT log file transfers using specified format
--password-file=FILE get password from FILE
--bwlimit=KBPS limit I/O bandwidth, KBytes per second
- --write-batch=PREFIX write batch fileset starting with PREFIX
- --read-batch=PREFIX read batch fileset starting with PREFIX
+ --write-batch=FILE write a batch to FILE
+ --read-batch=FILE read a batch from FILE
+ --checksum-seed=NUM set block/file checksum seed
+ -4 --ipv4 prefer IPv4
+ -6 --ipv6 prefer IPv6
-h, --help show this help screen
regardless of the timestamps. This might change in the future (feel
free to comment on this on the mailing list if you have an opinion).
+dit(bf(-K, --keep-dirlinks)) On the receiving side, if a symlink is
+pointing to a directory, it will be treated as matching a directory
+from the sender.
+
+dit(bf(--inplace)) This causes rsync not to create a new copy of the file
+and then move it into place. Instead rsync will overwrite the existing
+file, meaning that the rsync algorithm can't extract the full amount of
+network reduction it might otherwise.
+
+This option is useful for transfer of large files with block-based change
+or appended data, and also on systems that are disk bound not network bound.
+
+WARNING: The file's data will be in an inconsistent state during the
+transfer (and possibly afterward if the transfer gets interrupted), so you
+should not use this option to update files that are in use. Also note that
+rsync will be unable to update a file inplace that is not writable by the
+receiving user.
+
dit(bf(-l, --links)) When symlinks are encountered, recreate the
symlink on the destination.
dit(bf(-L, --copy-links)) When symlinks are encountered, the file that
-they point to (the referent) is copied, rather than the symlink.
+they point to (the referent) is copied, rather than the symlink. In older
+versions of rsync, this option also had the side-effect of telling the
+receiving side to follow symlinks, such as symlinks to directories. In a
+modern rsync such as this one, you'll need to specify --keep-dirlinks (-K)
+to get this extra behavior. The only exception is when sending files to
+an rsync that is too old to understand -K -- in that case, the -L option
+will still have the side-effect of -K on that older receiving rsync.
dit(bf(--copy-unsafe-links)) This tells rsync to copy the referent of
symbolic links that point outside the copied tree. Absolute symlinks
dit(bf(-W, --whole-file)) With this option the incremental rsync algorithm
is not used and the whole file is sent as-is instead. The transfer may be
faster if this option is used when the bandwidth between the source and
-target machines is higher than the bandwidth to disk (especially when the
+destination machines is higher than the bandwidth to disk (especially when the
"disk" is actually a networked filesystem). This is the default when both
-the source and target are on the local machine.
+the source and destination are specified as local paths.
dit(bf(--no-whole-file)) Turn off --whole-file, for use when it is the
default.
is only relevant without --delete because deletions are now done depth-first.
Requires the --recursive option (which is implied by -a) to have any effect.
-dit(bf(-B , --block-size=BLOCKSIZE)) This controls the block size used in
-the rsync algorithm. See the technical report for details.
+dit(bf(-B, --block-size=BLOCKSIZE)) This forces the block size used in
+the rsync algorithm to a fixed value. It is normally selected based on
+the size of each file being updated. See the technical report for details.
dit(bf(-e, --rsh=COMMAND)) This option allows you to choose an alternative
remote shell program to use for communication between the local and
dit(bf(--include-from=FILE)) This specifies a list of include patterns
from a file.
-If em(FILE) is bf(-) the list will be read from standard input.
+If em(FILE) is "-" the list will be read from standard input.
dit(bf(--files-from=FILE)) Using this option allows you to specify the
exact list of files to transfer (as read from the specified FILE or "-"
-for stdin). It also tweaks the default behavior of rsync to make
+for standard input). It also tweaks the default behavior of rsync to make
transferring just the specified files and directories easier. For
instance, the --relative option is enabled by default when this option
is used (use --no-relative if you want to turn that off), all
--partial option tells rsync to keep the partial file which should
make a subsequent transfer of the rest of the file much faster.
+dit(bf(--partial-dir=DIR)) Turns on --partial mode, but tells rsync to
+put a partially transferred file into DIR instead of writing out the
+file to the destination dir. Rsync will also use a file found in this
+dir as data to speed up the transfer (i.e. when you redo the send after
+rsync creates a partial file) and delete such a file after it has served
+its purpose.
+
+Rsync will create the dir if it is missing (just the last dir -- not the
+whole path). This makes it easy to use a relative path (such as
+"--partial-dir=.rsync-partial") to have rsync create the partial-directory
+in the destination file's directory (rsync will also try to remove the DIR
+if a partial file was found to exist at the start of the transfer and the
+DIR was specified as a relative path).
+
+If you are deleting files on the destination and your partial-dir is
+inside the destination hierarchy, make sure you specify an exclude to
+prevent the partial file from being deleted (it could get deleted at the
+end of the transfer when using --delete-after, or at the beginning of the
+transfer when using --delete). E.g. "--exclude=.rsync-partial/".
+
+IMPORTANT: the --partial-dir should not be writable by other users to
+avoid a security risk. E.g. AVOID "/tmp".
+
dit(bf(--progress)) This option tells rsync to print information
showing the progress of the transfer. This gives a bored user
something to watch.
result is an average transfer rate equaling the specified limit. A value
of zero specifies no limit.
-dit(bf(--write-batch=PREFIX)) Generate a set of files that can be
-transferred as a batch update. Each filename in the set starts with
-PREFIX. See the "BATCH MODE" section for details.
-
-dit(bf(--read-batch=PREFIX)) Apply a previously generated change batch,
-using the fileset whose filenames start with PREFIX. See the "BATCH
-MODE" section for details.
+dit(bf(--write-batch=FILE)) Record a file that can later be applied to
+another identical destination with --read-batch. See the "BATCH MODE"
+section for details.
+
+dit(bf(--read-batch=FILE)) Apply all of the changes stored in FILE, a
+file previously generated by --write-batch.
+If em(FILE) is "-" the batch data will be read from standard input.
+See the "BATCH MODE" section for details.
+
+dit(bf(-4, --ipv4) or bf(-6, --ipv6)) Tells rsync to prefer IPv4/IPv6
+when creating sockets. This only affects sockets that rsync has direct
+control over, such as the outgoing socket when directly contacting an
+rsync daemon, or the incoming sockets that an rsync daemon uses to
+listen for connections. One of these options may be required in older
+versions of Linux to work around an IPv6 bug in the kernel (if you see
+an "address already in use" error when nothing else is using the port,
+try specifying --ipv6 or --ipv4 when starting the daemon).
+
+dit(bf(--checksum-seed=NUM)) Set the MD4 checksum seed to the integer
+NUM. This 4 byte checksum seed is included in each block and file
+MD4 checksum calculation. By default the checksum seed is generated
+by the server and defaults to the current time(). This option
+is used to set a specific checksum seed, which is useful for
+applications that want repeatable block and file checksums, or
+in the case where the user wants a more random checksum seed.
+Note that setting NUM to 0 causes rsync to use the default of time()
+for checksum seed.
enddit()
manpagesection(BATCH MODE)
bf(Note:) Batch mode should be considered experimental in this version
-of rsync. The interface or behavior may change before it stabilizes.
+of rsync. The interface and behavior have now stabilized, though, so
+feel free to try this out.
Batch mode can be used to apply the same set of updates to many
identical systems. Suppose one has a tree which is replicated on a
hosts. In order to do this using batch mode, rsync is run with the
write-batch option to apply the changes made to the source tree to one
of the destination trees. The write-batch option causes the rsync
-client to store the information needed to repeat this operation against
-other destination trees in a batch update fileset (see below). The
-filename of each file in the fileset starts with a prefix specified by
-the user as an argument to the write-batch option. This fileset is
-then copied to each remote host, where rsync is run with the read-batch
-option, again specifying the same prefix, and the destination tree.
-Rsync updates the destination tree using the information stored in the
-batch update fileset.
+client to store in a "batch file" all the information needed to repeat
+this operation against other, identical destination trees.
+
+To apply the recorded changes to another destination tree, run rsync
+with the read-batch option, specifying the name of the same batch
+file, and the destination tree. Rsync updates the destination tree
+using the information stored in the batch file.
+
+For convenience, one additional file is creating when the write-batch
+option is used. This file's name is created by appending
+".sh" to the batch filename. The .sh file contains
+a command-line suitable for updating a destination tree using that
+batch file. It can be executed using a Bourne(-like) shell, optionally
+passing in an alternate destination tree pathname which is then used
+instead of the original path. This is useful when the destination tree
+path differs from the original destination tree path.
+
+Generating the batch file once saves having to perform the file
+status, checksum, and data block generation more than once when
+updating multiple destination trees. Multicast transport protocols can
+be used to transfer the batch update files in parallel to many hosts
+at once, instead of sending the same data to every host individually.
-The fileset consists of 4 files:
+Examples:
-itemize(
-it() bf(<prefix>.rsync_argvs) command-line arguments
-it() bf(<prefix>.rsync_flist) rsync internal file metadata
-it() bf(<prefix>.rsync_csums) rsync checksums
-it() bf(<prefix>.rsync_delta) data blocks for file update & change
+verb(
+ $ rsync --write-batch=foo -a host:/source/dir/ /adest/dir/
+ $ scp foo* remote:
+ $ ssh remote ./foo.sh /bdest/dir/
)
-The .rsync_argvs file contains a command-line suitable for updating a
-destination tree using that batch update fileset. It can be executed
-using a Bourne(-like) shell, optionally passing in an alternate
-destination tree pathname which is then used instead of the original
-path. This is useful when the destination tree path differs from the
-original destination tree path.
+verb(
+ $ rsync --write-batch=foo -a /source/dir/ /adest/dir/
+ $ ssh remote rsync --read-batch=- -a /bdest/dir/ <foo
+)
-Generating the batch update fileset once saves having to perform the
-file status, checksum and data block generation more than once when
-updating multiple destination trees. Multicast transport protocols can
-be used to transfer the batch update files in parallel to many hosts at
-once, instead of sending the same data to every host individually.
+In these examples, rsync is used to update /adest/dir/ from /source/dir/
+and the information to repeat this operation is stored in "foo" and
+"foo.sh". The host "remote" is then updated with the batched data going
+into the directory /bdest/dir. The differences between the two examples
+reveals some of the flexibility you have in how you deal with batches:
-Example:
+itemize(
-verb(
- $ rsync --write-batch=pfx -a /source/dir/ /adest/dir/
- $ rcp pfx.rsync_* remote:
- $ ssh remote rsync --read-batch=pfx -a /bdest/dir/
- # or alternatively
- $ ssh remote ./pfx.rsync_argvs /bdest/dir/
-)
+ it() The first example shows that the initial copy doesn't have to be
+ local -- you can push or pull data to/from a remote host using either the
+ remote-shell syntax or rsync daemon syntax, as desired.
-In this example, rsync is used to update /adest/dir/ with /source/dir/
-and the information to repeat this operation is stored in the files
-pfx.rsync_*. These files are then copied to the machine named "remote".
-Rsync is then invoked on "remote" to update /bdest/dir/ the same way as
-/adest/dir/. The last line shows the rsync_argvs file being used to
-invoke rsync.
+ it() The first example uses the created "foo.sh" file to get the right
+ rsync options when running the read-batch command on the remote host.
+
+ it() The second example reads the batch data via standard input so that
+ the batch file doesn't need to be copied to the remote machine first.
+ This example avoids the foo.sh script because it needed to use a modified
+ --read-batch option, but you could edit the script file if you wished to
+ make use of it (just be sure that no other option is trying to use
+ standard input, such as the "--exclude-from=-" option).
+
+)
Caveats:
-The read-batch option expects the destination tree it is meant to update
+The read-batch option expects the destination tree that it is updating
to be identical to the destination tree that was used to create the
batch update fileset. When a difference between the destination trees
-is encountered the update will fail at that point, leaving the
-destination tree in a partially updated state. In that case, rsync can
+is encountered the update might be discarded with no error (if the file
+appears to be up-to-date already) or the file-update may be attempted
+and then, if the file fails to verify, the update discarded with an
+error. This means that it should be safe to re-run a read-batch operation
+if the command got updated. If you wish to force the batched-update to
+always be attempted regardless of the file's size and date, use the -I
+option. If an error occurs, the destination tree will probably be in a
+partially updated state. In that case, rsync can
be used in its regular (non-batch) mode of operation to fix up the
destination tree.
-The rsync version used on all destinations should be identical to the
-one used on the original destination.
+The rsync version used on all destinations must be at least as new as the
+one used to generate the batch file.
-The -z/--compress option does not work in batch mode and yields a usage
-error. A separate compression tool can be used instead to reduce the
-size of the batch update files for transport to the destination.
-
-The -n/--dryrun option does not work in batch mode and yields a runtime
+The --dry-run (-n) option does not work in batch mode and yields a runtime
error.
+When reading a batch file, rsync will force the value of certain options
+to match the data in the batch file if you didn't set them to the same
+as the batch-writing command. Other options can (and should) be changed.
+For instance
+--write-batch changes to --read-batch, --files-from is dropped, and the
+--include/--exclude options are not needed unless --delete is specified
+without --delete-excluded.
+
+The code that creates the BATCH.sh file transforms any include/exclude
+options into a single list that is appended as a "here" document to the
+shell script file. An advanced user can use this to modify the exclude
+list if a change in what gets deleted by --delete is desired. A normal
+user can ignore this detail and just use the shell script as an easy way
+to run the appropriate --read-batch command for the batched data.
+
See bf(http://www.ils.unc.edu/i2dsi/unc_rsync+.html) for papers and technical
reports.
I've probably missed some people, my apologies if I have.
Especial thanks also to: David Dykstra, Jos Backus, Sebastian Krahmer,
-Martin Pool, Wayne Davison.
+Martin Pool, Wayne Davison, J.W. Schultz.
manpageauthor()