X-Git-Url: https://mattmccutchen.net/rsync/rsync.git/blobdiff_plain/416cef36e98fc2ce2c75bd76b8e97d892a3d88b8..bd685982389b78a158921b7839bdeca501338d19:/rsync.yo diff --git a/rsync.yo b/rsync.yo index 5eb69ee1..fcb8b43a 100644 --- a/rsync.yo +++ b/rsync.yo @@ -356,9 +356,11 @@ to the detailed description below for a complete description. verb( -D same as --devices --specials -t, --times preserve modification times -O, --omit-dir-times omit directories from --times + -J, --omit-link-times omit symlinks from --times --super receiver attempts super-user activities --fake-super store/recover privileged attrs using xattrs -S, --sparse handle sparse files efficiently + --preallocate allocate dest files before writing -n, --dry-run perform a trial run with no changes made -W, --whole-file copy files whole (w/o delta-xfer algorithm) -x, --one-file-system don't cross filesystem boundaries @@ -375,6 +377,8 @@ to the detailed description below for a complete description. verb( --delete-delay find deletions during, delete after --delete-after receiver deletes after transfer, not before --delete-excluded also delete excluded files from dest dirs + --ignore-missing-args ignore missing source args without error + --delete-missing-args delete missing source args from destination --ignore-errors delete even if there are I/O errors --force force deletion of dirs even if not empty --max-delete=NUM don't delete more than NUM files @@ -428,7 +432,7 @@ to the detailed description below for a complete description. verb( --log-file-format=FMT log updates using the specified FMT --password-file=FILE read daemon-access password from FILE --list-only list the files instead of copying them - --bwlimit=KBPS limit I/O bandwidth; KBytes per second + --bwlimit=RATE limit socket I/O bandwidth --write-batch=FILE write a batched update to FILE --only-write-batch=FILE like --write-batch but w/o updating dest --read-batch=FILE read a batched update from FILE @@ -444,7 +448,7 @@ Rsync can also be run as a daemon, in which case the following options are accepted: verb( --daemon run as an rsync daemon --address=ADDRESS bind to the specified address - --bwlimit=KBPS limit I/O bandwidth; KBytes per second + --bwlimit=RATE limit socket I/O bandwidth --config=FILE specify alternate rsyncd.conf file -M, --dparam=OVERRIDE override global daemon config parameter --no-detach do not detach from the parent @@ -564,7 +568,7 @@ dit(bf(-c, --checksum)) This changes the way rsync checks if the files have been changed and are in need of a transfer. Without this option, rsync uses a "quick check" that (by default) checks if each file's size and time of last modification match between the sender and receiver. This option -changes this to compare a 128-bit MD4 checksum for each file that has a +changes this to compare a 128-bit checksum for each file that has a matching size. Generating the checksums means that both sides will expend a lot of disk I/O reading all the data in the files in the transfer (and this is prior to any reading that will be done to transfer changed files), @@ -582,6 +586,9 @@ checksum that is generated as the file is transferred, but that automatic after-the-transfer verification has nothing to do with this option's before-the-transfer "Does this file need to be updated?" check. +For protocol 30 and beyond (first supported in 3.0.0), the checksum used is +MD5. For older protocols, the checksum used is MD4. + dit(bf(-a, --archive)) This is equivalent to bf(-rlptgoD). It is a quick way of saying you want recursion and want to preserve almost everything (with -H being a notable omission). @@ -669,7 +676,7 @@ quote(tt( rsync -avR /foo/./bar/baz.c remote:/tmp/)) That would create /tmp/bar/baz.c on the remote machine. (Note that the dot must be followed by a slash, so "/foo/." would not be abbreviated.) -(2) For older rsync versions, you would need to use a chdir to limit the +For older rsync versions, you would need to use a chdir to limit the source path. For example, when pushing files: quote(tt( (cd /foo; rsync -avR bar/baz.c remote:/tmp/) )) @@ -731,6 +738,12 @@ specify a backup suffix using the bf(--suffix) option (otherwise the files backed up in the specified directory will keep their original filenames). +Note that if you specify a relative path, the backup directory will be +relative to the destination directory, so you probably want to specify +either an absolute path or a path that starts with "../". If an rsync +daemon is the receiver, the backup dir cannot go outside the module's path +hierarchy, so take extra care not to delete it or copy into it. + dit(bf(--suffix=SUFFIX)) This option allows you to override the default backup suffix used with the bf(--backup) (bf(-b)) option. The default suffix is a ~ if no -bf(-backup-dir) was specified, otherwise it is an empty string. @@ -747,29 +760,45 @@ date is on the objects. In other words, if the source has a directory where the destination has a file, the transfer would occur regardless of the timestamps. -dit(bf(--inplace)) This option changes how rsync transfers a file when the -file's data needs to be updated: instead of the default method of creating +This option is a transfer rule, not an exclude, so it doesn't affect the +data that goes into the file-lists, and thus it doesn't affect deletions. +It just limits the files that the receiver requests to be transferred. + +dit(bf(--inplace)) This option changes how rsync transfers a file when +its data needs to be updated: instead of the default method of creating a new copy of the file and moving it into place when it is complete, rsync instead writes the updated data directly to the destination file. -This has several effects: (1) in-use binaries cannot be updated (either the -OS will prevent this from happening, or binaries that attempt to swap-in -their data will misbehave or crash), (2) the file's data will be in an -inconsistent state during the transfer, (3) a file's data may be left in an -inconsistent state after the transfer if the transfer is interrupted or if -an update fails, (4) a file that does not have write permissions can not be -updated, and (5) the efficiency of rsync's delta-transfer algorithm may be -reduced if some data in the destination file is overwritten before it can -be copied to a position later in the file (one exception to this is if you -combine this option with bf(--backup), since rsync is smart enough to use -the backup file as the basis file for the transfer). +This has several effects: + +quote(itemization( + it() Hard links are not broken. This means the new data will be visible + through other hard links to the destination file. Moreover, attempts to + copy differing source files onto a multiply-linked destination file will + result in a "tug of war" with the destination data changing back and forth. + it() In-use binaries cannot be updated (either the OS will prevent this from + happening, or binaries that attempt to swap-in their data will misbehave or + crash). + it() The file's data will be in an inconsistent state during the transfer + and will be left that way if the transfer is interrupted or if an update + fails. + it() A file that rsync cannot write to cannot be updated. While a super user + can update any file, a normal user needs to be granted write permission for + the open of the file for writing to be successful. + it() The efficiency of rsync's delta-transfer algorithm may be reduced if + some data in the destination file is overwritten before it can be copied to + a position later in the file. This does not apply if you use bf(--backup), + since rsync is smart enough to use the backup file as the basis file for the + transfer. +)) WARNING: you should not use this option to update files that are being accessed by others, so be careful when choosing to use this for a copy. -This option is useful for transfer of large files with block-based changes +This option is useful for transferring large files with block-based changes or appended data, and also on systems that are disk bound, not network -bound. +bound. It can also help keep a copy-on-write filesystem snapshot from +diverging the entire contents of a file that only has minor changes. The option implies bf(--partial) (since an interrupted transfer does not delete the file), but conflicts with bf(--partial-dir) and bf(--delay-updates). @@ -872,6 +901,17 @@ bf(--force) or bf(--delete) is in effect). See also bf(--keep-dirlinks) for an analogous option for the receiving side. +bf(--copy-dirlinks) applies to all symlinks to directories in the source. If +you want to follow only a few specified symlinks, a trick you can use is to +pass them as additional source args with a trailing slash, using bf(--relative) +to make the paths match up right. For example: + +quote(tt(rsync -r --relative src/./ src/./follow-me/ dest/)) + +This works because rsync calls bf(lstat)(2) on the source arg as given, and the +trailing slash makes bf(lstat)(2) follow the symlink, giving rise to a directory +in the file-list which overrides the symlink found during the scan of "src/./". + dit(bf(-K, --keep-dirlinks)) This option causes the receiving side to treat a symlink to a directory as though it were a real directory, but only if it matches a real directory from the sender. Without this option, the @@ -895,17 +935,25 @@ to modify your receiving hierarchy. See also bf(--copy-dirlinks) for an analogous option for the sending side. dit(bf(-H, --hard-links)) This tells rsync to look for hard-linked files in -the transfer and link together the corresponding files on the receiving -side. Without this option, hard-linked files in the transfer are treated +the source and link together the corresponding files on the destination. +Without this option, hard-linked files in the source are treated as though they were separate files. -When you are updating a non-empty destination, this option only ensures -that files that are hard-linked together on the source are hard-linked -together on the destination. It does NOT currently endeavor to break -already existing hard links on the destination that do not exist between -the source files. Note, however, that if one or more extra-linked files -have content changes, they will become unlinked when updated (assuming you -are not using the bf(--inplace) option). +This option does NOT necessarily ensure that the pattern of hard links on the +destination exactly matches that on the source. Cases in which the +destination may end up with extra hard links include the following: + +quote(itemization( + it() If the destination contains extraneous hard-links (more linking than + what is present in the source file list), the copying algorithm will not + break them explicitly. However, if one or more of the paths have content + differences, the normal file-update process will break those extra links + (unless you are using the bf(--inplace) option). + it() If you specify a bf(--link-dest) directory that contains hard links, + the linking of the destination files against the bf(--link-dest) files can + cause some paths in the destination to become linked together due to the + bf(--link-dest) associations. +)) Note that rsync can only detect hard links between files that are inside the transfer set. If rsync updates a file that has extra hard-link @@ -918,7 +966,10 @@ see the bf(--inplace) option for more caveats). If incremental recursion is active (see bf(--recursive)), rsync may transfer a missing hard-linked file before it finds that another link for that contents exists elsewhere in the hierarchy. This does not affect the accuracy of -the transfer, just its efficiency. One way to avoid this is to disable +the transfer (i.e. which files are hard-linked together), just its efficiency +(i.e. copying the data for a new, early copy of a hard-linked file that could +have been found later in the transfer in another member of the hard-linked +set of files). One way to avoid this inefficiency is to disable incremental recursion using the bf(--no-inc-recursive) option. dit(bf(-p, --perms)) This option causes the receiving rsync to set the @@ -996,27 +1047,38 @@ The source and destination systems must have compatible ACL entries for this option to work properly. See the bf(--fake-super) option for a way to backup and restore ACLs that are not compatible. -dit(bf(-X, --xattrs)) This option causes rsync to update the remote -extended attributes to be the same as the local ones. +dit(bf(-X, --xattrs)) This option causes rsync to update the destination +extended attributes to be the same as the source ones. For systems that support extended-attribute namespaces, a copy being done by a super-user copies all namespaces except system.*. A normal user only copies the user.* namespace. To be able to backup and restore non-user namespaces as a normal user, see the bf(--fake-super) option. +Note that this option does not copy rsyncs special xattr values (e.g. those +used by bf(--fake-super)) unless you repeat the option (e.g. -XX). This +"copy all xattrs" mode cannot be used with bf(--fake-super). + dit(bf(--chmod)) This option tells rsync to apply one or more -comma-separated "chmod" strings to the permission of the files in the -transfer. The resulting value is treated as though it was the permissions +comma-separated "chmod" modes to the permission of the files in the +transfer. The resulting value is treated as though it were the permissions that the sending side supplied for the file, which means that this option can seem to have no effect on existing files if bf(--perms) is not enabled. In addition to the normal parsing rules specified in the bf(chmod)(1) manpage, you can specify an item that should only apply to a directory by prefixing it with a 'D', or specify an item that should only apply to a -file by prefixing it with a 'F'. For example: +file by prefixing it with a 'F'. For example, the following will ensure +that all directories get marked set-gid, that no files are other-writable, +that both are user-writable and group-writable, and that both have +consistent executability across all bits: quote(--chmod=Dg+s,ug+w,Fo-w,+X) +Using octal mode numbers is also allowed: + +quote(--chmod=D2775,F664) + It is also legal to specify multiple bf(--chmod) options, as each additional option is just appended to the list of changes to make. @@ -1069,6 +1131,9 @@ it is preserving modification times (see bf(--times)). If NFS is sharing the directories on the receiving side, it is a good idea to use bf(-O). This option is inferred if you use bf(--backup) without bf(--backup-dir). +dit(bf(-J, --omit-link-times)) This tells rsync to omit symlinks when +it is preserving modification times (see bf(--times)). + dit(bf(--super)) This tells the receiving side to attempt super-user activities even if the receiving rsync wasn't run by the super-user. These activities include: preserving users via the bf(--owner) option, preserving @@ -1076,7 +1141,7 @@ all groups (not just the current user's groups) via the bf(--groups) option, and copying devices via the bf(--devices) option. This is useful for systems that allow such activities without being the super-user, and also for ensuring that you will get errors if the receiving side isn't -being running as the super-user. To turn off super-user activities, the +being run as the super-user. To turn off super-user activities, the super-user can use bf(--no-super). dit(bf(--fake-super)) When this option is enabled, rsync simulates @@ -1114,9 +1179,16 @@ dit(bf(-S, --sparse)) Try to handle sparse files efficiently so they take up less space on the destination. Conflicts with bf(--inplace) because it's not possible to overwrite data in a sparse fashion. -NOTE: Don't use this option when the destination is a Solaris "tmpfs" -filesystem. It doesn't seem to handle seeks over null regions -correctly and ends up corrupting the files. +dit(bf(--preallocate)) This tells the receiver to allocate each destination +file to its eventual size before writing data to the file. Rsync will only use +the real filesystem-level preallocation support provided by Linux's +bf(fallocate)(2) system call or Cygwin's bf(posix_fallocate)(3), not the slow +glibc implementation that writes a zero byte into each block. + +Without this option, larger files may not be entirely contiguous on the +filesystem, but with this option rsync will probably copy more slowly. If the +destination is not an extent-supporting filesystem (such as ext4, xfs, NTFS, +etc.), this option may have no positive effect at all. dit(bf(-n, --dry-run)) This makes rsync perform a trial run that doesn't make any changes (and produces mostly the same output as a real run). It @@ -1126,19 +1198,20 @@ to do before one actually runs it. The output of bf(--itemize-changes) is supposed to be exactly the same on a dry run and a subsequent real run (barring intentional trickery and system -call failures); if it isn't, that's a bug. Other output is the same to the -extent practical, but may differ in some areas. Notably, a dry run does not +call failures); if it isn't, that's a bug. Other output should be mostly +unchanged, but may differ in some areas. Notably, a dry run does not send the actual data for file transfers, so bf(--progress) has no effect, the "bytes sent", "bytes received", "literal data", and "matched data" statistics are too small, and the "speedup" value is equivalent to a run -where no file transfers are needed. +where no file transfers were needed. dit(bf(-W, --whole-file)) With this option rsync's delta-transfer algorithm is not used and the whole file is sent as-is instead. The transfer may be faster if this option is used when the bandwidth between the source and destination machines is higher than the bandwidth to disk (especially when the "disk" is actually a networked filesystem). This is the default when both -the source and destination are specified as local paths. +the source and destination are specified as local paths, but only if no +batch-writing option is in effect. dit(bf(-x, --one-file-system)) This tells rsync to avoid crossing a filesystem boundary when recursing. This does not limit the user's ability @@ -1164,10 +1237,18 @@ yet on the destination. If this option is combined with the bf(--ignore-existing) option, no files will be updated (which can be useful if all you want to do is delete extraneous files). +This option is a transfer rule, not an exclude, so it doesn't affect the +data that goes into the file-lists, and thus it doesn't affect deletions. +It just limits the files that the receiver requests to be transferred. + dit(bf(--ignore-existing)) This tells rsync to skip updating files that already exist on the destination (this does em(not) ignore existing directories, or nothing would get done). See also bf(--existing). +This option is a transfer rule, not an exclude, so it doesn't affect the +data that goes into the file-lists, and thus it doesn't affect deletions. +It just limits the files that the receiver requests to be transferred. + This option can be useful for those doing backups using the bf(--link-dest) option when they need to continue a backup run that got interrupted. Since a bf(--link-dest) run is copied into a new directory hierarchy (when it is @@ -1180,6 +1261,19 @@ dit(bf(--remove-source-files)) This tells rsync to remove from the sending side the files (meaning non-directories) that are a part of the transfer and have been successfully duplicated on the receiving side. +Note that you should only use this option on source files that are quiescent. +If you are using this to move files that show up in a particular directory over +to another host, make sure that the finished files get renamed into the source +directory, not directly written into it, so that rsync can't possibly transfer +a file that is not yet fully written. If you can't first write the files into +a different directory, you should use a naming idiom that lets rsync avoid +transferring files that are not yet finished (e.g. name the file "foo.new" when +it is written, rename it to "foo" when it is done, and then use the option +bf(--exclude='*.new') for the rsync transfer). + +Starting with 3.1.0, rsync will skip the sender-side removal (and output an +error) if the file's size or modify time has not stayed unchanged. + dit(bf(--delete)) This tells rsync to delete extraneous files from the receiving side (ones that aren't on the sending side), but only for the directories that are being synchronized. You must have asked rsync to @@ -1202,7 +1296,7 @@ going to be deleted. If the sending side detects any I/O errors, then the deletion of any files at the destination will be automatically disabled. This is to prevent temporary filesystem failures (such as NFS errors) on the -sending side causing a massive deletion of files on the +sending side from causing a massive deletion of files on the destination. You can override this with the bf(--ignore-errors) option. The bf(--delete) option may be combined with one of the --delete-WHEN options @@ -1263,6 +1357,23 @@ this way on the receiver, and for a way to protect files from bf(--delete-excluded). See bf(--delete) (which is implied) for more details on file-deletion. +dit(bf(--ignore-missing-args)) When rsync is first processing the explicitly +requested source files (e.g. command-line arguments or bf(--files-from) +entries), it is normally an error if the file cannot be found. This option +suppresses that error, and does not try to transfer the file. This does not +affect subsequent vanished-file errors if a file was initially found to be +present and later is no longer there. + +dit(bf(--delete-missing-args)) This option takes the behavior of (the implied) +bf(--ignore-missing-args) option a step farther: each missing arg will become +a deletion request of the corresponding destination file on the receiving side +(should it exist). If the destination file is a non-empty directory, it will +only be successfully deleted if --force or --delete are in effect. Other than +that, this option is independent of any other type of delete processing. + +The missing source files are represented by special file-list entries which +display as a "*missing" entry in the bf(--list-only) output. + dit(bf(--ignore-errors)) Tells bf(--delete) to go ahead and delete files even when there are I/O errors. @@ -1290,6 +1401,10 @@ file that is larger than the specified SIZE. The SIZE value can be suffixed with a string to indicate a size multiplier, and may be a fractional value (e.g. "bf(--max-size=1.5m)"). +This option is a transfer rule, not an exclude, so it doesn't affect the +data that goes into the file-lists, and thus it doesn't affect deletions. +It just limits the files that the receiver requests to be transferred. + The suffixes are as follows: "K" (or "KiB") is a kibibyte (1024), "M" (or "MiB") is a mebibyte (1024*1024), and "G" (or "GiB") is a gibibyte (1024*1024*1024). @@ -1304,7 +1419,7 @@ Examples: --max-size=1.5mb-1 is 1499999 bytes, and --max-size=2g+1 is dit(bf(--min-size=SIZE)) This tells rsync to avoid transferring any file that is smaller than the specified SIZE, which can help in not transferring small, junk files. -See the bf(--max-size) option for a description of SIZE. +See the bf(--max-size) option for a description of SIZE and other information. dit(bf(-B, --block-size=BLOCKSIZE)) This forces the block size used in rsync's delta-transfer algorithm to a fixed value. It is normally selected based on @@ -1397,7 +1512,7 @@ initial items are marked as perishable -- see the FILTER RULES section): quote(quote(tt(RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state .nse_depinfo *~ #* .#* ,* _$* *$ *.old *.bak *.BAK *.orig *.rej .del-* -*.a *.olb *.o *.obj *.so *.exe *.Z *.elc *.ln core .svn/ .git/ .bzr/))) +*.a *.olb *.o *.obj *.so *.exe *.Z *.elc *.ln core .svn/ .git/ .hg/ .bzr/))) then, files listed in a $HOME/.cvsignore are added to the list and any files listed in the CVSIGNORE environment variable (all cvsignore names @@ -1522,6 +1637,17 @@ quote(tt( rsync -a --files-from=:/path/file-list src:/ /tmp/copy)) This would copy all the files specified in the /path/file-list file that was located on the remote "src" host. +If the bf(--iconv) and bf(--protect-args) options are specified and the +bf(--files-from) filenames are being sent from one host to another, the +filenames will be translated from the sending host's charset to the +receiving host's charset. + +NOTE: sorting the list of files in the --files-from input helps rsync to be +more efficient, as it will avoid re-visiting the path elements that are shared +between adjacent entries. If the input is not sorted, some path elements +(implied directories) may end up being scanned multiple times, and rsync will +eventually unduplicate them after they get turned into file-list elements. + dit(bf(-0, --from0)) This tells rsync that the rules/filenames it reads from a file are terminated by a null ('\0') character, not a NL, CR, or CR+LF. This affects bf(--exclude-from), bf(--include-from), bf(--files-from), and any @@ -1529,21 +1655,31 @@ merged files specified in a bf(--filter) rule. It does not affect bf(--cvs-exclude) (since all names read from a .cvsignore file are split on whitespace). -If the bf(--iconv) and bf(--protect-args) options are specified and the -bf(--files-from) filenames are being sent from one host to another, the -filenames will be translated from the sending host's charset to the -receiving host's charset. - -dit(bf(-s, --protect-args)) This option sends all filenames and some options to +dit(bf(-s, --protect-args)) This option sends all filenames and most options to the remote rsync without allowing the remote shell to interpret them. This means that spaces are not split in names, and any non-wildcard special characters are not translated (such as ~, $, ;, &, etc.). Wildcards are expanded on the remote host by rsync (instead of the shell doing it). -If you use this option with bf(--iconv), the args will also be translated +If you use this option with bf(--iconv), the args related to the remote +side will also be translated from the local to the remote character-set. The translation happens before wild-cards are expanded. See also the bf(--files-from) option. +You may also control this option via the RSYNC_PROTECT_ARGS environment +variable. If this variable has a non-zero value, this option will be enabled +by default, otherwise it will be disabled by default. Either state is +overridden by a manually specified positive or negative version of this option +(note that bf(--no-s) and bf(--no-protect-args) are the negative versions). +Since this option was first introduced in 3.0.0, you'll need to make sure it's +disabled if you ever need to interact with a remote rsync that is older than +that. + +Rsync can also be configured (at build time) to have this option enabled by +default (with is overridden by both the environment and the command-line). +This option will eventually become a new default setting at some +as-yet-undetermined point in the future. + dit(bf(-T, --temp-dir=DIR)) This option instructs rsync to use DIR as a scratch directory when creating temporary copies of the files transferred on the receiving side. The default behavior is to create each temporary @@ -1684,7 +1820,7 @@ You may specify an empty string to indicate that no file should be skipped. Simple character-class matching is supported: each must consist of a list of letters inside the square brackets (e.g. no special classes, such as -"[:alpha:]", are supported). +"[:alpha:]", are supported, and '-' has no special meaning). The characters asterisk (*) and question-mark (?) have no special meaning. @@ -1693,10 +1829,37 @@ matches 2 suffixes): verb( --skip-compress=gz/jpg/mp[34]/7z/bz2) -The default list of suffixes that will not be compressed is this (several -of these are newly added for 3.0.0): - -verb( gz/zip/z/rpm/deb/iso/bz2/t[gb]z/7z/mp[34]/mov/avi/ogg/jpg/jpeg) +The default list of suffixes that will not be compressed is this (in this +version of rsync): + +bf(7z) +bf(ace) +bf(avi) +bf(bz2) +bf(deb) +bf(gpg) +bf(gz) +bf(iso) +bf(jpeg) +bf(jpg) +bf(lz) +bf(lzma) +bf(lzo) +bf(mov) +bf(mp3) +bf(mp4) +bf(ogg) +bf(png) +bf(rar) +bf(rpm) +bf(rzip) +bf(tbz) +bf(tgz) +bf(tlz) +bf(txz) +bf(xz) +bf(z) +bf(zip) This list will be replaced by your bf(--skip-compress) list in all but one situation: a copy from a daemon rsync will add your skipped suffixes to @@ -1926,6 +2089,9 @@ specify an empty string, updated files will not be mentioned in the log file. For a list of the possible escape characters, see the "log format" setting in the rsyncd.conf manpage. +The default FORMAT used if bf(--log-file) is specified and this option is not +is '%i %n%L'. + dit(bf(--stats)) This tells rsync to print a verbose set of statistics on the file transfer, allowing you to tell how effective rsync's delta-transfer algorithm is for your data. This option is equivalent to bf(--info=stats2) @@ -1934,10 +2100,23 @@ with 2 or more bf(-v) options. The current statistics are as follows: quote(itemization( it() bf(Number of files) is the count of all "files" (in the generic - sense), which includes directories, symlinks, etc. - it() bf(Number of files transferred) is the count of normal files that - were updated via rsync's delta-transfer algorithm, which does not include created - dirs, symlinks, etc. + sense), which includes directories, symlinks, etc. The total count will + be followed by a list of counts by filetype (if the total is non-zero). + For example: "(reg: 5, dir: 3, link: 2, dev: 1, special: 1)" lists the + totals for regular files, directories, symlinks, devices, and special + files. If any of value is 0, it is completely omitted from the list. + it() bf(Number of created files) is the count of how many "files" (generic + sense) were created (as opposed to updated). The total count will be + followed by a list of counts by filetype (if the total is non-zero). + it() bf(Number of deleted files) is the count of how many "files" (generic + sense) were created (as opposed to updated). The total count will be + followed by a list of counts by filetype (if the total is non-zero). + Note that this line is only output if deletions are in effect, and only + if protocol 31 is being used (the default for rsync 3.1.x). + it() bf(Number of regular files transferred) is the count of normal files + that were updated via rsync's delta-transfer algorithm, which does not + include dirs, symlinks, etc. Note that rsync 3.1.0 added the word + "regular" into this heading. it() bf(Total file size) is the total sum of all file sizes in the transfer. This does not count any size for directories or special files, but does include the size of symlinks. @@ -1992,8 +2171,9 @@ in level-2 (assuming that a period is your local decimal point). Backward compatibility note: versions of rsync prior to 3.1.0 do not support human-readable level 1, and they default to level 0. Thus, specifying one or -two bf(-h) options behaves the same in old and new versions as long as you -didn't specify a bf(--no-h) option prior to one or more bf(-h) options. +two bf(-h) options will behave in a comparable manner in old and new versions +as long as you didn't specify a bf(--no-h) option prior to one or more bf(-h) +options. See the bf(--list-only) option for one difference. dit(bf(--partial)) By default, rsync will delete any partially transferred file if the transfer is interrupted. In some circumstances @@ -2091,11 +2271,16 @@ creation of a bunch of useless directories when the sending rsync is recursively scanning a hierarchy of files using include/exclude/filter rules. +Note that the use of transfer rules, such as the bf(--min-size) option, does +not affect what goes into the file list, and thus does not leave directories +empty, even if none of the files in a directory match the transfer rule. + Because the file-list is actually being pruned, this option also affects what directories get deleted when a delete is active. However, keep in mind that excluded files and directories can prevent existing items from -being deleted (because an exclude hides source files and protects -destination files). +being deleted due to an exclude both hiding source files and protecting +destination files. See the perishable filter-rule option for how to avoid +this. You can prevent the pruning of certain empty directories from the file-list by using a global "protect" filter. For instance, this option would ensure @@ -2141,18 +2326,24 @@ was finishing the matched part of the file. When the file transfer finishes, rsync replaces the progress line with a summary line that looks like this: -verb( 1238099 100% 146.38kB/s 0:00:08 (xfer#5, to-check=169/396)) +verb( 1,238,099 100% 146.38kB/s 0:00:08 (xfr#5, to-chk=169/396)) -In this example, the file was 1238099 bytes long in total, the average rate +In this example, the file was 1,238,099 bytes long in total, the average rate of transfer for the whole file was 146.38 kilobytes per second over the 8 seconds that it took to complete, it was the 5th transfer of a regular file during the current rsync session, and there are 169 more files for the receiver to check (to see if they are up-to-date or not) remaining out of the 396 total files in the file-list. -dit(bf(-P)) The bf(-P) option is equivalent to bf(--partial) bf(--progress). Its -purpose is to make it much easier to specify these two options for a long -transfer that may be interrupted. +In an incremental recursion scan, rsync won't know the total number of files +in the file-list until it reaches the ends of the scan, but since it starts to +transfer files during the scan, it will display a line with the text "ir-chk" +(for incremental recursion check) instead of "to-chk" until the point that it +knows the full size of the list, at which point it will switch to using +"to-chk". Thus, seeing "ir-chk" lets you know that the total count of files +in the file list is still going to increase (and each time it does, the count +of files left to check will increase by the number of the files added to the +list). There is also a bf(--info=progress2) option that outputs statistics based on the whole transfer, rather than individual files. Use this flag without @@ -2161,9 +2352,14 @@ want to see how the transfer is doing without scrolling the screen with a lot of names. (You don't need to specify the bf(--progress) option in order to use bf(--info=progress2).) +dit(bf(-P)) The bf(-P) option is equivalent to bf(--partial) bf(--progress). Its +purpose is to make it much easier to specify these two options for a long +transfer that may be interrupted. + dit(bf(--password-file)) This option allows you to provide a password in a file for accessing an rsync daemon. The file must not be world readable. -It should contain just the password as a single line. +It should contain just the password as the first line of the file (all +other lines are ignored). This option does not supply a password to a remote shell transport such as ssh; to learn how to do that, consult the remote shell's documentation. @@ -2184,6 +2380,14 @@ without using this option. For example: verb( rsync -av --list-only foo* dest/) +Starting with rsync 3.1.0, the sizes output by bf(--list-only) are affected +by the bf(--human-readable) option. By default they will contain digit +separators, but higher levels of readability will output the sizes with +unit suffixes. Note also that the column width for the size output has +increased from 11 to 14 characters for all human-readable levels. Use +bf(--no-h) if you want just digits in the sizes, and the old column width +of 11 characters. + Compatibility note: when requesting a remote listing of files from an rsync that is version 2.6.3 or older, you may encounter an error if you ask for a non-recursive listing. This is because a file listing implies the bf(--dirs) @@ -2192,13 +2396,27 @@ avoid this problem, either specify the bf(--no-dirs) option (if you don't need to expand a directory's content), or turn on recursion and exclude the content of subdirectories: bf(-r --exclude='/*/*'). -dit(bf(--bwlimit=KBPS)) This option allows you to specify a maximum -transfer rate in kilobytes per second. This option is most effective when -using rsync with large files (several megabytes and up). Due to the nature -of rsync transfers, blocks of data are sent, then if rsync determines the -transfer was too fast, it will wait before sending the next data block. The -result is an average transfer rate equaling the specified limit. A value -of zero specifies no limit. +dit(bf(--bwlimit=RATE)) This option allows you to specify the maximum transfer +rate for the data sent over the socket, specified in units per second. The +RATE value can be suffixed with a string to indicate a size multiplier, and may +be a fractional value (e.g. "bf(--bwlimit=1.5m)"). If no suffix is specified, +the value will be assumed to be in units of 1024 bytes (as if "K" or "KiB" had +been appended). See the bf(--max-size) option for a description of all the +available suffixes. A value of zero specifies no limit. + +For backward-compatibility reasons, the rate limit will be rounded to the +nearest KiB unit, so no rate smaller than 1024 bytes per second is possible. + +Rsync writes data over the socket in blocks, and this option both limits the +size of the blocks that rsync writes, and tries to keep the average transfer +rate at the requested limit. Some "burstiness" may be seen where rsync writes +out a block of data and then sleeps to bring the average rate into compliance. + +Due to the internal buffering of data, the bf(--progress) option may not be an +accurate reflection on how fast the data is being sent. This is because some +files can show up as being rapidly sent when the data is quickly buffered, +while other can show up as very slow when the flushing of the output buffer +occurs. This may be fixed in a future version. dit(bf(--write-batch=FILE)) Record a file that can later be applied to another identical destination with bf(--read-batch). See the "BATCH MODE" @@ -2273,9 +2491,9 @@ If rsync was complied without support for IPv6, the bf(--ipv6) option will have no effect. The bf(--version) output will tell you if this is the case. -dit(bf(--checksum-seed=NUM)) Set the MD4 checksum seed to the integer +dit(bf(--checksum-seed=NUM)) Set the checksum seed to the integer NUM. This 4 byte checksum seed is included in each block and file -MD4 checksum calculation. By default the checksum seed is generated +checksum calculation. By default the checksum seed is generated by the server and defaults to the current code(time()). This option is used to set a specific checksum seed, which is useful for applications that want repeatable block and file checksums, or @@ -2306,11 +2524,10 @@ allows you to specify a specific IP address (or hostname) to bind to. This makes virtual hosting possible in conjunction with the bf(--config) option. See also the "address" global option in the rsyncd.conf manpage. -dit(bf(--bwlimit=KBPS)) This option allows you to specify a maximum -transfer rate in kilobytes per second for the data the daemon sends. -The client can still specify a smaller bf(--bwlimit) value, but their -requested value will be rounded down if they try to exceed it. See the -client version of this option (above) for some extra details. +dit(bf(--bwlimit=RATE)) This option allows you to specify the maximum transfer +rate for the data the daemon sends over the socket. The client can still +specify a smaller bf(--bwlimit) value, but no larger value will be allowed. +See the client version of this option (above) for some extra details. dit(bf(--config=FILE)) This specifies an alternate config file than the default. This is only relevant when bf(--daemon) is specified. @@ -2532,6 +2749,39 @@ itemization( explicitly included or it would be excluded by the "*") ) +The following modifiers are accepted after a "+" or "-": + +itemization( + it() A bf(/) specifies that the include/exclude rule should be matched + against the absolute pathname of the current item. For example, + "-/ /etc/passwd" would exclude the passwd file any time the transfer + was sending files from the "/etc" directory, and "-/ subdir/foo" + would always exclude "foo" when it is in a dir named "subdir", even + if "foo" is at the root of the current transfer. + it() A bf(!) specifies that the include/exclude should take effect if + the pattern fails to match. For instance, "-! */" would exclude all + non-directories. + it() A bf(C) is used to indicate that all the global CVS-exclude rules + should be inserted as excludes in place of the "-C". No arg should + follow. + it() An bf(s) is used to indicate that the rule applies to the sending + side. When a rule affects the sending side, it prevents files from + being transferred. The default is for a rule to affect both sides + unless bf(--delete-excluded) was specified, in which case default rules + become sender-side only. See also the hide (H) and show (S) rules, + which are an alternate way to specify sending-side includes/excludes. + it() An bf(r) is used to indicate that the rule applies to the receiving + side. When a rule affects the receiving side, it prevents files from + being deleted. See the bf(s) modifier for more info. See also the + protect (P) and risk (R) rules, which are an alternate way to + specify receiver-side includes/excludes. + it() A bf(p) indicates that a rule is perishable, meaning that it is + ignored in directories that are being deleted. For instance, the bf(-C) + option's default rules that exclude things like "CVS" and "*.o" are + marked as perishable, and will not prevent a directory that was removed + on the source from being deleted on the destination. +) + manpagesection(MERGE-FILE FILTER RULES) You can merge whole files into your filter rules by specifying either a @@ -2580,44 +2830,15 @@ itemization( "- foo + bar" is parsed as two rules (assuming that prefix-parsing wasn't also disabled). it() You may also specify any of the modifiers for the "+" or "-" rules - (below) in order to have the rules that are read in from the file - default to having that modifier set. For instance, "merge,-/ .excl" would + (above) in order to have the rules that are read in from the file + default to having that modifier set (except for the bf(!) modifier, which + would not be useful). For instance, "merge,-/ .excl" would treat the contents of .excl as absolute-path excludes, while "dir-merge,s .filt" and ":sC" would each make all their - per-directory rules apply only on the sending side. -) - -The following modifiers are accepted after a "+" or "-": - -itemization( - it() A bf(/) specifies that the include/exclude rule should be matched - against the absolute pathname of the current item. For example, - "-/ /etc/passwd" would exclude the passwd file any time the transfer - was sending files from the "/etc" directory, and "-/ subdir/foo" - would always exclude "foo" when it is in a dir named "subdir", even - if "foo" is at the root of the current transfer. - it() A bf(!) specifies that the include/exclude should take effect if - the pattern fails to match. For instance, "-! */" would exclude all - non-directories. - it() A bf(C) is used to indicate that all the global CVS-exclude rules - should be inserted as excludes in place of the "-C". No arg should - follow. - it() An bf(s) is used to indicate that the rule applies to the sending - side. When a rule affects the sending side, it prevents files from - being transferred. The default is for a rule to affect both sides - unless bf(--delete-excluded) was specified, in which case default rules - become sender-side only. See also the hide (H) and show (S) rules, - which are an alternate way to specify sending-side includes/excludes. - it() An bf(r) is used to indicate that the rule applies to the receiving - side. When a rule affects the receiving side, it prevents files from - being deleted. See the bf(s) modifier for more info. See also the - protect (P) and risk (R) rules, which are an alternate way to - specify receiver-side includes/excludes. - it() A bf(p) indicates that a rule is perishable, meaning that it is - ignored in directories that are being deleted. For instance, the bf(-C) - option's default rules that exclude things like "CVS" and "*.o" are - marked as perishable, and will not prevent a directory that was removed - on the source from being deleted on the destination. + per-directory rules apply only on the sending side. If the merge rule + specifies sides to affect (via the bf(s) or bf(r) modifier or both), + then the rules in the file must not specify sides (via a modifier or + a rule prefix such as bf(hide)). ) Per-directory rules are inherited in all subdirectories of the directory @@ -2829,27 +3050,26 @@ of the destination trees. The write-batch option causes the rsync client to store in a "batch file" all the information needed to repeat this operation against other, identical destination trees. -To apply the recorded changes to another destination tree, run rsync -with the read-batch option, specifying the name of the same batch -file, and the destination tree. Rsync updates the destination tree -using the information stored in the batch file. - -For convenience, one additional file is creating when the write-batch -option is used. This file's name is created by appending -".sh" to the batch filename. The .sh file contains -a command-line suitable for updating a destination tree using that -batch file. It can be executed using a Bourne (or Bourne-like) shell, -optionally -passing in an alternate destination tree pathname which is then used -instead of the original path. This is useful when the destination tree -path differs from the original destination tree path. - Generating the batch file once saves having to perform the file status, checksum, and data block generation more than once when updating multiple destination trees. Multicast transport protocols can be used to transfer the batch update files in parallel to many hosts at once, instead of sending the same data to every host individually. +To apply the recorded changes to another destination tree, run rsync +with the read-batch option, specifying the name of the same batch +file, and the destination tree. Rsync updates the destination tree +using the information stored in the batch file. + +For your convenience, a script file is also created when the write-batch +option is used: it will be named the same as the batch file with ".sh" +appended. This script file contains a command-line suitable for updating a +destination tree using the associated batch file. It can be executed using +a Bourne (or Bourne-like) shell, optionally passing in an alternate +destination tree pathname which is then used instead of the original +destination path. This is useful when the destination tree path on the +current host differs from the one used to create the batch file. + Examples: quote( @@ -2941,9 +3161,9 @@ bf(--links). If bf(--copy-links) is specified, then symlinks are "collapsed" by copying their referent, rather than the symlink. -rsync also distinguishes "safe" and "unsafe" symbolic links. An -example where this might be used is a web site mirror that wishes -ensure the rsync module they copy does not include symbolic links to +Rsync can also distinguish "safe" and "unsafe" symbolic links. An +example where this might be used is a web site mirror that wishes to +ensure that the rsync module that is copied does not include symbolic links to bf(/etc/passwd) in the public section of the site. Using bf(--copy-unsafe-links) will cause any links to be copied as the file they point to on the destination. Using bf(--safe-links) will cause @@ -3032,7 +3252,10 @@ dit(bf(CVSIGNORE)) The CVSIGNORE environment variable supplements any ignore patterns in .cvsignore files. See the bf(--cvs-exclude) option for more details. dit(bf(RSYNC_ICONV)) Specify a default bf(--iconv) setting using this -environment variable. +environment variable. (First supported in 3.0.0.) +dit(bf(RSYNC_PROTECT_ARGS)) Specify a non-zero numeric value if you want the +bf(--protect-args) option to be enabled by default, or a zero value to make +sure that it is disabled by default. (First supported in 3.1.0.) dit(bf(RSYNC_RSH)) The RSYNC_RSH environment variable allows you to override the default shell used as the transport for rsync. Command line options are permitted after the command name, just as in the bf(-e) option. @@ -3110,7 +3333,7 @@ Jean-loup Gailly and Mark Adler. manpagesection(THANKS) -Especial thanks go out to: John Van Essen, Matt McCutchen, Wesley W. Terpstra, +Special thanks go out to: John Van Essen, Matt McCutchen, Wesley W. Terpstra, David Dykstra, Jos Backus, Sebastian Krahmer, Martin Pool, and our gone-but-not-forgotten compadre, J.W. Schultz.