Make idev, hlink and file_struct + strings use allocation

[rsync/rsync.git] / NEWS

diff --git a/NEWS b/NEWS
index 526ec27..82f734b 100644 (file)
--- a/NEWS
+++ b/NEWS
@@ -1,26 +1,160 @@
-NEWS for rsync version 2.5.7
+NEWS for rsync 2.6.1 (UNRELEASED)
+Protocol: 27 (unchanged)
+Changes since 2.6.0:
+
+  ENHANCEMENTS:
+
+    * Lower memory use and more optimal transfer of data over
+      the socket (see the INTERNAL section for details).
+
+    * The RSYNC_PROXY environment variable can now contain a
+      "USER:PASS@" prefix before the "HOST:PORT" information.
+      (Bardur Arantsson)
+
+    * The --progress output now mentions how far along in the
+      transfer we are, including both a count of files transferred
+      and a percentage of the total file-count that we're processed.
+
+  BUG FIXES:
+
+    * When -x (--one-file-system) is combined with -L (--copy-links)
+      or --copy-unsafe-links, no symlinked files are skipped, even
+      if the referant file is on a different filesystem.
+
+    * The --link-dest code now works properly for a non-root user
+      when the UIDs of the source and destination differ and -o was
+      specified, and when the group of the source can't be used on
+      the destination and -g was specified.  (Wayne Davison)
+
+    * Fixed a bug in the handling of -H (hard-links) that might
+      cause the expanded PATH/NAME value of the current item to
+      get overwritten (due to an expanded-name caching bug).
+      (Wayne Davison)
+      
+    * We now reset the "new data has been sent" flag at the start
+      of each file we send.  This makes sure that an interrupted
+      transfer with the --partial option set doesn't keep a shorter
+      temp file than the current basis file when no new data has been
+      transfered over the wire for that file.  (Wayne Davison)
+
+    * Fixed a byte-order problem in --batch-mode on big-endian
+      machines.  (Jay Fenlason)
+
+    * Fixed configure bug when running "./configure --disable-ipv6".
+      (Wayne Davison)
+
+    * Fixed "make test" bug when build dir is not the source dir.
+
+    * When using --cvs-exclude, the exclude items we get from a
+      directory's .cvsignore file once again only affect that one
+      directory (and not all following directories too).
+
+    * When transferring a file that has group 0 with -g specified
+      (typically via -a) and not enough privs to retain the group,
+      rsync no longer complains about "chown" failing.
+
+    * When specifying the USER@HOST: prefix for a file, the USER
+      part can now contain an '@', if needed (i.e. the last '@'
+      is used to find the HOST, not the first).
+
+    * Fixed some bugs in the handling of group IDs for non-root
+      users:  (1) It properly handles a group that the sender didn't
+      have a name for (it would previously skip changing the group
+      on any files in that group).  (2) If --numeric-ids is used,
+      rsync no longer attempts to set groups that the user doesn't
+      have the permission to set.
+
+  INTERNAL:
+
+    * Most of the I/O is now buffered, which results in a pretty
+      large speedup when running under MS Windows.  (Craig Barratt)
+
+    * Optimizations to the name-handling/comparing code have made
+      some significant reductions in user-CPU time for large file
+      sets.  (Wayne Davison)
+
+    * Some variable-type cleanup that makes the code more consistent.
+      (Wayne Davison)
+
+    * Reduced memory requirements of hard link preservation.
+      (J.W. Schultz)
+
+    * Implemented a new algorithm for hard-link handling that speeds
+      up the code significantly.  (J.W. Schultz and Wayne Davison)
+
+    * The --hard-link option now uses the first existing file in the
+      group of linked files as the basis for the transfer.  This
+      prevents the sub-optimal transfer of a file's data when a new
+      hardlink is added on the sending side and it sorts alphabetically
+      earlier in the list than the files that are already present on the
+      receiving side.
+
+    * Got rid of support for protocol versions 17 and 18 (which are
+      both over 6 years old).  (Wayne Davison)
+
+    * More optimal data transmission for --hard-links (protocol 28).
+
+    * More optimal data transmission for --checksum (protocol 28).
+
+    * Less memory is used when --checksum is specified.
+
+    * Less memory is used in the file list (a per-file savings).
+
+    * Changed hardlink info and file_struct + strings to use
+      allocation pools.  This reduces memory use for large
+      filesets and permits freeing memory to the OS.  (J.W. Schultz) 
+
+    * The 2 pipes used between the receiver and generator processes
+      (which are forked on the same machine) were reduced to 1 pipe
+      and the protocol improved so that (1) it is now impossible to
+      have the "redo" pipe fill up and hang rsync, and (2) trailing
+      messages from the receiver don't get lost on their way through
+      the generator over to the sender (the latter mainly affected
+      hard-link messages).  (Wayne Davison)
+
+    * The reading & writing of the file list in batch-mode is now
+      handled by the same code that sends & receives the list over
+      the wire.  This makes it much easier to maintain.
+
+    * Optimized the -x (--one-file-system) flag's handling of any
+      mount-point directories we encounter (it no longer scans the
+      contents of the mount-point dirs, just to throw away the data).
+
+    * Improved the internal uid/gid code to be more portable and
+      a little more optimized.
+
+\f
+NEWS for rsync 2.6.0 (1 Jan 2004)

 Protocol: 27 (changed)
 Protocol: 27 (changed)

-Changes since version 2.5.6:
+Changes since 2.5.7:

 
   ENHANCEMENTS:
 
 
   ENHANCEMENTS:
 

+    * "ssh" is now the default remote shell for rsync.  If you want to
+      change this, configure like this:  "./configure --with-rsh=rsh".
+

     * Added --files-from, --no-relative, --no-implied-dirs, and --from0.
       Note that --from0 affects the line-ending character for all the
     * Added --files-from, --no-relative, --no-implied-dirs, and --from0.
       Note that --from0 affects the line-ending character for all the

-      --*-from options. (Wayne Davison)
+      files read by the --*-from options. (Wayne Davison)

 
 

-    * Length of csum2 is now per-file starting with protocol verison
+    * Length of csum2 is now per-file starting with protocol version

       27. (J.W. Schultz)
 
       27. (J.W. Schultz)
 

-    * Per-file dynamic block size is now sqrt(file length).
-      The per-file checksum size is determined according
-      to an algorythm provided by Donovan Baarda which
-      reduces the probability of rsync algorithm
-      corrupting data and falling back using the whole md4
+    * Per-file dynamic block size is now sqrt(file length).  The
+      per-file checksum size is determined according to an algorithm
+      provided by Donovan Baarda which reduces the probability of rsync
+      algorithm corrupting data and falling back using the whole md4

       checksums. (J.W. Schultz, Donovan Baarda)
 
     * The --stats option no longer includes the (debug) malloc summary
       unless the verbose option was specified at least twice.
 
       checksums. (J.W. Schultz, Donovan Baarda)
 
     * The --stats option no longer includes the (debug) malloc summary
       unless the verbose option was specified at least twice.
 

+    * Added a new error/warning code for when files vanish from the
+      sending side.  Made vanished source files not interfere with the
+      file-deletion pass when --delete-after was specified.
+
+    * Various trailing-info sections are now preceded by a newline.
+

   BUG FIXES:
 
     * Fixed several exclude/include matching bugs when using wild-cards.
   BUG FIXES:
 
     * Fixed several exclude/include matching bugs when using wild-cards.


@@ -29,27 +163,36 @@ Changes since version 2.5.6:
       cause anyone problems since it makes the matching work more like
       what people are expecting. (Wayne Davison)
 
       cause anyone problems since it makes the matching work more like
       what people are expecting. (Wayne Davison)
 

-      - "**/foo" now matches at the base of the transfer (i.e. /foo).
+      - A pattern with a "**" no longer causes a "*" to match slashes.
+        For example, with "/*/foo/**", "foo" must be 2 levels deep.
+       [If your string has BOTH "*" and "**" wildcards, changing the
+       "*" wildcards to "**" will provide the old behavior in all
+       versions.]

 
 

-      - An non-anchored wildcard term floats to match beyond the base of
+      - "**/foo" now matches at the base of the transfer (like /foo
+        does).  [Use "/**/foo" to get the old behavior in all versions.]
+
+      - A non-anchored wildcard term floats to match beyond the base of

         the transfer.  E.g. "CVS/R*" matches at the end of the path,
         the transfer.  E.g. "CVS/R*" matches at the end of the path,

-        just like the non-wildcard term "CVS/Root" does.
+        just like the non-wildcard term "CVS/Root" does. [Use "/CVS/R*"
+        to get the old behavior in all versions.]

 
       - Including a "**" in the match term causes it to be matched
         against the entire path, not just the name portion, even if
         there aren't any interior slashes in the term.  E.g. "foo**bar"
         would exclude "/path/foo-bar" (just like before) as well as
 
       - Including a "**" in the match term causes it to be matched
         against the entire path, not just the name portion, even if
         there aren't any interior slashes in the term.  E.g. "foo**bar"
         would exclude "/path/foo-bar" (just like before) as well as

-        "/foo-path/baz-bar" (for instance).
+        "/foo-path/baz-bar" (unlike before).  [Use "foo*bar" to get the
+        old behavior in all versions.]

 
     * The exclude list specified in the daemon's config file is now
       properly applied to the pulled items no matter how deep the
 
     * The exclude list specified in the daemon's config file is now
       properly applied to the pulled items no matter how deep the

-      user's file args are in the source tree.  (Wayne Davison)
+      user's file-args are in the source tree.  (Wayne Davison)

 
     * For protocol version >= 27, mdfour_tail() is called when the
       block size (including checksum_seed) is a multiple of 64.
       Previously it was not called, giving the wrong MD4 checksum.
       (Craig Barratt)
 
     * For protocol version >= 27, mdfour_tail() is called when the
       block size (including checksum_seed) is a multiple of 64.
       Previously it was not called, giving the wrong MD4 checksum.
       (Craig Barratt)

-  
+

     * For protocol version >= 27, a 64 bit bit counter is used in
       mdfour.c as required by the RFC.  Previously only a 32 bit bit
       counter was used, causing incorrect MD4 file checksums for
     * For protocol version >= 27, a 64 bit bit counter is used in
       mdfour.c as required by the RFC.  Previously only a 32 bit bit
       counter was used, causing incorrect MD4 file checksums for


@@ -68,110 +211,56 @@ Changes since version 2.5.6:
 
     * Fixed file I/O error detection.  (John Van Essen)
 
 
     * Fixed file I/O error detection.  (John Van Essen)
 

+    * Fixed bogus "malformed address {hostname}" message in rsyncd log
+      when checking IP address against hostnames from "hosts allow"
+      and "hosts deny" parameters in config file.
+
+    * Print heap statistics when verbose >= 2 instead of when >= 1.
+

     * Fixed a compression (-z) bug when syncing a mostly-matching file
       that contains already-compressed data.  (Yasuoka Masahiko and
       Wayne Davison)
 
     * Fixed a compression (-z) bug when syncing a mostly-matching file
       that contains already-compressed data.  (Yasuoka Masahiko and
       Wayne Davison)
 

-  INTERNAL:
-
-    * Eliminated vestigial support for old versions that we stopped
-      supporting. (J.W. Schultz)
-
-    * Simplified some of the option-parsing code. (Wayne Davison)
-
-    * Some cleanup made to the exclude code, as well as some new
-      defines added to enhance readability. (Wayne Davison)
-
-
-NEWS for rsync version 2.5.6, aka the dwd-between-jobs release
-Protocol: 26 (unchanged)
-Changes since version 2.5.5:
-
-  ENHANCEMENTS:
+    * Fixed a bug in the --backup code that could cause deleted files
+      to not get backed up.

 
 

-    * The --delete-after option now implies --delete.  (Wayne Davison)
+    * When the backup code makes new directories, create them with mode
+      0700 instead of 0755 (since the directory permissions in the
+      backup tree are not yet copied from the main tree).

 
 

-    * The --suffix option can now be used with --backup-dir.  (Michael
-      Zimmerman)
-
-    * Combining "::" syntax with the -rsh/-e option now uses the
-      specified remote-shell as a transport to talk to a (newly-spawned)
-      server-daemon.  This allows someone to use daemon features, such
-      as modules, over a secure protocol, such as ssh.  (JD Paul)
-
-    * The rsync:// syntax for daemon connections is now accepted in the
-      destination field.
-
-    * If the file name given to --include-from or --exclude-from is "-",
-      rsync will read from standard input.  (J.W. Schultz)
-
-    * New option --link-dest which is like --compare-dest except that
-      unchanged files are hard-linked in to the destination directory.
-      (J.W. Schultz)
+    * Call setgroups() in a more portable manner.

 
 

-    * Don't report an error if an excluded file disappears during an
-      rsync run.  (Eugene Chupriyanov and Bo Kersey)
+    * Improved file-related error messages to better indicate exactly
+      what pathname failed. (Wayne Davison)

 
 

-    * Added .svn to --cvs-exclude list to support subversion.  (Jon
-      Middleton)
+    * Fixed some bugs in the handling of --delete and --exclude when
+      using the --relative (-R) option. (Wayne Davison)

 
 

-    * Properly support IPv6 addresses in the rsyncd.conf "hosts allow"
-      and "hosts deny" fields.  (Hideaki Yoshifuji)
+    * Fixed bug that prevented regular files from replacing
+      special files and caused a directory in --link-dest or
+      --compare-dest to block the creation of a file with the
+      same path.  A directory still cannot be replaced by a
+      regular file unless --delete specified.  (J.W. Schultz)

 
 

-    * Changed exclude file handling to permit DOS or MAC style line
-      terminations.  (J.W. Schultz)
+    * Detect and report when open or opendir succeed but read and
+      readdir fail caused by network filesystem issues and truncated
+      files.  (David Norwood, Michael Brown, J.W. Schultz)

 
 

-    * Ignore errors from chmod when -p/-a/--preserve-perms is not set.
-      (Dave Dykstra)
-
-  BUG FIXES:
-
-    * Fix "forward name lookup failed" errors on AIX 4.3.3.  (John
-      L. Allen, Martin Pool)
-
-    * Generate each file's rolling-checksum data as we send it, not
-      in a separate (memory-eating) pass before hand.  This prevents
-      timeout errors on really large files. (Stefan Nehlsen)
-
-    * Fix compilation on Tru64.  (Albert Chin, Zoong Pham)
-
-    * Better handling of some client-server errors.  (Martin Pool)
-
-    * Fixed a crash that would occur when sending a list of files that
-      contains a duplicate name (if it sorts to the end of the file
-      list) and using --delete.  (Wayne Davison)
-
-    * Fixed the file-name duplicate-removal code when dealing with multiple
-      dups in a row. (Wayne Davison)
-
-    * Fixed a bug that caused rsync to lose the exit status of its child
-      processes and sometimes return an exit code of 0 instead of showing
-      an error.  (David R. Staples, Dave Dykstra)
-
-    * Fixed bug in --copy-unsafe-links that caused it to be completely
-      broken.  (Dave Dykstra)
-
-    * Prevent infinite recursion in cleanup code under certain circumstances.
-      (Sviatoslav Sviridov and Marc Espie)
-
-    * Fixed a bug that prevented rsync from creating intervening directories
-      when --relative-paths/-R is set.  (Craig Barratt)
-
-    * Prevent "Connection reset by peer" messages from Cygwin. (Randy O'Meara)
+    * Added a fix that should give ssh time to restore the tty settings
+      if the user presses Ctrl-C at an ssh password prompt.

 
   INTERNAL:
 
 
   INTERNAL:
 

-    * Many code cleanups and improved internal documentation.  (Martin
-      Pool, Nelson Beebe)
-
-    * Portability fixes. (Dave Dykstra and Wayne Davison)
-
-    * More test cases.  (Martin Pool)
+    * Eliminated vestigial support for old versions that we stopped
+      supporting. (J.W. Schultz)

 
 

-    * Some test-case fixes.  (Brian Poole, Wayne Davison)
+    * Simplified some of the option-parsing code. (Wayne Davison)

 
 

-    * Updated included popt to the latest vendor drop, version 1.6.4.
-      (Jos Backus)
+    * Some cleanup made to the exclude code, as well as some new
+      defines added to enhance readability. (Wayne Davison)

 
 

-    * Updated config.guess and config.sub to latest versions; this
-      means rsync should build on more platforms.  (Paul Green)
+    * Changed the protocol-version code so that it can interact at a
+      lower protocol level than the maximum supported by both sides.
+      Added an undocumented option, --protocol=N, to force the value
+      we advertise to the other side (primarily for testing purposes).
+      (Wayne Davison)