X-Git-Url: https://mattmccutchen.net/rsync/rsync.git/blobdiff_plain/c10b0bdd50af38eedd4f06abb417babd05d143c3..58379559cc969196f702a3ec99195ace95c4b3d5:/TODO

diff --git a/TODO b/TODO
index 8b286130..cff16496 100644
--- a/TODO
+++ b/TODO
@@ -35,9 +35,99 @@ use chroot
 Performance
 
   Traverse just one directory at a time.  Tridge says it's possible.
- 
-  Can possibly also be smarter about memory use while looking for hard
-  links by reducing the refcount as we find alternative names.
+
+  At the moment rsync reads the whole file list into memory at the
+  start, which makes us use a lot of memory and also not pipeline
+  network access as much as we could.
+
+  We need to be careful of duplicate names getting into the file list.
+  See clean_flist.  This could happen if multiple arguments include
+  the same file.  Bad.  
+
+  I think duplicates are only a problem if they're both flowing
+  through the pipeline at the same time.  For example we might have
+  updated the first occurrence after reading the checksums for the
+  second.  So possibly we just need to make sure that we don't have
+  both in the pipeline at the same time.  
+
+  Possibly if we did one directory at a time that would be sufficient.
+
+  Alternatively we could pre-process the arguments to make sure no
+  duplicates will ever be inserted.  There could be some bad cases
+  when we're collapsing symlinks.
+
+  We could have a hash table.
+
+Memory accounting
+
+  At exit, show how much memory was used for the file list, etc.
+
+  Also we do a wierd exponential-growth allocation in flist.c.  I'm
+  not sure this makes sense with modern mallocs.  At any rate it will
+  make us allocate a huge amount of memory for large file lists.
+
+Hard-link handling
+
+  At the moment hardlink handling is very expensive, so it's off by
+  default.  It does not need to be so.  
+
+  We can rule out hardlinked directories since they will probably
+  screw us up in all kinds of ways.  They simply should not be used.
+
+  At the moment rsync only cares about hardlinks to regular files.  I
+  guess you could also use them for sockets, devices and other beasts,
+  but I have not seen them.
+
+  When trying to reproduce hard links, we only need to worry about
+  files that have more than one name (nlinks>1 && !S_ISDIR). 
+
+  The basic point of this is to discover alternate names that refer to
+  the same file.  All operations, including creating the file and
+  writing modifications to it need only to be done for the first name.
+  For all later names, we just create the link and then leave it
+  alone.
+
+  If hard links are to be preserved:
+
+    Before the generator/receiver fork, the list of files is received
+    from the sender (recv_file_list), and a table for detecting hard
+    links is built.
+
+    The generator looks for hard links within the file list and does
+    not send checksums for them, though it does send other metadata.
+
+    The sender sends the device number and inode with file entries, so
+    that files are uniquely identified.
+
+    The receiver goes through and creates hard links (do_hard_links)
+    after all data has been written, but before directory permissions
+    are set.
+
+  At the moment device and inum are sent as 4-byte integers, which
+  will probably cause problems on large filesystems.  On Linux the
+  kernel uses 64-bit ino_t's internally, and people will soon have
+  filesystems big enough to use them.  We ought to follow NFS4 in
+  using 64-bit device and inode identification, perhaps with a
+  protocol version bump.
+
+  Once we've seen all the names for a particular file, we no longer
+  need to think about it and we can deallocate the memory.
+
+  We can also have the case where there are links to a file that are
+  not in the tree being transferred.  There's nothing we can do about
+  that.  Because we rename the destination into place after writing,
+  any hardlinks to the old file are always going to be orphaned.  In
+  fact that is almost necessary because otherwise we'd get really
+  confused if we were generating checksums for one name of a file and
+  modifying another.
+
+  At the moment the code seems to make a whole second copy of the file
+  list, which seems unnecessary.
+
+  We should have a test case that exercises hard links.  Since it
+  might be hard to compare ./tls output where the inodes change we
+  might need a little program to check whether several names refer to
+  the same file.
 
 IPv6
 
@@ -98,6 +188,10 @@ logging
   monitor progress in a log file can do so more easily.  See
   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=48108
 
+rsyncd over ssh
+
+  There are already some patches to do this.
+
 PLATFORMS ------------------------------------------------------------
 
 Win32
@@ -113,6 +207,10 @@ Win32
   we are correct to call close(), because shutdown() discards
   untransmitted data.
 
+DOCUMENTATION --------------------------------------------------------
+
+Update README
+
 BUILD FARM -----------------------------------------------------------
 
 Add machines
@@ -139,7 +237,9 @@ SIGHUP
 
 hang/timeout friendliness
 
-  On 
+verbose output
+  
+  Indicate whether files are new, updated, or deleted
 
 internationalization
 
@@ -158,3 +258,4 @@ rsyncsh
    current host, directory and so on.  We can probably even do
    completion of remote filenames.
 
+%K%