-*- indented-text -*-
-URGENT ---------------------------------------------------------------
+BUGS ---------------------------------------------------------------
+
+There seems to be a bug with hardlinks
+
+ mbp/2 build$ ls -l /tmp/a /tmp/b -i
+ /tmp/a:
+ total 32
+ 2568307 -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a1
+ 2568307 -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a2
+ 2568307 -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a3
+ 2568310 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 a4
+ 2568310 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 a5
+ 2568310 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b1
+ 2568310 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b2
+ 2568310 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b3
+
+ /tmp/b:
+ total 32
+ 2568309 -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a1
+ 2568309 -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a2
+ 2568309 -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a3
+ 2568311 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 a4
+ 2568311 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 a5
+ 2568311 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b1
+ 2568311 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b2
+ 2568311 -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b3
+ mbp/2 build$ rm -r /tmp/b && ./rsync -avH /tmp/a/ /tmp/b
+ building file list ... done
+ created directory /tmp/b
+ ./
+ a1
+ a4
+ a2 => a1
+ a3 => a2
+ wrote 350 bytes read 52 bytes 804.00 bytes/sec
+ total size is 232 speedup is 0.58
+ mbp/2 build$ rm -r /tmp/b
+ mbp/2 build$ ls -l /tmp/b
+ ls: /tmp/b: No such file or directory
+ mbp/2 build$ rm -r /tmp/b && ./rsync -avH /tmp/a/ /tmp/b
+ rm: cannot remove `/tmp/b': No such file or directory
+ mbp/2 build$ rm -f -r /tmp/b && ./rsync -avH /tmp/a/ /tmp/b
+ building file list ... done
+ created directory /tmp/b
+ ./
+ a1
+ a4
+ a2 => a1
+ a3 => a2
+ wrote 350 bytes read 52 bytes 804.00 bytes/sec
+ total size is 232 speedup is 0.58
+ mbp/2 build$ ls -l /tmp/b
+ total 32
+ -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a1
+ -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a2
+ -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a3
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 a4
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 a5
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b1
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b2
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b3
+ mbp/2 build$ ls -l /tmp/a
+ total 32
+ -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a1
+ -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a2
+ -rw-rw-r-- 3 mbp mbp 29 Mar 25 17:30 a3
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 a4
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 a5
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b1
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b2
+ -rw-rw-r-- 5 mbp mbp 29 Mar 25 17:30 b3
+
+
+Progress indicator can produce corrupt output when transferring directories:
+
+ main/binary-arm/
+ main/binary-arm/admin/
+ main/binary-arm/base/
+ main/binary-arm/comm/8.56kB/s 0:00:52
+ main/binary-arm/devel/
+ main/binary-arm/doc/
+ main/binary-arm/editors/
+ main/binary-arm/electronics/s 0:00:53
+ main/binary-arm/games/
+ main/binary-arm/graphics/
+ main/binary-arm/hamradio/
+ main/binary-arm/interpreters/
+ main/binary-arm/libs/6.61kB/s 0:00:54
+ main/binary-arm/mail/
+ main/binary-arm/math/
+ main/binary-arm/misc/
+
+lchmod
+
+ I don't think we handle this properly on systems that don't have the
+ call.
+
+
+DAEMON --------------------------------------------------------------
+
+server-imposed bandwidth limits
+rsyncd over ssh
-IMPORTANT ------------------------------------------------------------
+ There are already some patches to do this.
-Cross-test versions
+ BitKeeper uses a server whose login shell is set to bkd. That's
+ probably a reasonable approach.
- Part of the regression suite should be making sure that we don't
- break backwards compatibility: old clients vs new servers and so
- on. Ideally we would test the cross product of versions.
- It might be sufficient to test downloads from well-known public
- rsync servers running different versions of rsync. This will give
- some testing and also be the most common case for having different
- versions and not being able to upgrade.
+FEATURES ------------------------------------------------------------
+
+
+--dry-run is insufficiently dry
+
+ Mark Santcroos points out that -n fails to list files which have
+ only metadata changes, though it probably should.
+
+ There may be a Debian bug about this as well.
+
use chroot
http://lists.samba.org/pipermail/rsync/2001-August/thread.html
http://lists.samba.org/pipermail/rsync/2001-September/thread.html
+
--files-from
Avoids traversal. Better option than a pile of --include statements
for people who want to generate the file list using a find(1)
command or a script.
+
+supplementary groups
+
+ Perhaps allow supplementary groups to be specified in rsyncd.conf;
+ then make the first one the primary gid and all the rest be
+ supplementary gids.
+
+
File list structure in memory
Rather than one big array, perhaps have a tree in memory mirroring
which should just take a small change to the parser code.
+
Errors
If we hang or get SIGINT, then explain where we were up to. Perhaps
eof" give a message that is more detailed if possible and also more
helpful.
+ If we get an error writing to a socket, then we should perhaps
+ continue trying to read to see if an error message comes across
+ explaining why the socket is closed. I'm not sure if this would
+ work, but it would certainly make our messages more helpful.
+
+ What happens if a directory is missing -x attributes. Do we lose
+ our load? (Debian #28416) Probably fixed now, but a test case
+ would be good.
+
+
File attributes
Device major/minor numbers should be at least 32 bits each. See
do this is to just disable gzip (with a warning) when talking to old
versions.
+ After we get the @RSYNCD greeting from the server, we know it's
+ version but we have not yet sent the command line, so we could just
+ remove the -z option if the server is too old.
-logging
+ For ssh invocation it's not so simple, because we actually use the
+ command line to start the remote process. However, we only actually
+ do compression in token.c, and we could therefore once we discover
+ the remote version emit an error if it's too old. I'm not sure if
+ that's a good tradeoff or not.
- Perhaps flush stdout after each filename, so that people trying to
- monitor progress in a log file can do so more easily. See
- http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=48108
-
- At the connections that just get a list of modules are not logged,
- but they should be.
-
-rsyncd over ssh
-
- There are already some patches to do this.
proxy authentication
<mbp> it would be nice to improve it
<mbp> that would also work well with --dryrun
+TDB:
+
+ Rather than storing the file list in memory, store it in a TDB.
+
+ This *might* make memory usage lower while building the file list.
+
+ Hashtable lookup will mean files are not transmitted in order,
+ though... hm.
+
+ This would neatly eliminate one of the major post-fork shared data
+ structures.
+
+
+chmod:
+
+ On 12 Mar 2002, Dave Dykstra <dwd@bell-labs.com> wrote:
+ > If we would add an option to do that functionality, I would vote for one
+ > that was more general which could mask off any set of permission bits and
+ > possibly add any set of bits. Perhaps a chmod-like syntax if it could be
+ > implemented simply.
+
+ I think that would be good too. For example, people uploading files
+ to a web server might like to say
+
+ rsync -avzP --chmod a+rX ./ sourcefrog.net:/home/www/sourcefrog/
+
+ Ideally the patch would implement as many of the gnu chmod semantics
+ as possible. I think the mode parser should be a separate function
+ that passes back something like (mask,set) description to the rest of
+ the program. For bonus points there would be a test case for the
+ parser.
+
+ (Debian #23628)
+
+
+--diff
+
+ Allow people to specify the diff command. (Might want to use wdiff,
+ gnudiff, etc.)
+
+ Just diff the temporary file with the destination file, and delete
+ the tmp file rather than moving it into place.
+
+ Interaction with --partial.
+
+ Security interactions with daemon mode?
+
+ (Suggestion from david.e.sewell)
+
+
+Incorrect timestamps (Debian #100295)
+
+ A bit hard to believe, but apparently it happens.
+
+
+Check "refuse options works"
+
+ We need a test case for this...
+
+ Was this broken when we changed to popt?
+
+
+PERFORMANCE ----------------------------------------------------------
+
+MD4 file_sum
+
+ If we're doing a local transfer, or using -W, then perhaps don't
+ send the file checksum. If we're doing a local transfer, then
+ calculating MD4 checksums uses 90% of CPU and is unlikely to be
+ useful.
+
+ Indeed for transfers over zlib or ssh we can also rely on the
+ transport to have quite strong protection against corruption.
+
+ Perhaps we should have an option to disable this, analogous to
+ --whole-file, although it would default to disabled. The file
+ checksum takes up a definite space in the protocol -- we can either
+ set it to 0, or perhaps just leave it out.
+
+MD4
+
+ Perhaps borrow an assembler MD4 from someone?
+
+ Make sure we call MD4 with properly-sized blocks whenever possible
+ to avoid copying into the residue region?
+
+String area code
+
+ Test whether this is actually faster than just using malloc(). If
+ it's not (anymore), throw it out.
+
+
+
PLATFORMS ------------------------------------------------------------
Win32
we are correct to call close(), because shutdown() discards
untransmitted data.
+DEVELOPMENT ----------------------------------------------------------
+
+Splint
+
+ Build rsync with SPLINT to try to find security holes. Add
+ annotations as necessary. Keep track of the number of warnings
+ found initially, and see how many of them are real bugs, or real
+ security bugs. Knowing the percentage of likely hits would be
+ really interesting for other projects.
+
+Torture test
+
+ Something that just keeps running rsync continuously over a data set
+ likely to generate problems.
+
+Cross-testing
+
+ Run current rsync versions against significant past releases.
+
+Memory debugger
+
+ jra recommends Valgrind:
+
+ http://devel-home.kde.org/~sewardj/
+
+Release script
+
+ Update spec files
+
+ Build tar file; upload
+
+ Send announcement to mailing list and c.o.l.a.
+
+ Make freshmeat announcement
+
+ Update web site
+
+
+
+TESTING --------------------------------------------------------------
+
+Cross-test versions
+
+ Part of the regression suite should be making sure that we don't
+ break backwards compatibility: old clients vs new servers and so
+ on. Ideally we would test both up and down from the current release
+ to all old versions.
+
+ We might need to omit broken old versions, or versions in which
+ particular functionality is broken
+
+ It might be sufficient to test downloads from well-known public
+ rsync servers running different versions of rsync. This will give
+ some testing and also be the most common case for having different
+ versions and not being able to upgrade.
+
+
+Test on kernel source
+
+ Download all versions of kernel; unpack, sync between them. Also
+ sync between uncompressed tarballs. Compare directories after
+ transfer.
+
+ Use local mode; ssh; daemon; --whole-file and --no-whole-file.
+
+ Use awk to pull out the 'speedup' number for each transfer. Make
+ sure it is >= x.
+
+
+Test large files
+
+ Sparse and non-sparse
+
+Mutator program
+
+ Insert bytes, delete bytes, swap blocks, ...
+
+configure option to enable dangerous tests
+
+If tests are skipped, say why.
+
+Test daemon feature to disallow particular options.
+
+Pipe program that makes slow/jerky connections.
+
+Versions of read() and write() that corrupt the stream, or abruptly fail
+
+Separate makefile target to run rough tests -- or perhaps just run
+them every time?
+
+
DOCUMENTATION --------------------------------------------------------
Update README
+Keep list of open issues and todos on the web site
+
+Update web site from CVS
+
+
+Perhaps redo manual as SGML
+
+ The man page is getting rather large, and there is more information
+ that ought to be added.
+
+ TexInfo source is probably a dying format.
+
+ Linuxdoc looks like the most likely contender. I know DocBook is
+ favoured by some people, but it's so bloody verbose, even with emacs
+ support.
+
+
BUILD FARM -----------------------------------------------------------
Add machines
SCO
-NICE -----------------------------------------------------------------
---no-detach and --no-fork options
+LOGGING --------------------------------------------------------------
- Very useful for debugging. Also good when running under a
- daemon-monitoring process that tries to restart the service when the
- parent exits.
+ Perhaps flush stdout after each filename, so that people trying to
+ monitor progress in a log file can do so more easily. See
+ http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=48108
-hang/timeout friendliness
+ At the connections that just get a list of modules are not logged,
+ but they should be.
+
+ If a child of the rsync daemon dies with a signal, we should notice
+ that when we reap it and log a message.
+
+ Keep stderr and stdout properly separated (Debian #23626)
+
+ Use a separate function for reporting errors; prefix it with
+ "rsync:" or "rsync(remote)", or perhaps even "rsync(local
+ generator): ".
verbose output
At end of transfer, show how many files were or were not transferred
correctly.
+-vv
+
+ Explain *why* every file is transferred or not (e.g. "local mtime
+ 123123 newer than 1283198")
+
+
+debugging of daemon
+
+ Add an rsyncd.conf parameter to turn on debugging on the server.
+
+
+
+NICE -----------------------------------------------------------------
+
+--no-detach and --no-fork options
+
+ Very useful for debugging. Also good when running under a
+ daemon-monitoring process that tries to restart the service when the
+ parent exits.
+
+hang/timeout friendliness
+
internationalization
Change to using gettext(). Probably need to ship this for platforms
Solicit translations.
- Does anyone care?
+ Does anyone care? Before we bother modifying the code, we ought to
+ get the manual translated first, because that's possibly more useful
+ and at any rate demonstrates desire.
+
rsyncsh
fairly directly into rsync commands: it just needs to remember the
current host, directory and so on. We can probably even do
completion of remote filenames.
+
+
+RELATED PROJECTS -----------------------------------------------------
+
+http://rsync.samba.org/rsync-and-debian/
+
+rsyncable gzip patch
+
+ Exhaustive, tortuous testing
+
+ Cleanups?
+
+rsyncsplit as alternative to real integration with gzip?
+
+reverse rsync over HTTP Range
+
+ Goswin Brederlow suggested this on Debian; I think tridge and I
+ talked about it previous in relation to rproxy.