Utilities

Here I collect Linux utilities that I have written that you might find useful.

cp2

Abbreviation for rsync -rltE --chmod=ugo=rwX, i.e., preserve data but use destination default security settings. cp2 has replaced cp as my copy command.

setexec

setexec makes a file executable or nonexecutable by changing the appropriate x permissions according to the same smart algorithm I contributed to rsync. cleanexec is a wrapper that makes a file executable if and only if file(1) says the data is executable; you might find find . -type f -exec cleanexec {} \; useful.

stat2

Shows just about all information about a file except its data.

retex

A TeX wrapper script that makes TeX compiling fit more nicely into build processes.  For example, it exits nonzero immediately if an error occurs, and it repeats until a fixed point is reached in order to handle LaTeX references correctly.

ftc, ftx

ftc packages a file tree in a single file of a simplistic format that I designed, and ftx extracts such package files.  They have the same purposes as tar -c and tar -x respectively but have no bells or whistles.  They handle binary files safely, but a package of only text files is itself a text file.

gitar

gitar ("git archive") uses the git backend to make really small packages out of file trees with lots of redundancy.  ungitar unpacks the packages; it requires ftx.

A .gitar package consists of an ftc package containing a bare git repository whose HEAD is the original tree and whose objects are all stored in a single pack.  Hence, git will represent similar files in the original tree as deltas.

gitar is great for compressing together several versions of the same piece of software.  I had seven versions of my custom rsync lying around, each about 585 KB as a tar-bz2 package.  I unpacked them all inside a single folder and gitared that folder; the resulting gitar package was only 865 KB.

Note (2008-06-01): A while ago, git gained a standardized binary format for "bundles"; I should change gitar and ungitar to work on bundles.

patchsync

patchsync synchronizes a trunk, a branch, and a patch that contains the differences between the two.  If the trunk or patch changes, it updates the branch; if the branch changes, it updates the patch.  I developed patchsync to help me follow branches of rsync, but I no longer use it for that purpose.  Depending on your situation, you may prefer a more sophisticated patch-management tool such as StGIT.

To set up a patchsync staging directory, run:

patchsync --new trunk patch branch where-to-create-staging

Then, to synchronize, run:

patchsync staging

Read the gigantic comment at the top of patchsync for much more information.

Version log

• "Original" version (retroactively numbered 1) (2006.09.03).
• Version 2 (2006.12.14): I essentially rewrote patchsync and made many enhancements.  Two of them are big: patchsync can do most of the work of creating a staging directory for you, and the synchronization state consists of hash codes of the trunk, branch, and patch instead of full copies.  The staging directory format has changed.  To migrate: synchronize with the old patchsync, delete the staging directory, and create a new one using the new patchsync.
• Version 2.1 (2006.12.15): Patchsync now erases patch-new and branch-new even if nothing changed.  So if your areas are in sync but you force patchsync to update one of them on a dry run, you can clean up by running patchsync again.  I forgot to update the version number in the file to 2.1.
• Version 2.2 (2006.12.16): I fixed the patch-work file names in the cmp tests during copying out; that means patchsync will correctly skip copying the patch out if its data didn't actually change.  I also factored the version number out of the --version message so there's one less thing I can forget to update.
• Version 2.3 (2007.01.07): Three minor improvements: define cp2 to exec rsync to reduce the number of processes created, add ./ in . ./settings to avoid searching the path for programs called settings, and update some comments.
• Version 2.4 (2007.07.05): Fix a serious bug in wdpp_from that broke link-dest when copying the branch out and "patchsync --new" when the staging directory wasn't inside the current directory.  Strengthen the weak error checking that allowed this bug to go unnoticed for so long.  Fix handling of absolute trunk, patch, and branch paths in "patchsync --new".  Remove some old comments.

Patchsync files below are named for their versions; I recommend renaming the one you download to just patchsync.

From 2006.12.16 to 2006.12.24, a development version of patchsync was mistakenly identified as version 2.2.  As of 2006.12.24, the real version 2.2 is posted.

continusync

Continusync is a perl script around inotifywait and rsync that performs continuous mirroring, as suggested by Buck Huppmann. It is currently experimental and rather inefficient, but it does appear to work in simple cases.  If you want to use it, I would be much obliged if you improved it as necessary and sent me the improved version.

Isolated Firefox

This is a firefox wrapper script that creates and uses a disposable profile.  You can make it harder for people to correlate your activities across multiple Web sites by browsing each site with a separate Firefox profile created by this script.  This script was inspired by the Facebook Beacon outrage.

To use this script, install it in your $PATH and name your master Firefox profile (from which the disposable ones will be copied) 00000000.master; then just run firefox.  Your mileage may vary.

Last update 2007-12-02: Initial posting.  Seems to work.

ntfsresizecopy

ntfsresizecopy copies an NTFS filesystem from one block device to another, resizing it to the size of the destination device in the process.  (It uses ntfsprogs.)  This is EXPERIMENTAL; after using this script, you should mount the destination read-only and check that everything looks intact.

An expanding copy is just done with ntfsclone followed by ntfsresize.  A shrinking copy is done by running ntfsclone and ntfsresize on devices specially crafted with the Linux device-mapper (requires dmsetup and losetup); you may save time by checking first that the shrinkage is possible with `ntfsresize -n -s SIZE SRC'.

The special shrinking technique should be applicable to any filesystem type that has an in-place shrinking command that doesn't write outside the new size.  Just change the calls to ntfsclone and ntfsresize; ntfsclone can be replaced by a dd of the beginning of the source for filesystems that don't have a sparse clone command.

Change log

• 2008-06-01: Initial posting.  Seems to work.