Don't let set_perms() re-stat an existing directory or it will not set