Set the file's gid to -1 if we don't have permissions to set it.