Properly handle the backup_suffix/backup_dir settings when deleting