bash variables and rsync excludes

By Paul Heinlein | Jul 17, 2018

bash and rsync don’t necessarily work and play well together when rsync’s --exclude option is set as a variable.

Try this:

# on source host, create test directory and files
cd /var/tmp
mkdir mydir
pushd mydir
touch file1.txt file2.txt file3.pdf file4.pdf
popd
# get whole tree to remote host; note LACK of trailing
# slash on source directory
rsync -av /var/tmp/mydir remote.host:/var/tmp

At this point, localhost:/var/tmp/mydir and remote.host:/var/tmp/mydir are identical.

Excluding a file

I want to exclude file1.txt from the rsync and delete it from the remote host.

# on source host. note INCLUSION of trailing slash on source
# directory; we presume remote directory already exists.
rsync -av \
  --exclude 'file1.txt' --delete-excluded \
  /var/tmp/mydir/ remote.host:/var/tmp/mydir

file1.txt no longer exists on the remote host. So far, so good!

Say, however, you want to pass the excluded file as a variable:

# reset remote directory so it's the same as the local source
rsync -av /var/tmp/mydir remote.host:/var/tmp
# set an exclude variable
EXCLUDE="--exclude 'file1.txt'"
rsync -av \
  $EXCLUDE --delete-excluded \
  /var/tmp/mydir/ remote.host:/var/tmp/mydir
# FAIL! file1.txt will still be on the remote host

Removing the single quote marks from the variable works, e.g,

EXCLUDE="--exclude file1.txt"

but then you can’t include file globs or wildcards in your exclude list.

I tried several variations that all failed:

# swap single and double quotes
EXCLUDE='--exclude "file1.txt"'
# escaped quotes
EXCLUDE="--exclude \"file1.txt\""
# all of the above with = sign in option, e.g.,
EXCLUDE="--exclude='file1.txt'"

An array of solution

The solution is to use a bash array, but I admit I do not understand why it works.

OPTLIST=(-a)
OPTLIST+=(-v)
OPTLIST+=(--delete-excluded)

for EXC in '*.txt' 'file4.pdf'; do
  OPTLIST+=(--exclude "$EXC")
done

# note quotes around array variable; this always works. without quotes
# it will fail if you have wildcards in your exclude pattern(s)
rsync "${OPTLIST[@]}" /var/tmp/mydir/ remote.host:/var/tmp/mydir

Howto