bash-completion fails to complete filenames with single quote

Binary package hint: bash-completion

Steps to reproduce:
1) Install bash-completion (d'oh).
2) Enable it either by editing /etc/bash.bashrc or $HOME/.bashrc. /etc/bash.bashrc should have the information.
3) Actual commands to reproduce in a shell:
cd /tmp
mkdir NewDir # This can be whatever. For demonstration purposes, we want an empty directory with write access.
cd NewDir
touch "A's_one"
touch "A's_two"
cat A<tab> # This completes to "cat A\'s_" as expected. Now enter "o<tab>" and bash will fail to complete to "A\'s_one"

Removing bash-completion makes <tab> behave as expected.

Ubuntu version: Lucid
bash-completion version: 1:1.1-3ubuntu2
bash version: 4.1-2ubuntu3 (reported as 4.1.5(1)-release by bash --version)
Related bugs: 476968