I've finally had an opportunity to test this, and from my limited knowledge of XFCE, it seems to still work fine without xubuntu-default-settings installed.
I rebuilt xfdesktop4 locally with the debdiff applied, and installed it by hand, and then installed the other dependencies and xfce4 with apt-get, and then logged into an XFCE session successfully.
I've finally had an opportunity to test this, and from my limited knowledge of XFCE, it seems to still work fine without xubuntu- default- settings installed.
I rebuilt xfdesktop4 locally with the debdiff applied, and installed it by hand, and then installed the other dependencies and xfce4 with apt-get, and then logged into an XFCE session successfully.