-march=native should be run locally

Maks Verver <email address hidden>
to distcc
Hi everyone,

I've set up distcc to be able to compile packages for a host running Gentoo on
a somewhat slow machine. Gentoo has a nice feature where you can tell the
package manager to use distcc for all packages, automatically adjustting
compiler paths and whatnot. In general, this works fine. There are few
packages that call the compiler incorrectly (e.g. "gcc" instead
of "i386-pc-linux-gnu") but since I've set-up distcc so that on the build
nodes the default compiler is not in the path, if this occurs the build fails
and I can restart it without using distcc. So far so good.

However, I do have problems with packages that call gcc with the -march=native
option. This option causes gcc to generate code for the native platform and
CPU (similar to specifying a specific architecture like
e.g. -march=athlon-xp). The problem is that this causes the build nodes to
generate instructions that are not available on the machine that initiated
the build!

This is annoying because this miscompilation is not detected while compiling
and linking (because the object files produced have a perfectly valid
format). It's also not immediately apparent when running the executable,
because not all object files are compiled remotely and even if they are, not
all object files contain illegal instructions. The result is that an
application may work fine for some inputs but may mysteriously abort with
an "invalid instruction" exception when it happens to come across code in a
miscompiled object file.

So my question with this is: is this a known problem? Is there a way to work
around this, short of "don't use distcc for these packages"? In my opinion,
distcc should either process requests locally when the -march=native option
is given (just like operations like linking are always done locally), or
(preferably?) rewrite them to read -march=arch-of-the-build-machine so the
target architecture is the same on all build nodes, but I don't know if this
is hard to implement (it's a very gcc-specific feature). Comments/opinions?