[arm] needs porting to thumb2

Matthias, does "only used in openjdk-6-jre-zero when using shark" mean that this is a none issue?

well, the package is in universe for now (openjdk-6-jre-zero). I do not intend to work on this one for lucid.

LLVM miscompiles when built on ARM lucid.

make[3]: Entering directory `/wd/llvm-configure/lib/Target/ARM'
...
llvm[3]: Compiling ARMJITInfo.cpp for Release build
/tmp/ccVyzBYd.s: Assembler messages:
/tmp/ccVyzBYd.s:31: Error: LR and PC should not both be in register list -- `ldmia sp!,{r0,r1,r2,r3,lr,pc}'
make[3]: *** [/wd/llvm-configure/lib/Target/ARM/Release/ARMJITInfo.o] Error 1
make[3]: Leaving directory `/wd/llvm-configure/lib/Target/ARM'
make[2]: *** [ARM/.makeall] Error 2
make[2]: Leaving directory `/wd/llvm-configure/lib/Target'
make[1]: *** [Target/.makeall] Error 2
make[1]: Leaving directory `/wd/llvm-configure/lib'
make: *** [all] Error 1

This bug are tracked upstream in llvm bugzilla:
http://llvm.org/bugs/show_bug.cgi?id=6065

This is not a miscompile in the sense that the compiler's code generation is not wrong; but there is non-Thumb-2 compatible inline assembler, which means that the code may not work if the package is built in Thumb-2.

The ldm ..., {...,lr,pc} instruction form is not permitted in Thumb-2. However, there are likely to be other problems also, which may result in buggy execution without necessarily causing compile failures.

I'm guessing that the JIT output is ARM code only (not Thumb), and that interworking is not currently handled fully for calls and returns between the C++/other high level code and the JIT output ... am I correct here?

*If* the following conditions are met, then I recommend to build the package with -marm :

1) If the JIT output calls/jumps to code which is not part of the JIT output, the destination code must

    a) be called via C++/high-level code which is part of llvm itself, or

    b) be part of llvm itself

2) Addresses of JITted code must not be pased out of the llvm package (i.e., external libraries must not invoke parts of the JITted code as a callback)

(If met, these conditions imply that switching between ARM and Thumb will not be needed when jumping into/out of the JITted code, providing that llvm itself was built as all ARM code.)

If the above conditions are not met, then we may have to look at this package more closely.

@Xerxes,

Now I look more closely, there is some Thumb / Thumb-2 JIT implementation.

It may be necessary for one of the JIT implementers for this package to comment on this. Are you able to comment yourself / do you know how to ask?

<http://llvm.org/bugs/show_bug.cgi?id=6065>:
> Would making two pops for __thumb__ instead of one big one solve this issue?

I think the answer is yes; my concern is that there may be other issues floating about which do not show up as compiler errors.

If building the package with -marm is safe (see the above post) then this is still probably the simplest, safest solution.

I fill in the bits i know about the llvm jit.

The llvm jit output armv4t code unless the JIT ExecutionEngine have been instructed that the host cpu supports anything better.
Currently there dont exist any code in the llvm execution engine arm target backend to automatically determine the host cpu capabilities.

The JIT can be tuned by projects using the LLVM JIT (like openjdk shark) by passing some command line options using the
cl::ParseCommandLineOptions(args.size() - 1, (char **) &args[0]);
defined in
#include <llvm/Support/CommandLine.h>

the available tuning options are:
-mtriple=arm-gnu-eabi-linux currently used by the llvm arm backend to switch between thumb only and arm mode.
for thumb only mode then specify a triple that start with the thumb keyword instead of arm.
-mcpu=
tunes the generated code for various cpu models and automatically enables cpu attributes
examples of cpus to specify are

 arm1020e - Select the arm1020e processor.
 arm1020t - Select the arm1020t processor.
 arm1022e - Select the arm1022e processor.
 arm10e - Select the arm10e processor.
 arm10tdmi - Select the arm10tdmi processor.
 arm1136j-s - Select the arm1136j-s processor.
 arm1136jf-s - Select the arm1136jf-s processor.
 arm1156t2-s - Select the arm1156t2-s processor.
 arm1156t2f-s - Select the arm1156t2f-s processor.
 arm1176jz-s - Select the arm1176jz-s processor.
 arm1176jzf-s - Select the arm1176jzf-s processor.
 arm710t - Select the arm710t processor.
 arm720t - Select the arm720t processor.
 arm7tdmi - Select the arm7tdmi processor.
 arm7tdmi-s - Select the arm7tdmi-s processor.
 arm8 - Select the arm8 processor.
 arm810 - Select the arm810 processor.
 arm9 - Select the arm9 processor.
 arm920 - Select the arm920 processor.
 arm920t - Select the arm920t processor.
 arm922t - Select the arm922t processor.
 arm926ej-s - Select the arm926ej-s processor.
 arm940t - Select the arm940t processor.
 arm946e-s - Select the arm946e-s processor.
 arm966e-s - Select the arm966e-s processor.
 arm968e-s - Select the arm968e-s processor.
 arm9e - Select the arm9e processor.
 arm9tdmi - Select the arm9tdmi processor.
 cortex-a8 - Select the cortex-a8 processor.
 cortex-a9 - Select the cortex-a9 processor.
 ep9312 - Select the ep9312 processor.
 generic - Select the generic processor.
 iwmmxt - Select the iwmmxt processor.
 mpcore - Select the mpcore processor.
 mpcorenovfp - Select the mpcorenovfp processor.
 strongarm - Select the strongarm processor.
 strongarm110 - Select the strongarm110 processor.
 strongarm1100 - Select the strongarm1100 processor.
 strongarm1110 - Select the strongarm1110 processor.
 xscale - Select the xscale processor.

-mattr=
manually set cpu features
Available features for this target:

 neon - Enable NEON instructions.
 neonfp - Use NEON for single-precision FP.
 thumb2 - Enable Thumb2 instructions.
 v4t - ARM v4T.
 v5t - ARM v5T.
 v5te - ARM v5TE, v5TEj, v5TExp.
 v6 - ARM v6.
 v6t2 - ARM v6t2.
 v7a - ARM v7A.
 vfp2 - Enable V...

@Dave
Here comes the answer to if we can use -marm explained by jyasskin on #llvm
"If you put a CallInst in your LLVM IR whose target is an llvm::Function, and that Function is an external declaration (i.e containing no BasicBlocks), then llvm will try to use dlsym() to look up the code for that function.
If, however, you first call ExecutionEngine::addGlobalMapping(f, &some_real_function), then the JIT will emit calls to some_real_function(...) instead."

So we will have trouble fullfill the 1.a condition that external functions should only be called via C++/high-level code which is part of llvm itself since the JIT will call these functions directly.

If the JIT runs openjdk shark then it will call jni code directly, make direct calls to libjvm.so.

Some clarification to your guess "I'm guessing that the JIT output is ARM code only (not Thumb)":
It turned out that the LLVM switches on the thumb backend by default since it have autodetected the host system to be a
Host: thumb-unknown-linux-gnueabi

The current implementation of the llvm thumb backend currently works well when using llvm as a static compiler to create assembler files:

But dont work so well when used for the ExecutionEngine JIT since the thumb backend happen to lack specifications for how to generate machine code fore all the thumb instructions. It currently only know what instruction it wants to use but then hits an assertion that the specification for the machine code are missing: This bug are tracked upstream http://llvm.org/bugs/show_bug.cgi?id=6223

example: see attached file for differences between the static compilation and jit using llc and lli with thumb/tumb2 emission.

From http://llvm.org/bugs/show_bug.cgi?id=6223

Xerxes Rånby wrote:

> I am considering implementing the missing parts to make the
> thumb backend work with the ExecutionEngine JIT in order for
> the llvm JIT to be used on Ubuntu Lucid ARM that targets
> thumb2.
>
> see: llvm needs porting to thumb2
> https://bugs.launchpad.net/ubuntu/+source/llvm/+bug/513735
>
> Any pointers to how to implement the missing parts for the
> thumb JIT would be mosty welcome.
>
> I will start trying to add the missing machinecode fields for
> the JIT to use to the ARMInstrThumb.tb and ARMInstrThumb2.tb
> tablegen files.

Adding Thumb-2 JIT support is not necessarily required.

The real requirement is to implement ARM/Thumb interworking - for Ubuntu it sounds like this will be needed even if the JIT generates Thumb-2 code.

The reason for this is that direct symbol reference resolves to a PLT veneer generated at link time (always in ARM), whereas dlsym() will resolve directly to the target symbol in the containing shared library (which may be Thumb), as shown by the example below --- and it sounds like the llvm JIT output may need to handle both.

I documented some more detailed info on the porting requirements for ARM/Thumb interworking on https://wiki.ubuntu.com/ARM/Thumb2PortingHowto, which may help explain the implications and what porting is likely to be needed.

$ cat <<EOF | gcc -o dlsym-test -xc - -ldl && ./dlsym-test
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
int main(void)
{
    printf("printf = %p\ndlsym("printf") = %p\n", printf, dlsym(RTLD_NEXT, "printf"));
    return 0;
}
EOF

printf = 0x83a8
dlsym("printf") = 0x4006e1cd