If the watchdog thread is sleeping on a watchdog_item and
vm_execute_jit calls watchdog_disarm and returns before the watchdog wakes up,
then the watchdog thread will wake up and try to use the now uninitialized
item->abs_timeout.
Fix this by adding an in_use flag, and another condition variable.
watchdog_disarm now wakes the watchdog thread,
and waits till it releases the item.
Thanks to Michael Scheidell for providing feedback on this bug.
Internal version is an LLVM 2.8 with 2 patches backported from LLVM 2.9 to fix a
crash on AVX chips.
So drop support for building with external LLVM 2.8, and add support for
building with external LLVM 2.9 instead.
Caveat:
stack smashing protection is broken on LLVM 2.9 so it is disabled
Example on Debian:
apt-get install llvm-2.9-dev
./configure --enable-llvm --with-system-llvm=/usr/bin/llvm-config-2.9
Don't create a new watchdog thread everytime a bytecode is run.
Instead try to reuse a previously create watchdog thread.
Watchdog thread will idle for a max of 10s, and then exit, and new watchdog being
created as needed.
can't use setjmp inside a function that is not the parent.
just write and use some macros.
Otherwise the "else" from handler.Set() was never reached, which caused a unit
test to fail.
Writing to errs() when stderr is not open results in write Error getting flagged
in LLVM's raw_fd_ostream, which results in a call to llvm_report_error.
However since jmp_buf is not valid anymore we end up crashing on longjmp.
Fix:
- reset the thread-local recovery (jmp_buf*) to NULL when exiting the block
where it is valid
- clear errors on errs() on shutdown to avoid the report_fatal_error
Parallel cl_load() crash (bb #2333).
Reason is twofold:
- cache.c had 2 'static' global variables, thus trying to initialize same cache
from multiple threads
- bytecode2llvm.cpp: something in LLVM 2.7 is crashing when loading in
parallel
Fix is to drop the 'static' on the variable (cache is per engine already).
This also fixes a potential memory leak in clamd!
The other part of the fix is to turn on the mutex around bytecode compilation
always. We don't call cl_load in parallel, so this doesn't affect clamd, but
some may need to call cl_load in parallel.
* llvm2.8:
Regenerate configure and Makefile.
LLVM 2.8 changed llvm.mem{cpy,move,set}.* to take 5 arguments.
LLVM 2.8 API update.
Don't cast the iterator directly.
Use CallSite instead of CI->getOperand.
Support building with external LLVM 2.8.
Conflicts:
libclamav/c++/Makefile.in
libclamav/c++/bytecode2llvm.cpp
libclamav/c++/configure
Use AutoUpgrade to upgrade these calls.
Intrinsic::getDeclaration can't be used because in 2.7 it needs 1 type,
in 2.8 it needs 2/3, and there is no function to query how many it needs.
Avoid quadratic load times: run module passes at the end, and run
rtcheck inserter at the end also.
Also optimize away some simple situations that the compiler couldn't,
like result of __is_bigendian() which is a constant at load time (but not at
compile time).
If we would have run LLVM's -O1 optimizers these would have been
optimized already, but we don't do that for 2 reasons:
- optimizations may introduce new bugs
- may take a bit longer time
Just run some simple transforms, and some custom optimizations for the bigendian
case.
GCC (4.4 at least) requires and creates functions with 16-byte stack alignment
on 32-bit Linux too.
If they happen to use SSE instructions they will crash if incoming stack
alignment is not 16-byte.
LLVM claims that 4-byte stack alignment is the standard, and it only aligns to
4-byte, hence the crash. Apparently older versions of GCC/glibc would crash
if it set alignment to 16-byte.
But since the oldest GCC we support is 4.1.2, and disable LLVM on anything older
just set stack alignment to 16-byte for all functions. LLVM will realign stack
as needed. To be safe turn this on only on Linux.
This should really be fixed by either GCC or LLVM, but until that happens (see
LLVM PR8152) lets use this workaround.
bytecode selfcheck running under JIT ran too early, and spawned a thread.
Then clamd forked. threads + fork = bad idea.
So prevent the thread from being spawned in selfcheck mode.
So at the time of fork clamd will still be single threaded as in 0.96.1.
The fix for bb #1924 already fixed most of these, by using the stackprotectreq
attribute to determine if a function was already verified, not to verify it
twice.
However very simple bytecodes don't get that attribute applied. Fix this by
always applying the stackprotect attribute, and checking for that attribute
instead of stackprotectreq.
Detect PaX and fallback to intepreter if needed (bb #2092).
Recent PaX versions deny the RWX mapping, but older versions silently change it
to RW, which causes the program to die as soon as it tries to execute JITed
code.
Add selfcheck on startup (bb #2092).
This will run a very simple bytecode on startup in both JIT and interpreter
mode. The bytecode only calls 1 libclamav API and returns.
Bounds checks were too strict, causing the bytecode to abort when it shouldn't.
This happened when trying to access the last byte of an array, the verifier
was too conservative and considered to be out of bounds, when in fact it wasn't.
This is an update of the runtime verifier from the bytecode compiler.
