Intro

For the last six months or so, I have been attempting to figure out an efficient way to fuzz 32-bit ARM binaries while working in an x86 environment. I went into this knowing next to nothing, but have come out the other end with several key pieces of knowledge and a preferred workflow. If you are experienced in this area, you may be saying “Just use QEMU!”. You are correct, but there are two reasons this wasnt as easy as it should have been.

AFL++

For the uninitiated, AFL++ (American Fuzzy Lop) is a tool used to fuzz command line programs. A typical install and usage is this: Compile and install AFL++ on your host, and use AFL’s compiler stand-ins to instrument the program during compile. Then, using afl-fuzz perform fuzzing of the target. Simple. This is all fine for programs matching the host CPU architecture, but what if we wanted to fuzz programs created for other, sometimes exotic, architectures?

QEMU Mode

QEMU is a system emulation suite. It can be used to run just a program via emulation or to emulate a whole sytem. More on emulating a system later. AFL++ can be built with a handy mode called QEMU mode. It does exactly what it sounds like. Running AFL with the ‘-Q’ flag tells afl-fuzz to run the target via emulation. This does not just run the binary however, it will inject instrumentation at runtime. If you are trying to find vulenrabilities in a target program meant to run on embedded systems, this is likely your first and most widely known option. This comes with the caviate that your code coverage as compared to compiling the program with AFL will be significantly worse. If you have the source code to the program and cross-compile it using a standard gcc toolchain you will be limited in your fuzzing results.

To increase the code coverage when the source code is available, we can look to two possible methods: Cross-Compiling using AFL or full system emulation.

Cross-Compiling

Admitedly, this is the worst option. However it was a good exercise for me in solving problems to achieve an interesting end goal. If you try to compile a program which was written for ARM devices with afl-gcc or any other AFL compiler, you will receive a ton of errors. I wont go over them here. Your best choice is to use ‘afl-clang-fast’ as you can specify the flag ‘–target=’ to compile for a target CPU architecture. This does not solve everything however, as a crucial piece of AFL is not built to compile ARM binaries. We can get around this by cross-compiling the object file ‘afl-compiler-rt.o’ with a gcc toolchain:

img

This will allow us to compile and isntrument our ARM programs. Once we have our target binaries, we still need a way to run them on our x86 system. Passing the QEMU mode flag in afl-fuzz gives us a warning that the binary is already instrumented.

img

This leads us into the real solution to compiling, instrumenting and fuzzing ARM binaries: Docker with QEMU system emulation.

QEMU AFL++

The official docker image for AFL++ has an ARM64 variant. This gave me the clue that we could potentially compile and install AFL on a 32-bit ARM docker image. A little googling howed there to be an armv7 (32-bit) Debian docker image in their official repo. After installing ‘qemu-user-static’ and running the Docker image, it is as simple as git pulling the latest repo AFLplusplus and following the install instructions (https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/INSTALL.md). Once we have all dependencies, running ‘make all LLVM_CONFIG=llvm-config-14’ will make all the instrumentation and fuzzing binaries for AFL.

Since this install is on an armv7 Docker image, we can now compile and fuzz our target program “natively” by replacing all mentions of the gcc crosscompiling toolchain with ‘afl-clang-fast’. Note: to get your source code into the Docker image, I recomment passing the ‘-v’ flag. Example: ‘docker run -it -v /home/user/sourcecode:/src arm32v7/debian’. This will mount your host’s /home/user/sourcecode to /src in the Docker container.

Then, once everything has been compiled and instrumented, its as simple as running afl-fuzz on your target.

That is all for now.