Despite much recent interest in compiler fuzzing, the practical impact of
fuzzer-found miscompilations on real-world applications has barely been
assessed. We present the first quantitative and qualitative study of the
tangible impact of fuzzer-found compiler bugs. We follow a novel methodology
where the impact of a miscompilation bug is evaluated based on (1) whether the
bug appears to trigger during compilation; (2) the extent to which generated
assembly code changes syntactically due to triggering of the bug; and (3) how
likely such changes are to cause runtime divergences during execution. The
study is conducted with respect to the compilation of more than 10 million
lines of C/C++ code from 309 Debian packages, using 12% of the historical and
now fixed miscompilation bugs found by four state-of-the-art fuzzers in the
Clang/LLVM compiler, as well as 18 other bugs found by the Alive formal
verification tool or human users. The results show that almost half of the
fuzzer-found bugs propagate to the generated binaries for some applications,
but barely affect their syntax and only cause two failures in total when
running their regression test suites. Our manual analysis of a selection of
bugs suggests that these bugs cannot trigger on the packages considered in the
analysis, and that in general they affect only corner cases which have a low
probability of occurring in practice. User-reported and Alive bugs do not
exhibit a higher impact, with less frequently triggered bugs and one test
failure.