Allow builds on platforms with unsigned chars by default

Certain architectures, such as ARM, use unsigned chars by default
so require the `-fsigned-char` for certain value comparisons to
make sense and in order to compile.