Fix warnings

[sjaeckel]

Pulled out from #751

[sjaeckel]

IMO we should modify our builds to catch the parts that we want to as well.

We already run ubsan in the meta builds, why aren't those things caught there?

[sjaeckel]

Is this correct? Shouldn't this instead return CRYPT_OK if len==0?

[karel-m]

not sure as we do not accept zero len in ssh_read_authorized_keys

[sjaeckel]

FYI: I re-wrote this to the same form as before, since it is clearer and easier to understand IMO.

[karel-m]

I am fine with that

[karel-m]

Just FYI some options/flags I have used:

Sanitizers:

make clean
make -j$(nproc) CC=clang CFLAGS="-Wall -Wextra -I../libtommath -DUSE_LTM -DLTM_DESC -O1 -g3 -fno-omit-frame-pointer -fno-common -fsanitize=address,undefined -fsanitize-address-use-after-scope" LDFLAGS="-fsanitize=address,undefined" EXTRALIBS="-L../libtommath -ltommath" all
ASAN_OPTIONS=detect_leaks=1:abort_on_error=0:symbolize=1 UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=0 ./test

Max compiler warnings:

make clean
make -j$(nproc) CC=clang CFLAGS="-Wall -Wextra -Wstrict-prototypes -Wmissing-prototypes -Wmissing-variable-declarations -Wshadow -Wformat=2 -Wno-format-nonliteral -Wunreachable-code -Wbad-function-cast -Wwrite-strings -Wpointer-arith -Wimplicit-fallthrough -Wformat-security -Wreturn-type -Wincompatible-pointer-types -Wimplicit-function-declaration -Wint-conversion -Wfloat-conversion -Wstrict-overflow -Wvla -Walloca -Wnull-dereference -Wredundant-decls -Wnested-externs -Wdeclaration-after-statement -Wmissing-field-initializers -Wold-style-definition -I../libtommath -DUSE_LTM -DLTM_DESC -O2" EXTRALIBS="-L../libtommath -ltommath" all