Bug#748061: SASL's version of MD5_CTX lacks 4 bytes

[Michael Tautschnig]

Package: cyrus-sasl2
Version: 2.1.26.dfsg1-9
Usertags: goto-cc

During a rebuild of all packages in a clean sid chroot (and cowbuilder+pbuilder)
the build failed with the following error. Please note that we use our research
compiler tool-chain (using tools from the cbmc package), which permits extended
reporting on type inconsistencies at link time.

[...]
libtool: link: gcc -shared  -fPIC -DPIC  .libs/otp.o .libs/otp_init.o .libs/plugin_common.o   -L/usr/lib/mit-krb5 -L/usr/x86_64-linux-gnu/lib -lcrypto -lresolv  -O2 -Wl,-z -Wl,relro -Wl,-z -Wl,now -Wl,-z -Wl,defs   -Wl,-soname -Wl,libotp.so.2 -o .libs/libotp.so.2.0.25

error: conflicting function declarations "otp_client_plug_init"
old definition in module otp file ../../plugins/otp.c line 1832
signed int (struct sasl_utils *utils, signed int maxversion, signed int *out_version, struct sasl_client_plug **pluglist, signed int *plugcount)
new definition in module otp_init file ../../plugins/otp_init.c line 41
signed int (struct sasl_utils *, signed int, signed int *, struct sasl_client_plug **, signed int *)
Makefile:658: recipe for target 'libotp.la' failed
make[4]: *** [libotp.la] Error 64
make[4]: Leaving directory '/srv/jenkins-slave/workspace/sid-goto-cc-cyrus-sasl2/cyrus-sasl2-2.1.26.dfsg1/build-mit/plugins'
Makefile:580: recipe for target 'all-recursive' failed

While the above declarations appear to be sane, closer inspection yields
different implementations of the MD5_CTX type. This will either be openssl's
version:

struct MD5state_st {
  unsigned int A;
  unsigned int B;
  unsigned int C;
  unsigned int D;
  unsigned int Nl;
  unsigned int Nh;
  unsigned int [16l] data;
  unsigned int num;
}

Or SASL's built-in variant:

typedef struct {
  unsigned int [4l] state;
  unsigned int [2l] count;
  unsigned char [64l] buffer;
} MD5_CTX

First of all, the missing 4 bytes (the num field) may be problematic as SASL's
version seemingly mirrors the byte layout. As the above linking error shows,
indeed these functions appear to use either of the two types interchangeably.
Thus the data of the num field will get lost or become inconsistent (or
insufficient memory will be allocated).

I'm leaving it to the maintainer to judge the actual severity of this as I'm
lacking any actual knowledge of the inner workings of the code.

Best,
Michael

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 859 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-cyrus-sasl2-debian-devel/attachments/20140513/16a7a95f/attachment.sig>