[debian-mysql] ABI break between 5.5 and 5.6?
Norvald H. Ryeng
norvald.ryeng at oracle.com
Tue Sep 16 14:31:48 UTC 2014
Hi Robie,
On Wed, 10 Sep 2014 12:24:42 +0200, Robie Basak <robie.basak at ubuntu.com>
wrote:
> Hi Morgan,
>
> Thank you for replying so quickly.
>
>> The followup question of course is if the breakage has a direct effect
>> on Debian applications. We may actually be lucky in this instance,
>> and the incompatibility is limited in scope. If there are issues,
>> what we may be able to do is effectively reserve libmysqlclient.so.19
>> for Debian to reconcile the problem. This is something to discuss.
>
> OK, so what do we do now?
We have a preliminary analysis. There are still a few details to
examine, but I believe we have a result we can share now. It may change
if the details turn out to be not exactly as expected, though.
I'm sorry it's a bit long, but I thought it would be better to give you
everything we have than to try to shorten it down to a more comfortable
length. There's a recommendation for further action at the end.
1. Bug description
==================
There is a mismatch between libmysqlclient 18.0 (MySQL 5.5) and
libmysqlclient 18.1 (MySQL 5.6) in the enum "mysql_option" in
mysql/mysql.h in the MySQL C API. The soname for library is identical:
"libmyqlclient.so.18".
We have the following mapping of mismatches:
Value 18.0 (MySQL 5.5) 18.1 (MySQL 5.6)
----- ----------------------------- -----------------------------
24 MYSQL_ENABLE_CLEARTEXT_PLUGIN <-> MYSQL_OPT_BIND
36 n/a <-> MYSQL_ENABLE_CLEARTEXT_PLUGIN
The MYSQL_ENABLE_CLEARTEXT_PLUGIN enables the usage of a client side
plugin to send the password in clear text over the connection (which may
be encrypted) to the server, instead of sending hashes/digests, which is
the normal. This is used by the PAM plugin on the server in order to
provide the password to PAM.
The MYSQL_OPT_BIND option is used to specify the hostname/IP address
that the client will use when connecting to the server. This can be used
in case the client has several IP addresses and wants to be specific
about which address is used for the outgoing connection.
With MYSQL_ENABLE_CLEARTEXT_PLUGIN, a my_bool (i.e., char) parameter is
passed. With MYSQL_OPT_BIND, a char* parameter is passed. The incorrect
interpretations would be that a char* is interpreted as a my_bool, which
would likely be true, and a my_bool interpreted as a char*, which will
be a wild pointer.
2. Versions/editions impacted
=============================
The MYSQL_ENABLE_CLEARTEXT_PLUGIN option is available in both
community and commercial editions of MySQL 5.5 and newer. It is used
with the PAM plugin that comes with the commercial editions of MySQL
Server 5.5 and newer. The plugin interface is available in the
community (open source) edition, but not the plugin itself. Third
parties may have implemented their own plugins using that interface.
The MYSQL_OPT_BIND option is available in both community and commercial
editions of MySQL 5.6.
3. Problematic use cases
========================
There is no problem if applications are run with the same client
library that they were linked with at compile time. The problem occurs
when different library versions are used at compile time and run time.
There are two situations that can occur:
- Compiled with the 18.0 (5.5) library and run with the 18.1 (5.6)
library.
- Compiled with the 18.1 (5.6) library and run with the 18.0 (5.5)
library.
Of these, only the first one is a supported case.
3.1. Compiling with 18.0 (MySQL 5.5) and running with 18.1 (MySQL 5.6)
----------------------------------------------------------------------
This was intended to be a supported combination. The major number is
the same, indicating that the client library in 5.6 is fully
compatible with the library in 5.5. The ABI minor number is bumped,
indicating that there are functions in the 5.6 library that are not in
the 5.5 library.
We've identified only one use case where the bug will manifest itself.
3.1.1. Using MYSQL_ENABLE_CLEARTEXT_PLUGIN (built for 5.5) and running
with the 5.6 library.
----------------------------------------------------------------------
This is a likely scenario when moving a MySQL 5.5 stack to new 5.6 host,
a natural upgrade path.
There are two cases here, MYSQL_ENABLE_CLEARTEXT_PLUGIN set to 0 or 1.
The effect of this will be that MYSQL_ENABLE_CLEARTEXT_PLUGIN is not
used and the MYSQL_OPT_BIND option will get an argument of 1 or 0. When
testing this, we haven't seen crashes, but error messages that may be
cryptic to the user: "Unknown MySQL server host '' (-2)".
Analysis: A failure to connect is a good result. If the client (despite
our test results) is able to connect, the user will most likely be
rejected because the server side plugin that expects the clear text
password will not get it, but the details are plugin specific.
3.2. Compiling with 18.1 (MySQL 5.6) and running with 18.0 (MySQL 5.5)
----------------------------------------------------------------------
This is an unsupported combination of compile time and run time
libraries. The user is compiling the application with a client library
with ABI version 18.1 and running it with version 18.0. The soname is
the same to signify backwards compatibility, but the user is expecting
forward compatibility, which is not generally available. That the
application is able to load the library at all is a feature/weakness
of the standard ABI versioning system used in Linux.
The ABI minor number bump indicates that new functionality has been
added. If an application uses any such functionality, the expected
result would be that the application won't start when presented with
the older library at run time. Depending on how the application loads
the library, a crash may also occur. This is expected behavior for
shared libraries.
In the case that the application is able to successfully load the
library and use it, we've identified two use cases where the bug will
manifest itself.
3.2.1. Using option MYSQL_OPT_BIND (5.6 only) and running with 5.5
library
------------------------------------------------------------------
The MYSQL_OPT_BIND can be used to bind the client to a specific IP on a
host with several IP addresses. Using MYSQL_OPT_BIND is expected to be
relatively common, but using an older library at run time than at
compile time is not expected to be common.
When running an application using this feature with the 5.5 library:
a) The wanted binding is not done and traffic will be sent out from
the default interface. Depending on the setup, the client may
succeed or fail. The connection may fail because the server isn't
available on the default interface, or because the user isn't
allowed to connect from the default interface ("Host 'foo' is not
allowed to connect to this MySQL server").
In case the server name resolves differently on the different
client network interfaces, the wrong server may be contacted. If
using SSL, the client may check that the server certificate is
signed by an approved CA and refuse to connect if it isn't.
b) It is likely that MYSQL_ENABLE_CLEARTEXT_PLUGIN will be
enabled. However, it won't be used unless the following criteria
are met:
- the application is able to find a server with the same host
name/IP address on the default interface
- the server SSL and CA certificates pass the client's
verification
- the user exists on the server
- the user record indicates that the user should be
authenticated by a plugin that requires cleartext passwords
Analysis: The impact of this is most likely a failure to connect, but
it may also in some corner cases connect to the wrong server, in which
case the result is application specific. But this is a use case in
which the user can't expect anything. In general, when running with an
older library than what was available at compile time (even without
ABI breaks), the expected result is that the application may not start
or will fail to execute correctly.
3.2.2. Using MYSQL_ENABLE_CLEARTEXT_PLUGIN (built for 5.6) and running
with the 5.5 library.
----------------------------------------------------------------------
The MYSQL_ENABLE_CLEARTEXT_PLUGIN option will be ignored. If error
checking is done correctly, the application will notice that
libmysqlclient returns an error.
In case error checking isn't done, the user will most likely be
rejected because the server side plugin that expects the clear text
password will not get it, but the details are plugin specific.
Analysis: The impact of this is low. The user is using an unsupported
combination of libraries and experiences a failure to connect.
4. Affected packages in Debian
==============================
We've done a recursive search for packages in Debian unstable that
depend on libmysqlclient18. For these packages, we have downloaded the
source code and search for the two option names affected by the bug.
The packages that match this search are
- libmysqlcppconn7
- libmapistore0
- tarantool-mysql-plugin
- Percona packages
- MariaDB packages
4.1. libmysqlcppconn7
---------------------
Source package: mysql-connector-c++ (version 1.1.3)
Files: driver/mysql_connection_options.h
This library uses both the MYSQL_ENABLE_CLEARTEXT_PLUGIN and
MYSQL_OPT_BIND options. This is expected since it is a general MySQL
connector.
We haven't finished analyzing if this needs to be recompiled.
None of the packages that depend on this package are affected.
4.2. libmapistore0
---------------
Source package: openchange
Files: mapiproxy/util/mysql.po,
mapiproxy/libmapiproxy/backends/openchangedb_mysql.po.
mapiproxy/libmapistore/backends/indexing_mysql.po
mapiproxy/libmapistore/backends/namedprops_mysql.po
This is an MS Exchange protocol implementation. It matches the search
for MYSQL_ENABLE_CLEARTEXT_PLUGIN. The .po files are binary objects
shipped in the source tarball. They have been removed in the latest
package upload since there is no source code for these files.
This package doesn't need to be recompiled, but it should be watched in
case the files reappear before we've completely dealt with this issue.
4.3. tarantool-mysql-plugin
---------------------------
Source package: tarantool
Files: src/plugin/mysql/libmysql.so
Tarantool is an in-memory database and Lua application server. It matches
the search
for MYSQL_ENABLE_CLEARTEXT_PLUGIN. The .so file is a binary object
shipped in the source tarball. The package produces lintian errors since
there is no source code for this file.
It won't help to recompile this package. Since the .so file isn't built
from source, it must be recompiled by upstream.
4.4. Percona packages
---------------------
Packages percona-xtrabackup and percona-xtradb-cluster-server-5.5
(source package percona-xtradb-cluster-5.5) match our search. They
should be analyzed further.
4.5. MariaDB packages
---------------------
MariaDB matches our search and may have the same ABI breakage between
5.5 and 10.0 as MySQL has between 5.5 and 5.6. The MariaDB packages
should be analyzed further.
5. Possible solutions
=====================
Since the same value is used for two different enum values in 5.5 and
5.6, it is not possible to fix this without breaking either 5.5 or 5.6
applications.
There are two options: Bump or don't bump the ABI major version number.
5.1. Bump the version number
----------------------------
Bumping the major version number is theoretically the right thing when
the ABI changes in an incompatible way. However, as a bug fix it will
cause problems for existing applications since they must be recompiled
in order to use it.
If the user doesn't recompile all applications, they will continue to
use the 18.1 library. If the library is removed, the applications won't
run at all.
Impact: All users have to recompile all their applications.
5.2. Don't bump the version number
----------------------------------
This means to do nothing to fix the problem, just accepting that the
bug is there in 5.6 and that it must be fixed in 5.7. It will have
minor impact on some users as described in use case 3.1.1. Most users
won't notice it.
Impact: A minority of users of the commercial edition of MySQL will have
to recompile some applications.
6. Conclusion
=============
The impact for the supported use case is low. We've only been able to
produce connection failures. The use case itself is uncommon, limited to
a relatively small number of users of the commercial edition of MySQL.
Unsupported use cases have more severe consequences, but these use cases
imply that the user is doing something that in normal cases may cause a
crash.
The number of affected packages in Debian is small. One package may have
to be rebuilt, and one other must be recompiled by upstream and then
rebuilt in Debian.
6.1. Recommendation
-------------------
Based on the options listed in Sect. 5, I recommend that we don't do
anything to fix this in 5.6, but instead bump the ABI major version
number in 5.7 to stop the bug from propagating into the next GA release.
> I think this issue extends beyond Debian and Ubuntu because of users'
> binary compatibility assumptions. Consider a user bringing in a binary
> exectuable built somewhere else (another distribution, or built by hand
> against your published source or binaries). This user expects certain
> behaviour, and this behaviour will change on a switch to 5.6.
>
> The same applies vice versa. If I build a binary on Debian or Ubuntu
> that links against libmysqlclient.so.18, I expect it to work against any
> other system's libmysqlclient.so.18 regardless of whether that system
> has been built against 5.5, 5.6, from a distribution, or directly from
> upstream.
>
> So whatever we do, I think we need to agree and follow the same path. I
> don't think it's a case of reserving libmysqlclient.so.19 just for
> Debian; it's a case of coordinating the same soname bump for everyone.
>
> On the other hand, we might be able to pragmatically conclude that
> nobody will be affected, so no action is needed. Then the bugs would
> effectively be "Won't Fix" with the enum one remaining fundamentally
> unfixable for the lifetime of 5.6.
As mentioned above, I recommend the last option.
Regards,
Norvald
More information about the pkg-mysql-maint
mailing list