Bug#742409: [Sereal] 2.06 fails to build on some architectures: (#47)

[demerphq]

On 16 September 2014 21:06, Niko Tyni <ntyni at debian.org> wrote:

> (Cc'ing Yves instead of the scary github address as he
>  explicitly asked for that in the upstream ticket.)
>
>
Thanks. :-)


> On Wed, Jul 16, 2014 at 05:06:29PM +0200, gregor herrmann wrote:
> > On Tue, 15 Jul 2014 13:03:56 -0700, Yves Orton wrote:
>
> > > This is so frustrating. Is there anyone in the Debian community that
> knows
> > > these platforms that can help?
> >
> > Hm, probably but I'm not sure who knows about all of them.
> > Cc'ing the Debian bug report.
>
> Hi, I looked at the Sereal-Encoder problems briefly. The current state
> with Debian libsereal-encoder-perl 3.002-1 (unpatched 3.002) is that it
> only fails on 64-bit big-endian architectures (s390x and powerpc64).
>
> The failures are all in t/700_roundtrip/ with the snappy tests, and
> involve array refs and long strings. I was able to reduce at least one
> of them to this:
>
>   #!/usr/bin/perl -w
>   use blib;
>   use Sereal::Decoder;
>   use Sereal::Encoder;
>   my $opt = { snappy => 1, use_protocol_v1 => 1};
>
>   my $len = shift || 509;
>
>   my $s="A"x$len;
>   my $e =  Sereal::Encoder->new($opt)->encode([$s, $s]);
>   my $d =  Sereal::Decoder->new($opt)->decode($e);
>
>   # print $e;
>   print "" . ($d->[0] eq $s ? "" : "not ") . "ok 1 first element\n";
>   print "" . ($d->[1] eq $s ? "" : "not ") . "ok 2 second element\n";
>
> which gives "not ok 2" on s390x when the length is 509 or greater.
>
> The problem is with the encoder: amd64 and s390x encoded contents differ,
> and feeding the amd64 encoded output to an s390x decoder works fine.
> The second element of the decoded array is just the string "3" in the
> incorrect case.
>
>
Yes, indeed. Snappy is getting something wrong.

IIRC, prior to this patch:

commit 910842fc8aa3b9666926e2a40ad8f7c6d5e1c6d5
Author: Yves Orton <yves.orton at booking.com>
Date:   Tue Jul 15 13:32:14 2014 +0200

    rework alignedness checks in Sereal code so that we do unaligned only
on x86

Snappy did not work at all.

https://github.com/Sereal/Sereal/commit/910842fc8aa3b9666926e2a40ad8f7c6d5e1c6d5


> I see the length of the encoded output shortens dramatically at the 508 ->
> 509 step, looks like some compression kicks in.
>
> I'm attaching the binary output from both s390x (incorrect) and amd64
> (correct) for reference for length==509. Hope they make it through,
> the md5sums should be
>
>  8787f3dd9a3bfa23bd91eb08c40f3716  509.amd64
>  276cbe59ce957915cc92b52fb0b7b51c  509.s390x
>
> All this makes me think that the problem is in snappy/csnappy_compress.c,
> but I haven't really delved into it yet.
>
> Hope this helps. Yves, please let me know if I can help in any way.
>

Well I am wondering if you can look the patch I referenced above and see if
you can work out what you need to do to get s390x to do the right thing? As
you have identified yourself this is clearly a problem with the snappy
support. But we pass test on other big-endian 64 bit platform as far as I
understand (although I might be wrong).

I have CC'ed Jarkko as he did some of the work getting us to build properly
on platforms with strict alignment and big-endian encodings.

Yves

-- 
perl -Mre=debug -e "/just|another|perl|hacker/"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-perl-maintainers/attachments/20140916/6a6a4858/attachment.html>