[Pkg-samba-maint] Bug#445566: compiler bug pin-down

[Herbert Valerio Riedel]

as usual, I have something to add few minutes after I've sent the
mail... :-)

On Mon, 2007-12-31 at 10:33 +0100, Herbert Valerio Riedel wrote:
[..]
> which when compiled with  4.2.3 20071123 (prerelease) (Debian 4.2.2-4),
> causes a segfault when using -O2, but works when either removing the
> 'static' modifier and thus avoiding inlining of do_segfault, or by using
> an optimization level which does avoid that... 
> 
> the generated assembler code is quite broken for the optimized case:
> 
> (I've only pasted and commented the relevant section containing the 5
> nops)
> 
> // r4 points to outbuf (= source buffer)
> // sp points to inbuf  (= target buffer)
> #APP
>         nop
>         mov     r2, sp
>         add     r1, sp, #56 // upper loop-bound; should have been #12

I actually wanted to say 'should have been #24' :-)

this overrun is what causes the segfault... and which goes away when not
inlining, but the source-ptr not updating...

> 	cmp     r4, r1 // upper loop-bound check
>         add     r2, r2, #2 // increment target buffer (NB: source buffer is not incremented!)
>         bne     .L2 // repeat loop if upper loop-bound not reached yet

...stays even when avoiding inlining; it's somehow the optimizer that
gets confused by

in_buf[2*i] = (   out_buf[2*i] | out_buf[(2*i)+1]<<8  ) & 0xFF;
in_buf[(2*i)+1] =  ( out_buf[2*i] | out_buf[(2*i)+1]<<8 ) >> 8;

which gcc correctly optimizes to

in_buf[2*i] = out_buf[2*i];
in_buf[(2*i)+1] = out_buf[(2*i)+1];

but then gets confused; if one takes away that confusion from gcc by
replacing the former two lines by their latter equivalent optimized
variants, code generation is fine for all optimization levels...

cheers,
hvr