Bug#628224: Fwd: Re: Bug#628224: enabling hardware acceleration crashes vlc with segmentation fault

[Rémi Denis-Courmont]

tags 628224 - moreinfo
reassign 628224 libva1
thanks

	Hello,

> I attached the "backtrace" followed by the "backtrace full".

For whatever reason, libva appears to call strlen() on an invalid string 
pointer - which is not coming from VLC.

-- 
Rémi Denis-Courmont
http://www.remlab.net/
http://fi.linkedin.com/in/remidenis
-------------- next part --------------
(gdb) backtrace
#0  __strlen_sse2_bsf () at ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S:52
#1  0xb2bb2bfd in vaInitialize () from /usr/lib/libva.so.1
#2  0xb2d0bfa9 in Open (i_codec_id=28) at vaapi.c:144
#3  vlc_va_NewVaapi (i_codec_id=28) at vaapi.c:490
#4  0xb2d06b0e in ffmpeg_GetFormat (p_codec=0x87f05e0, pi_fmt=0xb1241904) at video.c:1177
#5  0xb0fae9d9 in ?? () from /usr/lib/i686/cmov/libavcodec.so.52
#6  0xb0fb1857 in ?? () from /usr/lib/i686/cmov/libavcodec.so.52
#7  0xb0fb1fe9 in ?? () from /usr/lib/i686/cmov/libavcodec.so.52
#8  0xb11368f5 in avcodec_decode_video2 () from /usr/lib/i686/cmov/libavcodec.so.52
#9  0xb11369b9 in avcodec_decode_video () from /usr/lib/i686/cmov/libavcodec.so.52
#10 0xb2d07fe7 in DecodeVideo (p_dec=0x87d429c, pp_block=0xb327224c) at video.c:550
#11 0xb761e3b5 in DecoderDecodeVideo (p_dec=0x87d429c, p_block=0x881eef8) at input/decoder.c:1466
#12 0xb761db1c in DecoderProcessVideo (p_dec=0x87d429c, p_block=<value optimized out>)
    at input/decoder.c:1835
#13 DecoderProcess (p_dec=0x87d429c, p_block=<value optimized out>) at input/decoder.c:2007
#14 0xb761dd39 in DecoderThread (p_this=0x87d429c) at input/decoder.c:892
#15 0xb767b6d9 in thread_entry (data=0x8951dc0) at misc/threads.c:58
#16 0xb75c5c39 in start_thread (arg=0xb3272b70) at pthread_create.c:304
#17 0xb752e8ee in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
(gdb) backtrace full
#0  __strlen_sse2_bsf () at ../sysdeps/i386/i686/multiarch/strlen-sse2-bsf.S:52
No locals.
#1  0xb2bb2bfd in vaInitialize () from /usr/lib/libva.so.1
No symbol table info available.
#2  0xb2d0bfa9 in Open (i_codec_id=28) at vaapi.c:144
        i_profile = VAProfileH264High
        i_surface_count = 17
        attrib = {type = 3000040602, value = 3000046883}
#3  vlc_va_NewVaapi (i_codec_id=28) at vaapi.c:490
        fail = <value optimized out>
        p_va = 0x881fb28
#4  0xb2d06b0e in ffmpeg_GetFormat (p_codec=0x87f05e0, pi_fmt=0xb1241904) at video.c:1177
        ppsz_name = {0xb2d115e0 "PIX_FMT_YUV420P", 0xb2d115f0 "PIX_FMT_YUYV422", 0x0 <repeats 34 times>, 
          0xb2d11600 "PIX_FMT_VDPAU_H264", 0x0 <repeats 14 times>, 0xb2d11613 "PIX_FMT_VAAPI_MOCO", 
          0xb2d11626 "PIX_FMT_VAAPI_IDCT", 0xb2d11639 "PIX_FMT_VAAPI_VLD", 0x0 <repeats 13 times>}
        i = <value optimized out>
        p_dec = 0x87d429c
        p_sys = 0x87f0990
#5  0xb0fae9d9 in ?? () from /usr/lib/i686/cmov/libavcodec.so.52
No symbol table info available.
#6  0xb0fb1857 in ?? () from /usr/lib/i686/cmov/libavcodec.so.52
No symbol table info available.
#7  0xb0fb1fe9 in ?? () from /usr/lib/i686/cmov/libavcodec.so.52
No symbol table info available.
#8  0xb11368f5 in avcodec_decode_video2 () from /usr/lib/i686/cmov/libavcodec.so.52
No symbol table info available.
#9  0xb11369b9 in avcodec_decode_video () from /usr/lib/i686/cmov/libavcodec.so.52
No symbol table info available.
#10 0xb2d07fe7 in DecodeVideo (p_dec=0x87d429c, pp_block=0xb327224c) at video.c:550
        i_used = <value optimized out>
---Type <return> to continue, or q <return> to quit---
        b_gotpicture = 0
        p_pic = <value optimized out>
        i_display_date = <value optimized out>
        p_sys = <value optimized out>
        b_drawpicture = 1
        b_null_size = 0
        p_block = <value optimized out>
#11 0xb761e3b5 in DecoderDecodeVideo (p_dec=0x87d429c, p_block=0x881eef8) at input/decoder.c:1466
        p_owner = 0x87d4580
        p_input = 0xb30005c4
        p_pic = <value optimized out>
        i_lost = 0
        i_decoded = 0
        i_displayed = 0
#12 0xb761db1c in DecoderProcessVideo (p_dec=0x87d429c, p_block=<value optimized out>)
    at input/decoder.c:1835
        p_owner = 0x87d4580
#13 DecoderProcess (p_dec=0x87d429c, p_block=<value optimized out>) at input/decoder.c:2007
        b_flush = false
        p_owner = 0x87d4580
        b_flush_request = false
#14 0xb761dd39 in DecoderThread (p_this=0x87d429c) at input/decoder.c:892
        canc = 0
        p_block = 0x881eef8
        p_dec = 0x87d429c
        p_owner = 0x87d4580
#15 0xb767b6d9 in thread_entry (data=0x8951dc0) at misc/threads.c:58
        obj = 0x87d429c
        func = 0xb761dcb0 <DecoderThread>
#16 0xb75c5c39 in start_thread (arg=0xb3272b70) at pthread_create.c:304
---Type <return> to continue, or q <return> to quit---
        __res = <value optimized out>
        __ignore1 = <value optimized out>
        __ignore2 = <value optimized out>
        pd = 0xb3272b70
        now = <value optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {-1218617356, 0, 4001536, -1289280360, -600978403, 
                718454293}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, 
              cleanup = 0x0, canceltype = 0}}}
        not_first_call = <value optimized out>
        freesize = <value optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#17 0xb752e8ee in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
No locals.
(gdb)