Bug#657200: Endless loop in avformat_find_stream_info()

[Max Kellermann]

Package: libavformat53
Version: 4:0.8-1
Severity: important

When calling avformat_find_stream_info() on a broken mp3 file,
libavformat enters an endless loop.  This is a vulnerability that can
be used as a remote DoS attack on radio players such as MPD, therefore
severity important.

Demo file: http://www.blarg.de/broken.mp3

Dump of a gdb session demonstrating the problem:

gdb --args ffprobe broken.mp3 
Reading symbols from /usr/bin/ffprobe...Reading symbols from /usr/lib/debug/.build-id/d1/8d41702259479824206b4584cfa11b04d6b7b3.debug...done.
done.
(gdb) run
Starting program: /usr/bin/ffprobe broken.mp3
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
[Thread debugging using libthread_db enabled]
avprobe version 0.8-4:0.8-1, Copyright (c) 2007-2011 the Libav developers
  built on Jan 22 2012 21:45:34 with gcc 4.6.2
[mp3 @ 0x6209a0] Format detected only with low score of 25, misdetection possible!
[mp3 @ 0x6228c0] Header missing

...

^C
Program received signal SIGINT, Interrupt.
apply_param_change (avctx=0x6228c0, avpkt=<optimized out>) at /build/libav-QkFId0/libav-0.8/libavcodec/utils.c:1116
1116    /build/libav-QkFId0/libav-0.8/libavcodec/utils.c: No such file or directory.
        in /build/libav-QkFId0/libav-0.8/libavcodec/utils.c
(gdb) bt
#0  apply_param_change (avctx=0x6228c0, avpkt=<optimized out>) at /build/libav-QkFId0/libav-0.8/libavcodec/utils.c:1116
#1  0x00007ffff6cce833 in avcodec_decode_audio4 (avctx=0x6228c0, frame=0x7fffffffe1a0, got_frame_ptr=0x7fffffffe37c, 
    avpkt=0x7fffffffe310) at /build/libav-QkFId0/libav-0.8/libavcodec/utils.c:1218
#2  0x00007ffff774e78d in try_decode_frame (st=0x620fe0, avpkt=<optimized out>, options=<optimized out>)
    at /build/libav-QkFId0/libav-0.8/libavformat/utils.c:2170
#3  0x00007ffff77540ed in avformat_find_stream_info (ic=0x6209a0, options=0x0)
    at /build/libav-QkFId0/libav-0.8/libavformat/utils.c:2404
#4  0x0000000000402cc0 in open_input_file (filename=0x7fffffffeb39 "broken.mp3", fmt_ctx_ptr=0x7fffffffe770)
    at /build/libav-QkFId0/libav-0.8/avprobe.c:310
#5  probe_file (filename=0x7fffffffeb39 "broken.mp3") at /build/libav-QkFId0/libav-0.8/avprobe.c:341
#6  main (argc=<optimized out>, argv=<optimized out>) at /build/libav-QkFId0/libav-0.8/avprobe.c:450