Commit Graph

23233 Commits

Author SHA1 Message Date
Michael Niedermayer
7fc97160c2 avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 15:04:43 +02:00
Michael Niedermayer
4865948d2e avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 23:24:27 +02:00
Michael Niedermayer
891376d261 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 21:32:15 +02:00
Michael Niedermayer
24a5cd720d avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 20:28:52 +02:00
Michael Niedermayer
75f5fe165c avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c50)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 19:47:43 +02:00
Michael Niedermayer
665f70209a avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934)

Conflicts:

	libavcodec/pngdec.c
2014-10-03 18:36:41 +02:00
Michael Niedermayer
12c8e4021c avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:35:38 +02:00
Michael Niedermayer
f56095c4d7 avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 16:48:23 +02:00
Michael Niedermayer
76601e4ab8 avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f2)

Conflicts:

	libavcodec/mmvideo.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 15:12:37 +02:00
Michael Niedermayer
e90d620cb9 avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 05:07:33 +02:00
Michael Niedermayer
0ffa44340f avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6d)

Conflicts:

	libavcodec/mjpegdec.c
(cherry picked from commit 94371a404c663c3dae3d542fa43951567ab67f82)

Conflicts:

	libavcodec/mjpegdec.c
2014-10-03 02:30:00 +02:00
Michael Niedermayer
0bd9b78fb6 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:29:04 +02:00
Michael Niedermayer
e9d85860d0 avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-27 20:58:44 +02:00
Gianluigi Tiesi
e28c27e25f avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-21 17:04:19 +02:00
James Almer
9b9048a3dd x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067)

Conflicts:

	libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-20 14:14:30 +02:00
Michael Niedermayer
d49b57fe0f avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:07 +02:00
Michael Niedermayer
0f1882b9b2 avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:06 +02:00
Michael Niedermayer
0fc229450f avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:05 +02:00
Michael Niedermayer
1482253790 avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Michael Niedermayer
48443598ed avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Michael Niedermayer
30ffb80dca avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Christophe Gisquet
18342848f7 proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff463)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-17 16:49:56 +02:00
Luca Barbato
61a51598e3 lavc: move put_bits_left in put_bits.h
(cherry picked from commit afe03092dd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-17 16:49:56 +02:00
Michael Niedermayer
abc1fa7c5a avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a)

Conflicts:

	libavcodec/iff.c
(cherry picked from commit 656f930160)

Conflicts:

	libavcodec/iff.c
2014-08-10 22:31:02 +02:00
Anton Khirnov
f712d3669e cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e2)

Conflicts:

	libavcodec/cdgraphics.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-07 02:05:19 +02:00
Michael Niedermayer
b4632cec74 avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd26)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:37 +02:00
Michael Niedermayer
454f6c622c avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a59)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:36 +02:00
Michael Niedermayer
f70ee7d14f avcodec/dvdsub_parser: never return 0 when the input isnt 0
Fixes a infinite loop
Fixes Ticket3804

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfdb30d2f1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:34 +02:00
Michael Niedermayer
9d77a62936 avcodec/alsdec: Clear MPEG4AudioConfig so that no use of uninitialized memory is possible
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e6bd5481c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:43 +02:00
Michael Niedermayer
d13cf32099 avcodec/mss4: Fix () in MKVAL() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cf7ff0146c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
5626e11bd5 avcodec/mss34dsp: fix () in SOP* macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e720c5c81)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
8d1f38c4b1 avcodec/mlpdec: fix () in MSB_MASK() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa160af08b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
ea7fbd0b16 avcodec/golomb-test: fix () in EXTEND() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 97e6b5ee3a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
b2b4df8d51 avcodec/aac: fix () in IS_CODEBOOK_UNSIGNED macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa915d4193)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
5691d9a6c1 avcodec/libvorbisenc: dont add the duration to AV_NOPTS_VALUE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 19e66c7232)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
4b638ea42d avcodec/diracdec: move mc buffer allocation to per frame
Fixes out of array accesses for non default buffers with large strides

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a30f08505)
(cherry picked from commit 9c9fc79d9237d28e33161cb2e75082d8ad232b2e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
0607c8af2b avcodec/mjpegdec: Fix undefined shift
Fixes CID1194388

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b432960528)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
85b00d798f iavcodec/vc1dec: Fix missing {}
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb53beb81a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:21 +02:00
Michael Niedermayer
53c02d04e4 avcodec/g723_1: add assert to help static code analyzers
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1457f3fd90)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:21 +02:00
Michael Niedermayer
90c7dfb907 avcodec/diracdec: fix undefined behavior with shifts
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b8598f6ce6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:21 +02:00
Michael Niedermayer
7b38ea4003 avcodec/mjpegen: Fix declared argument size
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 256f530603)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
98fe429ba5 avcodec/x86/idct_sse2_xvid: fix non C99 inline function
Found-by: Matt Oliver <protogonoi@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 46d5625f44)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Anthoine Bourgeois
90cf672d7a avcodec/dirac_arith: Fix build with PIC and stack-check options
Fixes Ticket3540

The function dirac_get_arith_bit in libavcodec/dirac_arith.h can't be
built with PIC and check-stack because the asm code needs 6 registers
and PIC and check-stack options take 1 each and x86 is quite limited
in this area.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d8ab7f31dd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
a61998e4f4 avcodec/msrle: check return code for success before use
The check is possibly redundant, but better to check for errors
that dont occur than to skip the check and crash

Fixes CID1197060

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 754f84663e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
e5546d4871 avcodec/wma: use av_freep(), do not leave stale pointers in memory
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d167faafe9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
e897eb108b avcodec/libx264: move where x264opts is applied down so it isnt overridden by avctx & defaults
fixes x264opts opengop=1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 64b79141bd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
2af3a04164 avcodec/h264_mp4toannexb_bsf: prepend global headers before any in stream parameter sets
Fixes h264_mp4toannexb_bsf_failure.mkv

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 289b149cec)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
92ca3578bf avcodec/x86/mpegvideoenc_template: fix integer overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Martin Storsjö
401b3dafd9 adpcm: Write the proper predictor in trellis mode in IMA QT
The actual predictor value, set by the trellis code, never
was written back into the variable that was written into
the block header. This was accidentally removed in b304244b.

This significantly improves the audio quality of the trellis
case, which was plain broken since b304244b.

Encoding IMA QT with trellis still actually gives a slightly
worse quality than without trellis, since the trellis encoder
doesn't use the exact same way of rounding as in
adpcm_ima_qt_compress_sample and adpcm_ima_qt_expand_nibble.

Fixes part of Ticket3701

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa8f060b75)

Conflicts:
	tests/ref/acodec/adpcm-ima_qt-trellis
2014-06-10 00:24:30 +02:00
Martin Storsjö
71516ac175 adpcm: Fix trellis encoding of IMA QT
This was broken in 095be4fb - samples+ch (for the previous
non-planar case) equals &samples_p[ch][0]. The confusion
probably stemmed from the IMA WAV case where it originally
was &samples[avctx->channels + ch], which was correctly
changed into &samples_p[ch][1].

Fixes part of Ticket3701

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a32765c425)
2014-06-10 00:24:22 +02:00