path: root/doc/releases/v4.0.9.rst

Changes in TIFF v4.0.9
======================

.. table:: References
    :widths: auto

    ======================  ==========================================
    Current Version         v4.0.9 (:tag:`Release-v4-0-9`)
    Previous Version        :doc:`v4.0.8 <v4.0.8>`
    Master Download Site    `<https://download.osgeo.org/libtiff/>`_
    Master HTTP Site #1     `<http://www.simplesystems.org/libtiff/>`_
    Master HTTP Site #2     `<http://libtiff.maptools.org/>`_
    ======================  ==========================================

This document describes the changes made to the software between the
*previous* and *current* versions (see above).  If you don't
find something listed here, then it was not done in this timeframe, or
it was not considered important enough to be mentioned.  The following
information is located here:


Major changes
-------------

* None


Software configuration changes
------------------------------

* :file:`test/Makefile.am`: Add some tests for tiff2bw.
* :file:`.appveyor.yml`, :file:`.travis.yml`, :file:`build/travis-ci`: apply patches
  :file:`0001-ci-Travis-script-improvements.patch` and
  :file:`0002-ci-Invoke-helper-script-via-shell.patch` by Roger Leigh
  (sent to mailing list)
* :file:`.travis.yml`, :file:`build/travis-ci`: new files from
  :file:`0001-ci-Add-Travis-support-for-Linux-builds-with-Autoconf.patch` by
  Roger Leigh (sent to mailing list on 2017-06-08).
  This patch adds support for the Travis-CI service.
* :file:`.appveyor.yml`: new file from
  :file:`0002-ci-Add-AppVeyor-support.patch` by Roger Leigh (sent to mailing
  list on 2017-06-08).
  This patch adds a :file:`.appveyor.yml` file to the top-level.  This allows
  one to opt in to having a branch built on Windows with Cygwin,
  MinGW and MSVC automatically when a branch is pushed to GitHub,
  GitLab, BitBucket or any other supported git hosting service.
* :file:`CMakeLists.txt`, :file:`test/CMakeLists.txt`, :file:`test/TiffTestCommon.cmake`: apply
  patch :file:`0001-cmake-Improve-Cygwin-and-MingGW-test-support.patch` from Roger
  Leigh (sent to mailing list on 2017-06-08).
  This patch makes the CMake build system support running the tests
  with MinGW or Cygwin.

* :file:`test/tiffcp-lzw-compat.sh`, :file:`test/images/quad-lzw-compat.tiff`: new files
  to test old-style LZW decompression
* :file:`test/common.sh`, :file:`Makefile.am`, :file:`CMakeList.txt`: updated with above
* :file:`test/Makefile.am`: add missing reference to images/quad-lzw-compat.tiff
  to fix ``make distcheck``. Patch by Roger Leigh
* :file:`nmake.opt`: support a ``DEBUG=1`` option, so as to adjust ``OPTFLAGS`` and use
  ``/MDd`` runtime in debug mode.


Library changes
---------------

* :file:`libtiff/tif_color.c`: :c:func:`TIFFYCbCrToRGBInit`: stricter clamping to avoid
  :c:type:`int32` overflow in :c:func:`TIFFYCbCrtoRGB`.
  Fixes :oss-fuzz:`1844`.
  Credit to OSS Fuzz

* :file:`libtiff/tif_getimage.c`: :c:func:`initYCbCrConversion`: stricter validation for
  ``refBlackWhite`` coefficients values. To avoid invalid ``float->int32`` conversion
  (when ``refBlackWhite[0] == 2147483648.f``)
  Fixes :oss-fuzz:`1907`.
  Credit to OSS Fuzz

* :file:`libtiff/tif_dirinfo.c`, :file:`tif_dirread.c`: add :c:func:`_TIFFCheckFieldIsValidForCodec`,
  and use it in :c:func:`TIFFReadDirectory` so as to ignore fields whose tag is a
  codec-specified tag but this codec is not enabled. This avoids :c:func:`TIFFGetField`
  to behave differently depending on whether the codec is enabled or not, and
  thus can avoid stack based buffer overflows in a number of TIFF utilities
  such as :program:`tiffsplit`, :program:`tiffcmp`, :program:`thumbnail`, etc.
  Patch derived from :file:`0063-Handle-properly-CODEC-specific-tags.patch`
  (:bugzilla:`2580`) by Raphaël Hertzog.
  Fixes:
  :bugzilla:`2580`,
  :bugzilla:`2693`,
  :bugzilla:`2625` (:cve:`2016-10095`),
  :bugzilla:`2564` (:cve:`2015-7554`),
  :bugzilla:`2561` (:cve:`2016-5318`),
  :bugzilla:`2499` (:cve:`2014-8128`),
  :bugzilla:`2441`,
  :bugzilla:`2433`.

* :file:`libtiff/tif_swab.c`: if :c:macro:`DISABLE_CHECK_TIFFSWABMACROS` is defined, do not do
  the ``#ifdef TIFFSwabXXX`` checks. Make it easier for GDAL to rename the symbols
  of its internal libtiff copy.

* :file:`libtiff/tif_dirread.c`: fix regression of libtiff 4.0.8 in
  :c:func:`ChopUpSingleUncompressedStrip` regarding update of newly single-strip
  uncompressed files whose bytecount is 0. Before the change of 2016-12-03,
  the condition ``bytecount==0`` used to trigger an early exit/disabling of
  strip chop. Re-introduce that in update mode. Otherwise this cause
  later incorrect setting for the value of ``StripByteCounts``/``StripOffsets``.
  (:gdal-trac:`6924`).
* :file:`libtiff/tif_dirread.c`: :c:func:`TIFFFetchStripThing`: limit the number of items
  read in ``StripOffsets``/``StripByteCounts`` tags to the number of strips to avoid
  excessive memory allocation.
  Fixes :oss-fuzz:`2215`.
  Credit to OSS Fuzz
* :file:`libtiff/tif_getimage.c`: avoid many (harmless) :c:expr:`unsigned int` overflows.
* :file:`libtiff/tif_fax3.c`: avoid :c:expr:`unsigned int` overflow in :c:func:`Fax3Encode2DRow`. Could
  potentially be a bug with huge rows.
* :file:`libtiff/tif_jpeg.c`: avoid (harmless) :c:expr:`unsigned int` overflow on tiled images.
* :file:`libtiff/tif_dirread.c`: avoid :c:expr:`unsigned int` overflow in :c:func:`EstimateStripByteCounts`
  and ``BYTECOUNTLOOKSBAD`` when file is too short.
* :file:`libtiff/tif_predict.c`: decorate legitimate functions where :c:expr:`unsigned int`
  overflow occur with :c:macro:`TIFF_NOSANITIZE_UNSIGNED_INT_OVERFLOW`
* :file:`libtiff/tif_dirread.c`: avoid :c:expr:`unsigned int` overflow in :c:func:`EstimateStripByteCounts`
* :file:`libtiff/tiffiop.h`: add :c:macro:`TIFF_NOSANITIZE_UNSIGNED_INT_OVERFLOW` macro to
  disable CLang warnings raised by ``-fsanitize=undefined,unsigned-integer-overflow``
* :file:`libtiff/tif_jpeg.c`: add anti-denial of service measure to avoid excessive
  CPU consumption on progressive JPEGs with a huge number of scans.
  See `<http://www.libjpeg-turbo.org/pmwiki/uploads/About/TwoIssueswiththeJPEGStandard.pdf>`_.

  .. note::

      Only affects libtiff since 2014-12-29 where support of non-baseline JPEG
      was added.

* :file:`libtiff/tif_jpeg.c`: error out at decoding time if anticipated libjpeg
  memory allocation is above 100 MB. libjpeg in case of multiple scans,
  which is allowed even in baseline JPEG, if components are spread over several
  scans and not interleavedin a single one, needs to allocate memory (or
  backing store) for the whole strip/tile.
  See `<http://www.libjpeg-turbo.org/pmwiki/uploads/About/TwoIssueswiththeJPEGStandard.pdf>`_.
  This limitation may be overridden by setting the
  ``LIBTIFF_ALLOW_LARGE_LIBJPEG_MEM_ALLOC`` environment variable, or recompiling
  libtiff with a custom value of :c:macro:`TIFF_LIBJPEG_LARGEST_MEM_ALLOC` macro.
* :file:`libtiff/tif_jbig.c`: fix memory leak in error code path of :c:func:`JBIGDecode`.
  Fixes :bugzilla:`2706`.
  Reported by team OWL337
* :file:`libtiff/tif_dirread.c`: in :c:func:`TIFFReadDirEntryFloat`, check that a
  double value can fit in a float before casting. Patch by Nicolas RUFF
* :file:`libtiff/tiffiop.h`, :file:`libtiff/tif_jpeg.c`, :file:`libtiff/tif_jpeg_12.c`,
  :file:`libtiff/tif_read.c`: make :c:func:`TIFFReadScanline` works in
  :c:macro:`CHUNKY_STRIP_READ_SUPPORT` mode with JPEG stream with multiple scans.
  Also make configurable through a ``LIBTIFF_JPEG_MAX_ALLOWED_SCAN_NUMBER``
  environment variable the maximum number of scans allowed. Defaults to
  100.
* :file:`libtiff/tif_read.c`: :c:func:`TIFFFillTile`: add limitation to the number
  of bytes read in case td_stripbytecount[strip] is bigger than
  reasonable, so as to avoid excessive memory allocation (similarly to
  what was done for :c:func:`TIFFFileStrip` on 2017-05-10)
* :file:`libtiff/tif_getimage.c`: use :c:func:`_TIFFReadEncodedStripAndAllocBuffer`.
  Fixes :bugzilla:`2708` and
  :oss-fuzz:`2433`.
  Credit to OSS Fuzz
* :file:`libtiff/tif_read.c`, tiffiop.h: add a :c:func:`_TIFFReadEncodedStripAndAllocBuffer`
  function, variant of :c:func:`TIFFReadEncodedStrip` that allocates the
  decoded buffer only after a first successful :c:func:`TIFFFillStrip`. This avoids
  excessive memory allocation on corrupted files.
* :file:`libtiff/tif_dirwrite.c`: in :c:func:`TIFFWriteDirectoryTagCheckedXXXX`
  functions associated with LONG8/SLONG8 data type, replace assertion that
  the file is BigTIFF, by a non-fatal error.
  Fixes :bugzilla:`2712`
  Reported by team OWL337
* :file:`libtiff/tif_read.c`: :c:func:`TIFFStartTile`: set tif_rawcc to
  tif_rawdataloaded when it is set. Similarly to :c:func:`TIFFStartStrip`.
  This issue was revealed by the change of 2017-06-30 in :c:func:`TIFFFileTile`,
  limiting the number of bytes read. But it could probably have been hit
  too in CHUNKY_STRIP_READ_SUPPORT mode previously ?
  Fixes :oss-fuzz:`2454`
  Credit to OSS Fuzz
* :file:`libtiff/tif_error.c, tif_warning.c`: correctly use va_list when both
  an old-style and new-style warning/error handlers are installed.
  Patch by Paavo Helde (sent on the mailing list)
* :file:`libtiff/tif_getimage.c`: use :c:func:`_TIFFReadTileAndAllocBuffer`.
  Fixes :oss-fuzz:`2470`
  Credit to OSS Fuzz.
* :file:`libtiff/tif_read.c`, tiffiop.h: add a :c:func:`_TIFFReadEncodedTileAndAllocBuffer`
  and :c:func:`_TIFFReadTileAndAllocBuffer` variants of :c:func:`TIFFReadEncodedTile` and
  :c:func:`TIFFReadTile` that allocates the decoded buffer only after a first
  successful :c:func:`TIFFFillTile`. This avoids excessive memory allocation
  on corrupted files.
* :file:`libtiff/tif_pixarlog.c`: avoid excessive memory allocation on decoding
  when RowsPerStrip tag is not defined (and thus td_rowsperstrip == UINT_MAX)
  Fixes :oss-fuzz:`2554`
  Credit to OSS Fuzz
* :file:`libtiff/tif_lzw.c`: fix 4.0.8 regression in the decoding of old-style LZW
  compressed files.
* :file:`libtiff/tif_lzw.c`: fix potential out-of-buffer read on 1-byte LZW
  strips. Crashing issue only on memory mapped files, where the strip
  offset is the last byte of the file, and the file size is a multiple
  of one page size on the CPU architecture (typically 4096). Credit
  to myself :-)
* :file:`libtiff/tif_dir.c`: avoid potential null pointer dereference in
  :c:func:`_TIFFVGetField` on corrupted TIFFTAG_NUMBEROFINKS tag instance.
  Fixes :bugzilla:`2713`
* :file:`tools/tiff2pdf.c`: prevent heap buffer overflow write in "Raw"
  mode on ``PlanarConfig=Contig`` input images.
  Fixes :bugzilla:`2715`
  Reported by team OWL337
* :file:`libtiff/tif_read.c`: :c:func:`TIFFFillStrip` / :c:func:`TIFFFillTile`.
  Complementary fix for :bugzilla:`2708`
  in the :c:func:`isMapped` case, so as to avoid excessive memory allocation
  when we need a temporary buffer but the file is truncated.
* :file:`libtiff/tif_read.c`: :c:func:`TIFFFillStrip` / :c:func:`TIFFFillTile`.
  Complementary fix for :bugzilla:`2708`
  in the :c:func:`isMapped` case, so as to avoid excessive memory allocation
  when we need a temporary buffer but the file is truncated.
* :file:`libtiff/tif_read.c`: in :c:func:`TIFFFetchStripThing`, only grow the
  arrays that hold StripOffsets/StripByteCounts, when they are smaller
  than the expected number of striles, up to 1 million striles, and
  error out beyond. Can be tweaked by setting the environment variable
  ``LIBTIFF_STRILE_ARRAY_MAX_RESIZE_COUNT``.
  This partially goes against a change added on 2002-12-17 to accept
  those arrays of wrong sizes, but is needed to avoid denial of services.
  Fixes :oss-fuzz:`2350`
  Credit to OSS Fuzz
* :file:`libtiff/tif_read.c`: in :c:func:`TIFFFetchStripThing`, only grow the
  arrays that hold ``StripOffsets``/``StripByteCounts``, when they are smaller
  than the expected number of striles, up to 1 million striles, and
  error out beyond. Can be tweaked by setting the environment variable
  ``LIBTIFF_STRILE_ARRAY_MAX_RESIZE_COUNT``.
  This partially goes against a change added on 2002-12-17 to accept
  those arrays of wrong sizes, but is needed to avoid denial of services.
  Fixes :oss-fuzz:`2350`
  Credit to OSS Fuzz
* :file:`libtiff/tif_read.c`: add protection against excessive memory
  allocation attempts in :c:func:`TIFFReadDirEntryArray` on short files.
  Effective for mmap'ed case. And non-mmap'ed case, but restricted
  to 64bit builds.
  Fixes :bugzilla:`2675`
* :file:`libtiff/tif_read.c`: add protection against excessive memory
  allocation attempts in :c:func:`TIFFReadDirEntryArray` on short files.
  Effective for mmap'ed case. And non-mmap'ed case, but restricted
  to 64bit builds.
  Fixes :bugzilla:`2675`
* :file:`libtiff/tif_luv.c`: :c:func:`LogLuvInitState`: avoid excessive memory
  allocation when ``RowsPerStrip`` tag is missing.
  Fixes :oss-fuzz:`2683`
  Credit to OSS-Fuzz
* :file:`libtiff/tif_getimage.c`: :c:func:`gtTileContig` and :c:func:`gtTileSeparate`:
  properly break from loops on error when ``stoponerr`` is set, instead
  of going on iterating on row based loop.
* :file:`libtiff/tif_getimage.c`: fix fromskew computation when to-be-skipped
  pixel number is not a multiple of the horizontal subsampling, and
  also in some other cases. Impact ``putcontig8bitYCbCr44tile``,
  ``putcontig8bitYCbCr42tile``, ``putcontig8bitYCbCr41tile``,
  ``putcontig8bitYCbCr21tile`` and ``putcontig8bitYCbCr12tile``.
  Fixes :bugzilla:`2637` (discovered by Agostino Sarubbo)
  and :oss-fuzz:`2691` (credit to OSS Fuzz)
* :file:`libtiff/tif_luv.c`: further reduce memory requirements for temporary
  buffer when ``RowsPerStrip >= image_length`` in :c:func:`LogLuvInitState` and
  :c:func:`LogL16InitState`.
  Fixes :oss-fuzz:`2700`
  Credit to OSS Fuzz
* :file:`libtiff/tif_dirwrite.c`: replace assertion related to not finding the
  ``SubIFD`` tag by runtime check (in :c:func:`TIFFWriteDirectorySec`)
  Fixes :bugzilla:`2727`
  Reported by team OWL337
* :file:`libtiff/tif_dirwrite.c`: replace assertion to tag value not fitting
  on :c:type:`uint32` when selecting the value of ``SubIFD`` tag by runtime check
  (in :c:func:`TIFFWriteDirectoryTagSubifd`).
  Fixes :bugzilla:`2728`
  Reported by team OWL337
* :file:`libtiff/tif_jpeg.c`: accept reading the last strip of a JPEG compressed
  file if the codestream height is larger than the truncated height of the
  strip. Emit a warning in this situation since this is non compliant.
* :file:`libtiff/tiffiop.h`, :c:type:`tif_aux.c`: redirect :c:func:`SeekOK` macro to a :c:func:`_TIFFSeekoK`
  function that checks if the offset is not bigger than :c:macro:`INT64_MAX`, so as
  to avoid a ``-1`` error return code of :c:func:`TIFFSeekFile` to match a required
  seek to :c:macro:`UINT64_MAX`/``-1``.
  Fixes :bugzilla:`2726`
  Adapted from proposal by Nicolas Ruff.
* :file:`libtiff/tif_dirread.c`: add :c:macro:`NULL` check to avoid likely false positive
  null-pointer dereference warning by CLang Static Analyzer.
* :file:`libtiff/libtiff.def`: add :c:func:`TIFFReadRGBAStripExt` and :c:func:`TIFFReadRGBATileExt`
  Fixes :bugzilla:`2735`
* :file:`libtiff/tif_jpeg.c`: add compatibility with libjpeg-turbo 1.5.2 that
  honours ``max_memory_to_use > 0``.
  Cf `<https://github.com/libjpeg-turbo/libjpeg-turbo/issues/162>`_.
* :file:`libtiff/tif_getimage.c`: avoid floating point division by zero in
  :c:func:`initCIELabConversion`
  Fixes :oss-fuzz:`3733`
  Credit to OSS Fuzz


Tools changes
-------------

* :file:`tools/tiff2pdf.c`: prevent heap buffer overflow write in "Raw"
  mode on ``PlanarConfig=Contig`` input images.
  Fixes :bugzilla:`2715`
  Reported by team OWL337
* :file:`tools/tiffset.c`: fix setting a single value for the ``ExtraSamples`` tag
  (and other tags with variable number of values).
  So ``tiffset -s ExtraSamples 1 X``. This only worked
  when setting 2 or more values, but not just one.
* :file:`tools/fax2tiff.c` (``_FAX_Client_Data``): Pass ``FAX_Client_Data`` as the
  client data.  This client data is not used at all at the moment,
  but it makes the most sense.  Issue that the value of
  ``client_data.fd`` was passed where a pointer is expected was reported
  via email by Gerald Schade on Sun, 29 Oct 2017.
* :file:`tools/tiff2pdf.c` (``t2p_sample_realize_palette``): Fix possible
  arithmetic overflow in bounds checking code and eliminate
  comparison between signed and unsigned type.
* :file:`tools/tiff2bw.c` (:c:func:`main`): Free memory allocated in the :program:`tiff2bw`
  program.  This is in response to the report associated with
  :cve:`2017-16232` but does not solve the extremely high memory usage
  with the associated POC file.


Contributed software changes
----------------------------

None