merge into a single UTIL_isDirectory() method

refuse to compress directories
fix #1211, reported by @imba-tjd
2023-02-26 20:12:59 -08:00 · 2023-02-26 10:40:28 -08:00 · 2023-02-07 14:59:26 -08:00 · 2023-02-07 14:59:06 -08:00 · 2023-02-03 12:51:05 -08:00 · 2023-01-30 12:55:03 -05:00
51 changed files with 1385 additions and 801 deletions
@@ -0,0 +1,7 @@
+version: 2
+
+updates:
+  - package-ecosystem: github-actions
+    directory: /
+    schedule:
+      interval: weekly
@@ -4,27 +4,19 @@ This directory contains [GitHub Actions](https://github.com/features/actions) wo

 ## USAN, ASAN (`lz4-ubsan-x64`, `lz4-ubsan-x86`, `lz4-asan-x64`)

-For now, `lz4-ubsan-*` ignores the exit code of `make usan` and `make usan32`.
-Because there are several issues which may take relatively long time to resolve.
+For now, `lz4-ubsan-*` uses the `-fsanitize-recover=pointer-overflow` flag:
+there are known cases of pointer overflow arithmetic within `lz4.c` fast compression.
+These cases are not dangerous with known architecture,
+but they are not guaranteed to work by the C standard,
+which means that, in some future, some new architecture or some new compiler
+may decide to do something funny that could break this behavior.
+Hence, in anticipation, it's better to remove them.
+This has been already achieved in `lz4hc.c`.
+However, the same attempt in `lz4.c` resulted in massive speed losses,
+which is not an acceptable cost for preemptively solving a "potential future" problem
+not active anywhere today.
+Therefore, a more acceptable work-around will have to be found first.

-We'll fully enable it when we ensure `make usan` is ready for all commits and PRs.
-
-See [#983](https://github.com/lz4/lz4/pull/983) for details.
-
-
-## C Compilers (`lz4-c-compilers`)
-
- Our test doesn't use `gcc-4.5` due to installation issue of its package.  (`apt-get install gcc-4.5` fails on GH-Actions VM)
-
- Currently, the following 32bit executable tests fail with all versions of `clang`.
-  - `CC=clang-X CFLAGS='-O3' make V=1 -C tests clean test-lz4c32`
-  - `CC=clang-X CFLAGS='-O3 -mx32' make V=1 -C tests clean test-lz4c32`
-  - See [#991](https://github.com/lz4/lz4/issues/991) for details.
-
- Currently, the following 32bit executable tests fail with `gcc-11`
-  - `CC=gcc-11 CFLAGS='-O3' make V=1 -C tests clean test-lz4c32`
-  - `CC=gcc-11 CFLAGS='-O3 -mx32' make V=1 -C tests clean test-lz4c32`
-  - See [#991](https://github.com/lz4/lz4/issues/991) for details.


 ## cppcheck (`lz4-cppcheck`)
@@ -50,11 +42,7 @@ Also sometimes it reports false positives.

 # Difference with `.travis.yml`

-The following tests are not included yet.
-
- name: Compile OSS-Fuzz targets
-
-The following tests will not be included due to limitation of GH-Actions.
+The following tests are not be included due to limitation of GH-Actions.

 - name: aarch64 real-hw tests
 - name: PPC64LE real-hw tests
@@ -16,6 +16,10 @@ on: [push, pull_request]
 permissions:
  contents: read

+concurrency:
+  cancel-in-progress: true
+  group: ${{ github.workflow }}-${{ github.head_ref }}
+
 jobs:
  lz4-c-compilers:
    name: CC=${{ matrix.cc }}, ${{ matrix.os }}
@@ -56,8 +60,8 @@ jobs:
          { pkgs: 'lib32gcc-11-dev libx32gcc-11-dev',                   cc: clang,     cxx: clang++,     x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-latest, },
          { pkgs: 'clang-14  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-14,  cxx: clang++-14,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
          { pkgs: 'clang-13  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-13,  cxx: clang++-13,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
-          { pkgs: 'clang-12  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-12,  cxx: clang++-12,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
-          { pkgs: 'clang-11  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-11,  cxx: clang++-11,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
+          { pkgs: 'clang-12  lib32gcc-12-dev libx32gcc-12-dev',         cc: clang-12,  cxx: clang++-12,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
+          { pkgs: 'clang-11  lib32gcc-12-dev libx32gcc-12-dev',         cc: clang-11,  cxx: clang++-11,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
          { pkgs: 'clang-10  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-10,  cxx: clang++-10,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-20.04,  },
          { pkgs: 'clang-9   lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-9,   cxx: clang++-9,   x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-20.04,  },
          { pkgs: 'clang-8   lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-8,   cxx: clang++-8,   x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-20.04,  },
@@ -75,7 +79,7 @@ jobs:
      CXX: ${{ matrix.cxx }}
      FIXME__LZ4_CI_IGNORE : ' echo Error.  But we ignore it for now.'
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -176,7 +180,7 @@ jobs:
    name: Benchmark
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -203,7 +207,7 @@ jobs:
    name: Fuzzer test
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -224,7 +228,7 @@ jobs:
    name: LZ4 Makefile - support for standard variables
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: make standard_variables
      run: make V=1 standard_variables
@@ -234,7 +238,7 @@ jobs:
    name: LZ4 versions test
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -249,7 +253,7 @@ jobs:
    name: LZ4 inter-versions ABI test
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -264,7 +268,7 @@ jobs:
    name: LZ4 frame test
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -281,7 +285,7 @@ jobs:
    name: test different values of LZ4_MEMORY_USAGE
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: LZ4_MEMORY_USAGE
      run: make V=1 -C tests test-compile-with-lz4-memory-usage

@@ -290,7 +294,7 @@ jobs:
    name: Custom LZ4_DISTANCE_MAX
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: custom LZ4_DISTANCE_MAX; test LZ4_USER_MEMORY_FUNCTIONS
      run: |
        MOREFLAGS='-DLZ4_DISTANCE_MAX=8000' make V=1 check
@@ -306,7 +310,7 @@ jobs:
    name: Test lz4 compression on a block device
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: create a block device, compress it with lz4 # alternative : blindly use /dev/loop0, seems to always exist
      run: |
        make lz4
@@ -333,7 +337,7 @@ jobs:
    name: make cppcheck
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: apt-get install
      run: |
        sudo apt-get update
@@ -352,7 +356,7 @@ jobs:
    name: make staticAnalyze
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: apt-get install
      run: |
        sudo apt-get update
@@ -374,7 +378,7 @@ jobs:
    name: valgrind
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: apt-get install
      run: |
        sudo apt-get update
@@ -392,27 +396,20 @@ jobs:
  lz4-ubsan-x64:
    name: Linux x64 ubsan
    runs-on: ubuntu-latest
-    env:                        # Set environment variables
-      FIXME__LZ4_CI_IGNORE : ' echo Error.  But we ignore it for now.'
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: ubsan
-      #########################################################
-      # For now, we ignore the exit code of `make usan`.
-      # See "Known issues / lz4-ubsan-x64" in README.md
-      # When we'll resolve this issue, remove "|| $FIXME__LZ4_CI_IGNORE"
-      #########################################################
-      run: make V=1 clean usan MOREFLAGS='-Wcomma -Werror' || $FIXME__LZ4_CI_IGNORE
+      run: |
+        make clean
+        make V=1 usan


  lz4-ubsan-x86:
    name: Linux x86 ubsan
    runs-on: ubuntu-latest
-    env:                        # Set environment variables
-      FIXME__LZ4_CI_IGNORE : ' echo Error.  But we ignore it for now.'
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -421,21 +418,16 @@ jobs:
        sudo apt-get install lib32gcc-11-dev

    - name: ubsan32
-      #########################################################
-      # For now, we ignore the exit code of `make usan32`.
-      # See "Known issues / lz4-ubsaan-x86" in README.md.
-      # When we'll resolve this issue, remove "|| $FIXME__LZ4_CI_IGNORE"
-      #########################################################
-      run: CC=clang make V=1 clean usan32 MOREFLAGS='-Wcomma -Werror' || $FIXME__LZ4_CI_IGNORE
+      run: |
+        make clean
+        CC=clang make V=1 usan32


  lz4-asan-x64:
    name: Linux x64 ASAN
    runs-on: ubuntu-latest
-    env:                        # Set environment variables
-      FIXME__LZ4_CI_IGNORE : ' echo Error.  But we ignore it for now.'
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: setup
      run: sudo sysctl -w vm.mmap_min_addr=4096
@@ -450,7 +442,7 @@ jobs:
    name: lint unicode in ./lib/, ./tests/ and ./programs/
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: unicode lint
      run: bash ./tests/unicode_lint.sh

@@ -458,7 +450,7 @@ jobs:
    name: make examples
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: apt-get install
      run: |
        sudo apt-get update
@@ -497,7 +489,7 @@ jobs:
        dry-run: false
        sanitizer: ${{ matrix.sanitizer }}
    - name: Upload Crash
-      uses: actions/upload-artifact@v1
+      uses: actions/upload-artifact@v3
      if: failure() && steps.build.outcome == 'success'
      with:
        name: ${{ matrix.sanitizer }}-artifacts
@@ -549,7 +541,7 @@ jobs:
      XCC: ${{ matrix.xcc }}
      XEMU: ${{ matrix.xemu }}
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -590,7 +582,7 @@ jobs:
    name: macOS
    runs-on: macos-latest
    steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3

    - name: Environment info
      run: |
@@ -622,7 +614,7 @@ jobs:
    name: make
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: Environment info
      run: |
@@ -637,7 +629,7 @@ jobs:
    name: make travis-install
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: travis-install
      run: make V=1 clean travis-install
@@ -653,7 +645,7 @@ jobs:
    name: cmake
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout

    - name: Environment info
      run: |
@@ -674,7 +666,7 @@ jobs:
    name: make cmake
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
    - name: make cmake
      # V=1 for lz4 Makefile, VERBOSE=1 for cmake Makefile.
      run: make V=1 VERBOSE=1 clean cmake
@@ -685,8 +677,8 @@ jobs:
    name: Meson + Ninja
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2 # https://github.com/actions/checkout
-    - uses: actions/setup-python@v2 # https://github.com/actions/setup-python
+    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/setup-python@v4 # https://github.com/actions/setup-python
      with:
        python-version: '3.x'

@@ -703,10 +695,11 @@ jobs:
        echo && type python && which python && python --version
        echo && type meson && which meson && meson --version

-    - name: meson
+    - name: setup
      # 'run: >' replaces all newlines in the following block with spaces
      run: >
        meson setup
+        --fatal-meson-warnings
        --buildtype=debug
        -Db_lundef=false
        -Dauto_features=enabled
@@ -714,11 +707,15 @@ jobs:
        -Dcontrib=true
        -Dtests=true
        -Dexamples=true
-        contrib/meson build
+        build/meson builddir
+
+    - name: test
+      run: |
+        meson test -C builddir

    - name: staging
      run: |
-        cd build
+        cd builddir
        DESTDIR=./staging ninja install
        tree ./staging

@@ -731,7 +728,7 @@ jobs:
    name: git version tag checking for release
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
    - name: make -C tests checkTag
      if: startsWith(github.ref, 'refs/tags/v')   # If git tag name starts with 'v'
      run: |
@@ -757,7 +754,7 @@ jobs:

    runs-on: ${{ matrix.os }}
    steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3

    - name: init
      run: |
@@ -162,10 +162,10 @@ usan: clean
 	CC=$(CC) CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) test FUZZER_TIME="-T30s" NB_LOOPS=-i1

 .PHONY: usan32
-usan32: CFLAGS = -m32 -O3 -g -fsanitize=undefined
+usan32: CFLAGS = -m32 -O3 -g -fsanitize=undefined -fno-sanitize-recover=undefined -fsanitize-recover=pointer-overflow
 usan32: LDFLAGS = $(CFLAGS)
 usan32: clean
-	$(MAKE) test FUZZER_TIME="-T30s" NB_LOOPS=-i1
+	CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) V=1 test FUZZER_TIME="-T30s" NB_LOOPS=-i1

 SCANBUILD ?= scan-build
 SCANBUILD_FLAGS += --status-bugs -v --force-analyze-debug-code
@@ -9,6 +9,7 @@ api  : new experimental function `LZ4F_uncompressedUpdate()`, by @alexmohr
 cli  : `--list` works on `stdin` input, by @Low-power
 cli  : `--no-crc` does not produce (compression) nor check (decompression) checksums
 cli  : fix: `--test` and `--list` produce an error code when parsing invalid input
+cli  : fix: `--test -m` does no longer create decompressed file artifacts
 cli  : fix: support skippable frames when passed via `stdin`, reported by @davidmankin
 build: fix: Makefile respects CFLAGS directives passed via environment variable
 build: `LZ4_FREESTANDING`, new build macro for freestanding environments, by @t-mat
@@ -5,6 +5,7 @@ Projects for various integrated development environments (IDE)

 The following projects are included with the lz4 distribution:
 - `cmake` - CMake project
+- `meson` - Meson project
 - `VS2010` - Visual Studio 2010 project (which also works well with Visual Studio 2012, 2013, 2015)
 - `VS2017` - Visual Studio 2017 project

@@ -13,15 +13,19 @@

 project(
  'lz4',
-  ['c'],
+  'c',
  license: 'BSD-2-Clause-Patent AND GPL-2.0-or-later',
  default_options: [
    'c_std=c99',
    'buildtype=release',
    'warning_level=3'
  ],
-  version: 'DUMMY',
-  meson_version: '>=0.49.0'
+  version: run_command(
+    find_program('GetLz4LibraryVersion.py'),
+    '../../lib/lz4.h',
+    check: true
+  ).stdout().strip(),
+  meson_version: '>=0.58.0'
 )

 subdir('meson')
@@ -10,7 +10,7 @@

 lz4_source_root = '../../../../..'

-add_languages('cpp')
+add_languages('cpp', native: true)

 sources = files(
  lz4_source_root / 'contrib/gen_manual/gen_manual.cpp'
@@ -33,7 +33,7 @@ foreach mp : manual_pages
    output: '@0@_manual.html'.format(mp),
    command: [
      gen_manual,
-      lz4_version,
+      meson.project_version(),
      '@INPUT@',
      '@OUTPUT@',
    ],
@@ -17,41 +17,49 @@ sources = files(
  lz4_source_root / 'lib/xxhash.c'
 )

+if get_option('unstable')
+  sources += files(lz4_source_root / 'lib/lz4file.c')
+endif
+
 c_args = []

 if host_machine.system() == 'windows' and get_option('default_library') != 'static'
  c_args += '-DLZ4_DLL_EXPORT=1'
 endif

-if get_option('unstable')
-  compile_args += '-DLZ4_STATIC_LINKING_ONLY'
-  if get_option('default_library') != 'static'
-    c_args += '-DLZ4_PUBLISH_STATIC_FUNCTIONS'
-  endif
-endif
-
 liblz4 = library(
  'lz4',
  sources,
+  c_args: c_args,
  install: true,
-  version: lz4_version,
+  version: meson.project_version(),
  gnu_symbol_visibility: 'hidden'
 )

 liblz4_dep = declare_dependency(
  link_with: liblz4,
+  compile_args: compile_args,
  include_directories: include_directories(lz4_source_root / 'lib')
 )

-if get_option('tests') or get_option('programs') or get_option('examples')
-  liblz4_internal = static_library(
-    'lz4-internal',
-    objects: liblz4.extract_all_objects(recursive: true),
-    gnu_symbol_visibility: 'hidden'
-  )
+meson.override_dependency('liblz4', liblz4_dep)
+
+if get_option('tests') or get_option('programs') or get_option('examples') or get_option('ossfuzz')
+  if get_option('default_library') == 'shared'
+    liblz4_internal = static_library(
+      'lz4-internal',
+      objects: liblz4.extract_all_objects(recursive: true),
+      gnu_symbol_visibility: 'hidden'
+    )
+  elif get_option('default_library') == 'static'
+    liblz4_internal = liblz4
+  elif get_option('default_library') == 'both'
+    liblz4_internal = liblz4.get_static_lib()
+  endif

  liblz4_internal_dep = declare_dependency(
    link_with: liblz4_internal,
+    compile_args: compile_args,
    include_directories: include_directories(lz4_source_root / 'lib')
  )
 endif
@@ -61,7 +69,7 @@ pkgconfig.generate(
  name: 'lz4',
  filebase: 'liblz4',
  description: 'extremely fast lossless compression algorithm library',
-  version: lz4_version,
+  version: meson.project_version(),
  url: 'http://www.lz4.org/'
 )

@@ -73,4 +81,7 @@ install_headers(

 if get_option('default_library') != 'shared'
  install_headers(lz4_source_root / 'lib/lz4frame_static.h')
+  if get_option('unstable')
+    install_headers(lz4_source_root / 'lib/lz4file.h')
+  endif
 endif
@@ -0,0 +1,126 @@
+# #############################################################################
+# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
+# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
+# All rights reserved.
+#
+# This source code is licensed under both the BSD-style license (found in the
+# LICENSE file in the root directory of this source tree) and the GPLv2 (found
+# in the COPYING file in the root directory of this source tree).
+# #############################################################################
+
+cc = meson.get_compiler('c')
+
+fs = import('fs')
+pkgconfig = import('pkgconfig')
+
+lz4_source_root = '../../..'
+
+add_project_arguments('-DXXH_NAMESPACE=LZ4_', language: 'c')
+
+if get_option('debug')
+  add_project_arguments(cc.get_supported_arguments(
+      '-Wcast-qual',
+      '-Wcast-align',
+      '-Wshadow',
+      '-Wswitch-enum',
+      '-Wdeclaration-after-statement',
+      '-Wstrict-prototypes',
+      '-Wundef',
+      '-Wpointer-arith',
+      '-Wstrict-aliasing=1',
+      '-DLZ4_DEBUG=@0@'.format(get_option('debug-level'))
+    ),
+    language: 'c'
+  )
+endif
+
+compile_args = []
+
+if not get_option('align-test')
+  add_project_arguments('-DLZ4_ALIGN_TEST=0', language: 'c')
+endif
+
+if get_option('disable-memory-allocation')
+  if get_option('default_library') != 'static'
+    error('Memory allocation can only be disabled in static builds')
+  endif
+
+  add_project_arguments('-DLZ4_STATIC_LINKING_ONLY_DISABLE_MEMORY_ALLOCATION')
+  compile_args += '-DLZ4_STATIC_LINKING_ONLY_DISABLE_MEMORY_ALLOCATION'
+endif
+
+add_project_arguments(
+  '-DLZ4_DISTANCE_MAX=@0@'.format(get_option('distance-max')),
+  language: 'c'
+)
+compile_args += '-DLZ4_DISTANCE_MAX=@0@'.format(get_option('distance-max'))
+
+if not get_option('fast-dec-loop').auto()
+  add_project_arguments(
+    '-DLZ4_FAST_DEC_LOOP=@0@'.format(
+      get_option('fast-dec-loop').enabled() ? 1 : 0
+    ),
+    language: 'c'
+  )
+endif
+
+if get_option('force-sw-bitcount')
+  add_project_arguments('-DLZ4_FORCE_SW_BITCOUNT', language: 'c')
+endif
+
+if get_option('freestanding')
+  add_project_arguments('-DLZ4_FREESTANDING=1', language: 'c')
+  compile_args += '-DLZ4_FREESTANDING=1'
+endif
+
+if get_option('memory-usage') > 0
+  add_project_arguments(
+    '-DLZ4_MEMORY_USAGE=@0@'.format(get_option('memory-usage')),
+    language: 'c'
+  )
+  compile_args += '-DLZ4_MEMORY_USAGE=@0@'.format(get_option('memory-usage'))
+endif
+
+if get_option('unstable')
+  add_project_arguments('-DLZ4_STATIC_LINKING_ONLY', language: 'c')
+  compile_args += '-DLZ4_STATIC_LINKING_ONLY'
+  if get_option('default_library') != 'static'
+    add_project_arguments('-DLZ4_PUBLISH_STATIC_FUNCTIONS', language: 'c')
+    compile_args += '-DLZ4_PUBLISH_STATIC_FUNCTIONS'
+
+    add_project_arguments('-DLZ4F_PUBLISH_STATIC_FUNCTIONS', language: 'c')
+    compile_args += '-DLZ4F_PUBLISH_STATIC_FUNCTIONS'
+  endif
+endif
+
+if get_option('user-memory-functions')
+  add_project_arguments('-DLZ4_USER_MEMORY_FUNCTIONS', language: 'c')
+endif
+
+run_env = environment()
+
+subdir('lib')
+
+if get_option('programs')
+  subdir('programs')
+else
+  lz4 = disabler()
+  lz4cat = disabler()
+  unlz4 = disabler()
+endif
+
+if get_option('tests')
+  subdir('tests')
+endif
+
+if get_option('contrib')
+  subdir('contrib')
+endif
+
+if get_option('examples')
+  subdir('examples')
+endif
+
+if get_option('ossfuzz')
+  subdir('ossfuzz')
+endif
@@ -0,0 +1,35 @@
+fuzzers = [
+  'compress_frame_fuzzer',
+  'compress_fuzzer',
+  'compress_hc_fuzzer',
+  'decompress_frame_fuzzer',
+  'decompress_fuzzer',
+  'round_trip_frame_uncompressed_fuzzer',
+  'round_trip_fuzzer',
+  'round_trip_hc_fuzzer',
+  'round_trip_stream_fuzzer'
+]
+
+c_args = cc.get_supported_arguments(
+  '-Wno-unused-function',
+  '-Wno-sign-compare',
+  '-Wno-declaration-after-statement'
+)
+
+foreach f : fuzzers
+  lib = static_library(
+    f,
+    lz4_source_root / 'ossfuzz/@0@.c'.format(f),
+    lz4_source_root / 'ossfuzz/lz4_helpers.c',
+    lz4_source_root / 'ossfuzz/fuzz_data_producer.c',
+    c_args: c_args,
+    dependencies: [liblz4_internal_dep]
+  )
+
+  executable(
+    f,
+    lz4_source_root / 'ossfuzz/standaloneengine.c',
+    link_with: lib,
+    dependencies: [liblz4_internal_dep]
+  )
+endforeach
@@ -26,6 +26,36 @@ lz4 = executable(
  install: true
 )

+lz4cat = custom_target(
+  'lz4cat',
+  input: lz4,
+  output: 'lz4cat',
+  command: [
+    'ln',
+    '--symbolic',
+    '--force',
+    fs.name(lz4.full_path()),
+    '@OUTPUT@'
+  ]
+)
+
+unlz4 = custom_target(
+  'unlz4',
+  input: lz4,
+  output: 'unlz4',
+  command: [
+    'ln',
+    '--symbolic',
+    '--force',
+    fs.name(lz4.full_path()),
+    '@OUTPUT@'
+  ]
+)
+
+meson.override_find_program('lz4', lz4)
+
+run_env.prepend('PATH', meson.current_build_dir())
+
 install_man(lz4_source_root / 'programs/lz4.1')

 if meson.version().version_compare('>=0.61.0')
@@ -0,0 +1,158 @@
+# #############################################################################
+# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
+# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
+# All rights reserved.
+#
+# This source code is licensed under both the BSD-style license (found in the
+# LICENSE file in the root directory of this source tree) and the GPLv2 (found
+# in the COPYING file in the root directory of this source tree).
+# #############################################################################
+
+lz4_source_root = '../../../..'
+
+fuzzer_time = 90
+test_exes = {
+  'abiTest': {
+    'sources': files(lz4_source_root / 'tests/abiTest.c'),
+    'test': false,
+  },
+  'checkFrame': {
+    'sources': files(lz4_source_root / 'tests/checkFrame.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+  },
+  'checkTag': {
+    'sources': files(lz4_source_root / 'tests/checkTag.c'),
+    'test': false,
+  },
+  'datagen': {
+    'sources': files(lz4_source_root / 'tests/datagencli.c'),
+    'objects': lz4.extract_objects(lz4_source_root / 'programs/datagen.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+  },
+  'decompress-partial-usingDict.c': {
+    'sources': files(lz4_source_root / 'tests/decompress-partial-usingDict.c'),
+  },
+  'decompress-partial.c': {
+    'sources': files(lz4_source_root / 'tests/decompress-partial.c'),
+  },
+  'frametest': {
+    'sources': files(lz4_source_root / 'tests/frametest.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+    'args': ['-v', '-T@0@s'.format(fuzzer_time)],
+    'test': false,
+  },
+  'freestanding': {
+    'sources': files(lz4_source_root / 'tests/freestanding.c'),
+    'c_args': ['-ffreestanding', '-Wno-unused-parameter', '-Wno-declaration-after-statement'],
+    'link_args': ['-nostdlib'],
+    'build': cc.get_id() in ['gcc', 'clang'] and
+      host_machine.system() == 'linux' and host_machine.cpu_family() == 'x86_64',
+    'override_options': ['optimization=1'],
+  },
+  'fullbench': {
+    'sources': files(lz4_source_root / 'tests/fullbench.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+    'args': ['--no-prompt', '-i1', files(lz4_source_root / 'tests/COPYING')],
+    'test': false,
+  },
+  'fuzzer': {
+    'sources': files(lz4_source_root / 'tests/fuzzer.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+    'args': ['-T@0@s'.format(fuzzer_time)],
+    'test': false,
+  },
+  'roundTripTest': {
+    'sources': files(lz4_source_root / 'tests/roundTripTest.c'),
+    'test': false,
+  },
+}
+
+targets = {}
+
+foreach e, attrs : test_exes
+  if not attrs.get('build', true)
+    targets += {e: disabler()}
+    continue
+  endif
+
+  t = executable(
+    e,
+    attrs.get('sources'),
+    c_args: attrs.get('c_args', []),
+    link_args: attrs.get('link_args', []),
+    objects: attrs.get('objects', []),
+    dependencies: [liblz4_internal_dep],
+    include_directories: attrs.get('include_directories', []),
+    install: false,
+    override_options: attrs.get('override_options', [])
+  )
+
+  targets += {e: t}
+
+  if not attrs.get('test', true)
+    continue
+  endif
+
+  test(
+    e,
+    t,
+    args: attrs.get('params', []),
+    timeout: 120
+  )
+endforeach
+
+fs = import('fs')
+
+run_env.prepend('PATH', meson.current_build_dir())
+
+test_scripts = {
+  'lz4-basic': {
+    'depends': [lz4, lz4cat, unlz4, targets['datagen']],
+  },
+  'lz4-dict': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4-contentSize': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4-fast-hugefile': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4-frame-concatenation': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4-multiple': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4-multiple-legacy': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4-opt-parser': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4-skippable': {
+    'depends': [lz4],
+  },
+  'lz4-sparse': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4-testmode': {
+    'depends': [lz4, targets['datagen']],
+  },
+  'lz4hc-hugefile': {
+    'depends': [lz4, targets['datagen']],
+  },
+}
+
+foreach s, attrs : test_scripts
+  script = find_program(lz4_source_root / 'tests/test-@0@.sh'.format(s))
+
+  test(
+    '@0@'.format(s),
+    script,
+    depends: attrs.get('depends', []),
+    workdir: fs.parent(script.full_path()),
+    env: run_env,
+    timeout: 360
+  )
+endforeach
@@ -0,0 +1,40 @@
+# #############################################################################
+# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
+# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
+# All rights reserved.
+#
+# This source code is licensed under both the BSD-style license (found in the
+# LICENSE file in the root directory of this source tree) and the GPLv2 (found
+# in the COPYING file in the root directory of this source tree).
+# #############################################################################
+
+option('align-test', type: 'boolean', value: true,
+  description: 'See LZ4_ALIGN_TEST')
+option('contrib', type: 'boolean', value: false,
+  description: 'Enable contrib')
+option('debug-level', type: 'integer', min: 0, max: 7, value: 1,
+  description: 'Enable run-time debug. See lib/lz4hc.c')
+option('disable-memory-allocation', type: 'boolean', value: false,
+  description: 'See LZ4_STATIC_LINKING_ONLY_DISABLE_MEMORY_ALLOCATION. Static builds only')
+option('distance-max', type: 'integer', min: 0, max: 65535, value: 65535,
+  description: 'See LZ4_DISTANCE_MAX')
+option('examples', type: 'boolean', value: false,
+  description: 'Enable examples')
+option('fast-dec-loop', type: 'feature', value: 'auto',
+  description: 'See LZ4_FAST_DEC_LOOP')
+option('force-sw-bitcount', type: 'boolean', value: false,
+  description: 'See LZ4_FORCE_SW_BITCOUNT')
+option('freestanding', type: 'boolean', value: false,
+  description: 'See LZ4_FREESTANDING')
+option('memory-usage', type: 'integer', min: 0, max: 20, value: 0,
+  description: 'See LZ4_MEMORY_USAGE. 0 means use the LZ4 default')
+option('ossfuzz', type: 'boolean', value: true,
+  description: 'Enable ossfuzz')
+option('programs', type: 'boolean', value: false,
+  description: 'Enable programs')
+option('tests', type: 'boolean', value: false,
+  description: 'Enable tests')
+option('unstable', type: 'boolean', value: false,
+  description: 'Expose unstable interfaces')
+option('user-memory-functions', type: 'boolean', value: false,
+  description: 'See LZ4_USER_MEMORY_FUNCTIONS')
@@ -1,67 +0,0 @@
-# #############################################################################
-# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
-# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
-# All rights reserved.
-#
-# This source code is licensed under both the BSD-style license (found in the
-# LICENSE file in the root directory of this source tree) and the GPLv2 (found
-# in the COPYING file in the root directory of this source tree).
-# #############################################################################
-
-cc = meson.get_compiler('c')
-
-pkgconfig = import('pkgconfig')
-
-lz4_source_root = '../../..'
-
-lz4_version = meson.project_version()
-
-lz4_h_file = lz4_source_root / 'lib/lz4.h'
-GetLz4LibraryVersion_py = find_program('GetLz4LibraryVersion.py')
-lz4_version = run_command(GetLz4LibraryVersion_py, lz4_h_file, check: true).stdout().strip()
-message('Project version is now: @0@'.format(lz4_version))
-
-add_project_arguments('-DXXH_NAMESPACE=LZ4_', language: 'c')
-
-if get_option('debug')
-  add_project_arguments(cc.get_supported_arguments([
-        '-Wcast-qual',
-        '-Wcast-align',
-        '-Wshadow',
-        '-Wswitch-enum',
-        '-Wdeclaration-after-statement',
-        '-Wstrict-prototypes',
-        '-Wundef',
-        '-Wpointer-arith',
-        '-Wstrict-aliasing=1',
-        '-DLZ4_DEBUG=@0@'.format(get_option('debug-level')),
-      ]
-    ),
-    language: 'c',
-  )
-endif
-
-if get_option('memory-usage') > 0
-  add_project_arguments(
-    '-DLZ4_MEMORY_USAGE=@0@'.format(get_option('memory-usage')),
-    language: 'c'
-  )
-endif
-
-subdir('lib')
-
-if get_option('programs')
-  subdir('programs')
-endif
-
-if get_option('tests')
-  subdir('tests')
-endif
-
-if get_option('contrib')
-  subdir('contrib')
-endif
-
-if get_option('examples')
-  subdir('examples')
-endif
@@ -1,52 +0,0 @@
-# #############################################################################
-# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
-# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
-# All rights reserved.
-#
-# This source code is licensed under both the BSD-style license (found in the
-# LICENSE file in the root directory of this source tree) and the GPLv2 (found
-# in the COPYING file in the root directory of this source tree).
-# #############################################################################
-
-lz4_source_root = '../../../..'
-
-exes = {
-  'fullbench': {
-    'sources': files(lz4_source_root / 'tests/fullbench.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-  },
-  'fuzzer': {
-    'sources': files(lz4_source_root / 'tests/fuzzer.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-  },
-  'frametest': {
-    'sources': files(lz4_source_root / 'tests/frametest.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-  },
-  'roundTripTest': {
-    'sources': files(lz4_source_root / 'tests/roundTripTest.c'),
-  },
-  'datagen': {
-    'sources': files(lz4_source_root / 'tests/datagencli.c'),
-    'objects': lz4.extract_objects(lz4_source_root / 'programs/datagen.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-  },
-  'checkFrame': {
-    'sources': files(lz4_source_root / 'tests/checkFrame.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-  },
-  'checkTag': {
-    'sources': files(lz4_source_root / 'tests/checkTag.c'),
-  },
-}
-
-foreach e, attrs : exes
-  executable(
-    e,
-    attrs.get('sources'),
-    objects: attrs.get('objects', []),
-    dependencies: [liblz4_internal_dep],
-    include_directories: attrs.get('include_directories', []),
-    install: false
-  )
-endforeach
@@ -1,24 +0,0 @@
-# #############################################################################
-# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
-# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
-# All rights reserved.
-#
-# This source code is licensed under both the BSD-style license (found in the
-# LICENSE file in the root directory of this source tree) and the GPLv2 (found
-# in the COPYING file in the root directory of this source tree).
-# #############################################################################
-
-option('debug-level', type: 'integer', min: 0, max: 7, value: 1,
-  description: 'Enable run-time debug. See lib/lz4hc.c')
-option('unstable', type: 'boolean', value: false,
-  description: 'Expose unstable interfaces')
-option('programs', type: 'boolean', value: false,
-  description: 'Enable programs build')
-option('tests', type: 'boolean', value: false,
-  description: 'Enable tests build')
-option('contrib', type: 'boolean', value: false,
-  description: 'Enable contrib build')
-option('examples', type: 'boolean', value: false,
-  description: 'Enable examples build')
-option('memory-usage', type: 'integer', min: 0, value: 0,
-  description: 'See LZ4_MEMORY_USAGE. 0 means use the LZ4 default')
@@ -1,5 +1,5 @@
 name: lz4
-version: 1.9.3 
+version: 1.9.4 
 summary: Extremely Fast Compression algorithm 
 description: >
    LZ4 is lossless compression algorithm, providing compression
@@ -20,6 +20,7 @@ description: >
    LZ4 library is provided as open-source software using BSD 2-Clause license.
 confinement: strict
 grade: stable
+base: core20

 apps:
  lz4:
@@ -16,7 +16,7 @@ Distribution of this document is unlimited.

 ### Version

-1.6.2 (12/08/2020)
+1.6.3 (12/09/2022)


 Introduction
@@ -61,9 +61,9 @@ and associated error message explaining which parameter is unsupported.
 General Structure of LZ4 Frame format
 -------------------------------------

-| MagicNb | F. Descriptor | Block | (...) | EndMark | C. Checksum |
-|:-------:|:-------------:| ----- | ----- | ------- | ----------- |
-| 4 bytes |  3-15 bytes   |       |       | 4 bytes | 0-4 bytes   |
+| MagicNb | F. Descriptor | Data Block | (...) | EndMark | C. Checksum |
+|:-------:|:-------------:| ---------- | ----- | ------- | ----------- |
+| 4 bytes |  3-15 bytes   |            |       | 4 bytes | 0-4 bytes   |

 __Magic Number__

@@ -269,7 +269,7 @@ The size does not include the block checksum if present.

 _Block_Size_ shall never be larger than _Block_Maximum_Size_.
 Such an outcome could potentially happen for non-compressible sources.
-In such a case, such data block must be passed using uncompressed format.
+In such a case, such data block **must** be passed using uncompressed format.

 A value of `0x00000000` is invalid, and signifies an _EndMark_ instead.
 Note that this is different from a value of `0x80000000` (highest bit set),
@@ -397,6 +397,8 @@ and trigger an error if it does not fit within acceptable range.
 Version changes
 ---------------

+1.6.3 : minor : clarify Data Block
+
 1.6.2 : clarifies specification of _EndMark_

 1.6.1 : introduced terms "LZ4 Frame Header" and "LZ4 Frame Footer"
@@ -112,17 +112,21 @@ liblz4-dll.o: liblz4-dll.rc
 	$(WINDRES) -i liblz4-dll.rc -o liblz4-dll.o

 $(LIBLZ4): $(SRCFILES) liblz4-dll.o
+ifeq ($(BUILD_SHARED),yes)
 	@echo compiling dynamic library $(LIBVER)
 	$(CC) $(FLAGS) -DLZ4_DLL_EXPORT=1 -shared $^ -o dll/$@.dll -Wl,--out-implib,dll/$(LIBLZ4_EXP)
+endif

 else   # not windows

 $(LIBLZ4): $(SRCFILES)
+ifeq ($(BUILD_SHARED),yes)
 	@echo compiling dynamic library $(LIBVER)
 	$(CC) $(FLAGS) -shared $^ -fPIC -fvisibility=hidden $(SONAME_FLAGS) -o $@
 	@echo creating versioned links
 	$(LN_SF) $@ liblz4.$(SHARED_EXT_MAJOR)
 	$(LN_SF) $@ liblz4.$(SHARED_EXT)
+endif

 endif

@@ -185,6 +189,7 @@ install: lib liblz4.pc
  ifeq ($(BUILD_STATIC),yes)
 	$(INSTALL_DATA) liblz4.a $(DESTDIR)$(libdir)/liblz4.a
 	$(INSTALL_DATA) lz4frame_static.h $(DESTDIR)$(includedir)/lz4frame_static.h
+	$(INSTALL_DATA) lz4file.h $(DESTDIR)$(includedir)/lz4file.h
  endif
  ifeq ($(BUILD_SHARED),yes)
 # Traditionally, one installs the DLLs in the bin directory as programs
@@ -220,6 +225,7 @@ uninstall:
 	$(RM) $(DESTDIR)$(includedir)/lz4hc.h
 	$(RM) $(DESTDIR)$(includedir)/lz4frame.h
 	$(RM) $(DESTDIR)$(includedir)/lz4frame_static.h
+	$(RM) $(DESTDIR)$(includedir)/lz4file.h
 	@echo lz4 libraries successfully uninstalled

 endif
@@ -100,7 +100,7 @@ The following build macro can be selected to adjust source code behavior at comp
  passed as argument to become a compression state is suitably aligned.
  This test can be disabled if it proves flaky, by setting this value to 0.

- `LZ4_USER_MEMORY_FUNCTIONS` : replace calls to `<stdlib,h>`'s `malloc()`, `calloc()` and `free()`
+- `LZ4_USER_MEMORY_FUNCTIONS` : replace calls to `<stdlib.h>`'s `malloc()`, `calloc()` and `free()`
  by user-defined functions, which must be named `LZ4_malloc()`, `LZ4_calloc()` and `LZ4_free()`.
  User functions must be available at link time.

@@ -115,6 +115,24 @@ The following build macro can be selected to adjust source code behavior at comp
  (embedded, bootloader, etc).
  For more details, see description of this macro in `lib/lz4.h`.

+- `LZ4_HEAPMODE` : Select how stateless compression functions like `LZ4_compress_default()`
+  allocate memory for their hash table,
+  in memory stack (0:default, fastest), or in memory heap (1:requires malloc()).
+
+- `LZ4HC_HEAPMODE` :  Select how stateless HC compression functions like `LZ4_compress_HC()`
+  allocate memory for their workspace:
+  in stack (0), or in heap (1:default).
+  Since workspace is rather large, stack can be inconvenient, hence heap mode is recommended.
+
+- `LZ4F_HEAPMODE` : selects how `LZ4F_compressFrame()` allocates the compression state,
+  either on stack (default, value 0) or using heap memory (value 1).
+
+
+#### Makefile variables
+
+The following `Makefile` variables can be selected to alter the profile of produced binaries :
+- `BUILD_SHARED` : generate `libzstd` dynamic library (enabled by default)
+- `BUILD_STATIC` : generate `libzstd` static library (enabled by default)


 #### Amalgamation
@@ -37,7 +37,8 @@
 **************************************/
 /*
 * LZ4_HEAPMODE :
- * Select how default compression functions will allocate memory for their hash table,
+ * Select how stateless compression functions like `LZ4_compress_default()`
+ * allocate memory for their hash table,
 * in memory stack (0:default, fastest), or in memory heap (1:requires malloc()).
 */
 #ifndef LZ4_HEAPMODE
@@ -279,7 +280,7 @@ static const int LZ4_minLength = (MFLIMIT+1);
   static int g_debuglog_enable = 1;
 #  define DEBUGLOG(l, ...) {                          \
        if ((g_debuglog_enable) && (l<=LZ4_DEBUG)) {  \
-            fprintf(stderr, __FILE__ ": ");           \
+            fprintf(stderr, __FILE__  " %i: ", __LINE__); \
            fprintf(stderr, __VA_ARGS__);             \
            fprintf(stderr, " \n");                   \
    }   }
@@ -379,14 +380,16 @@ static void LZ4_write32(void* memPtr, U32 value) { *(U32*)memPtr = value; }

 /* __pack instructions are safer, but compiler specific, hence potentially problematic for some compilers */
 /* currently only defined for gcc and icc */
-typedef union { U16 u16; U32 u32; reg_t uArch; } __attribute__((packed)) LZ4_unalign;
+typedef struct { U16 u16; } __attribute__((packed)) LZ4_unalign16;
+typedef struct { U32 u32; } __attribute__((packed)) LZ4_unalign32;
+typedef struct { reg_t uArch; } __attribute__((packed)) LZ4_unalignST;

-static U16 LZ4_read16(const void* ptr) { return ((const LZ4_unalign*)ptr)->u16; }
-static U32 LZ4_read32(const void* ptr) { return ((const LZ4_unalign*)ptr)->u32; }
-static reg_t LZ4_read_ARCH(const void* ptr) { return ((const LZ4_unalign*)ptr)->uArch; }
+static U16 LZ4_read16(const void* ptr) { return ((const LZ4_unalign16*)ptr)->u16; }
+static U32 LZ4_read32(const void* ptr) { return ((const LZ4_unalign32*)ptr)->u32; }
+static reg_t LZ4_read_ARCH(const void* ptr) { return ((const LZ4_unalignST*)ptr)->uArch; }

-static void LZ4_write16(void* memPtr, U16 value) { ((LZ4_unalign*)memPtr)->u16 = value; }
-static void LZ4_write32(void* memPtr, U32 value) { ((LZ4_unalign*)memPtr)->u32 = value; }
+static void LZ4_write16(void* memPtr, U16 value) { ((LZ4_unalign16*)memPtr)->u16 = value; }
+static void LZ4_write32(void* memPtr, U32 value) { ((LZ4_unalign32*)memPtr)->u32 = value; }

 #else  /* safe and portable access using memcpy() */

@@ -803,23 +806,19 @@ LZ4_FORCE_INLINE void LZ4_putIndexOnHash(U32 idx, U32 h, void* tableBase, tableT
    }
 }

+/* LZ4_putPosition*() : only used in byPtr mode */
 LZ4_FORCE_INLINE void LZ4_putPositionOnHash(const BYTE* p, U32 h,
-                                  void* tableBase, tableType_t const tableType,
-                            const BYTE* srcBase)
+                                  void* tableBase, tableType_t const tableType)
 {
-    switch (tableType)
-    {
-    case clearedTable: { /* illegal! */ assert(0); return; }
-    case byPtr: { const BYTE** hashTable = (const BYTE**)tableBase; hashTable[h] = p; return; }
-    case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] = (U32)(p-srcBase); return; }
-    case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] = (U16)(p-srcBase); return; }
-    }
+    const BYTE** const hashTable = (const BYTE**)tableBase;
+    assert(tableType == byPtr); (void)tableType;
+    hashTable[h] = p;
 }

-LZ4_FORCE_INLINE void LZ4_putPosition(const BYTE* p, void* tableBase, tableType_t tableType, const BYTE* srcBase)
+LZ4_FORCE_INLINE void LZ4_putPosition(const BYTE* p, void* tableBase, tableType_t tableType)
 {
    U32 const h = LZ4_hashPosition(p, tableType);
-    LZ4_putPositionOnHash(p, h, tableBase, tableType, srcBase);
+    LZ4_putPositionOnHash(p, h, tableBase, tableType);
 }

 /* LZ4_getIndexOnHash() :
@@ -844,20 +843,18 @@ LZ4_FORCE_INLINE U32 LZ4_getIndexOnHash(U32 h, const void* tableBase, tableType_
    assert(0); return 0;  /* forbidden case */
 }

-static const BYTE* LZ4_getPositionOnHash(U32 h, const void* tableBase, tableType_t tableType, const BYTE* srcBase)
+static const BYTE* LZ4_getPositionOnHash(U32 h, const void* tableBase, tableType_t tableType)
 {
-    if (tableType == byPtr) { const BYTE* const* hashTable = (const BYTE* const*) tableBase; return hashTable[h]; }
-    if (tableType == byU32) { const U32* const hashTable = (const U32*) tableBase; return hashTable[h] + srcBase; }
-    { const U16* const hashTable = (const U16*) tableBase; return hashTable[h] + srcBase; }   /* default, to ensure a return */
+    assert(tableType == byPtr); (void)tableType;
+    { const BYTE* const* hashTable = (const BYTE* const*) tableBase; return hashTable[h]; }
 }

 LZ4_FORCE_INLINE const BYTE*
 LZ4_getPosition(const BYTE* p,
-                const void* tableBase, tableType_t tableType,
-                const BYTE* srcBase)
+                const void* tableBase, tableType_t tableType)
 {
    U32 const h = LZ4_hashPosition(p, tableType);
-    return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase);
+    return LZ4_getPositionOnHash(h, tableBase, tableType);
 }

 LZ4_FORCE_INLINE void
@@ -903,7 +900,7 @@ LZ4_prepareTable(LZ4_stream_t_internal* const cctx,

 /** LZ4_compress_generic() :
 *  inlined, to ensure branches are decided at compilation time.
- *  Presumed already validated at this stage:
+ *  The following conditions are presumed already validated:
 *  - source != NULL
 *  - inputSize > 0
 */
@@ -921,10 +918,10 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(
                 const int acceleration)
 {
    int result;
-    const BYTE* ip = (const BYTE*) source;
+    const BYTE* ip = (const BYTE*)source;

    U32 const startIndex = cctx->currentOffset;
-    const BYTE* base = (const BYTE*) source - startIndex;
+    const BYTE* base = (const BYTE*)source - startIndex;
    const BYTE* lowLimit;

    const LZ4_stream_t_internal* dictCtx = (const LZ4_stream_t_internal*) cctx->dictCtx;
@@ -932,7 +929,8 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(
        dictDirective == usingDictCtx ? dictCtx->dictionary : cctx->dictionary;
    const U32 dictSize =
        dictDirective == usingDictCtx ? dictCtx->dictSize : cctx->dictSize;
-    const U32 dictDelta = (dictDirective == usingDictCtx) ? startIndex - dictCtx->currentOffset : 0;   /* make indexes in dictCtx comparable with index in current context */
+    const U32 dictDelta =
+        (dictDirective == usingDictCtx) ? startIndex - dictCtx->currentOffset : 0;   /* make indexes in dictCtx comparable with indexes in current context */

    int const maybe_extMem = (dictDirective == usingExtDict) || (dictDirective == usingDictCtx);
    U32 const prefixIdxLimit = startIndex - dictSize;   /* used when dictDirective == dictSmall */
@@ -957,11 +955,11 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(

    DEBUGLOG(5, "LZ4_compress_generic_validated: srcSize=%i, tableType=%u", inputSize, tableType);
    assert(ip != NULL);
+    if (tableType == byU16) assert(inputSize<LZ4_64Klimit);  /* Size too large (not within 64K limit) */
+    if (tableType == byPtr) assert(dictDirective==noDict);   /* only supported use case with byPtr */
    /* If init conditions are not met, we don't have to mark stream
     * as having dirty context, since no action was taken yet */
    if (outputDirective == fillOutput && maxOutputSize < 1) { return 0; } /* Impossible to store anything */
-    if ((tableType == byU16) && (inputSize>=LZ4_64Klimit)) { return 0; }  /* Size too large (not within 64K limit) */
-    if (tableType==byPtr) assert(dictDirective==noDict);      /* only supported use case with byPtr */
    assert(acceleration >= 1);

    lowLimit = (const BYTE*)source - (dictDirective == withPrefix64k ? dictSize : 0);
@@ -981,7 +979,12 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(
    if (inputSize<LZ4_minLength) goto _last_literals;        /* Input too small, no compression (all literals) */

    /* First Byte */
-    LZ4_putPosition(ip, cctx->hashTable, tableType, base);
+    {   U32 const h = LZ4_hashPosition(ip, tableType);
+        if (tableType == byPtr) {
+            LZ4_putPositionOnHash(ip, h, cctx->hashTable, byPtr);
+        } else {
+            LZ4_putIndexOnHash(startIndex, h, cctx->hashTable, tableType);
+    }   }
    ip++; forwardH = LZ4_hashPosition(ip, tableType);

    /* Main Loop */
@@ -1004,9 +1007,9 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(
                if (unlikely(forwardIp > mflimitPlusOne)) goto _last_literals;
                assert(ip < mflimitPlusOne);

-                match = LZ4_getPositionOnHash(h, cctx->hashTable, tableType, base);
+                match = LZ4_getPositionOnHash(h, cctx->hashTable, tableType);
                forwardH = LZ4_hashPosition(forwardIp, tableType);
-                LZ4_putPositionOnHash(ip, h, cctx->hashTable, tableType, base);
+                LZ4_putPositionOnHash(ip, h, cctx->hashTable, tableType);

            } while ( (match+LZ4_DISTANCE_MAX < ip)
                   || (LZ4_read32(match) != LZ4_read32(ip)) );
@@ -1204,13 +1207,19 @@ _next_match:
        if (ip >= mflimitPlusOne) break;

        /* Fill table */
-        LZ4_putPosition(ip-2, cctx->hashTable, tableType, base);
+        {   U32 const h = LZ4_hashPosition(ip-2, tableType);
+            if (tableType == byPtr) {
+                LZ4_putPositionOnHash(ip-2, h, cctx->hashTable, byPtr);
+            } else {
+                U32 const idx = (U32)((ip-2) - base);
+                LZ4_putIndexOnHash(idx, h, cctx->hashTable, tableType);
+        }   }

        /* Test next position */
        if (tableType == byPtr) {

-            match = LZ4_getPosition(ip, cctx->hashTable, tableType, base);
-            LZ4_putPosition(ip, cctx->hashTable, tableType, base);
+            match = LZ4_getPosition(ip, cctx->hashTable, tableType);
+            LZ4_putPosition(ip, cctx->hashTable, tableType);
            if ( (match+LZ4_DISTANCE_MAX >= ip)
              && (LZ4_read32(match) == LZ4_read32(ip)) )
            { token=op++; *token=0; goto _next_match; }
@@ -1224,6 +1233,7 @@ _next_match:
            if (dictDirective == usingDictCtx) {
                if (matchIndex < startIndex) {
                    /* there was no match, try the dictionary */
+                    assert(tableType == byU32);
                    matchIndex = LZ4_getIndexOnHash(h, dictCtx->hashTable, byU32);
                    match = dictBase + matchIndex;
                    lowLimit = dictionary;   /* required for match length counter */
@@ -1377,9 +1387,10 @@ int LZ4_compress_fast_extState(void* state, const char* source, char* dest, int
 */
 int LZ4_compress_fast_extState_fastReset(void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration)
 {
-    LZ4_stream_t_internal* ctx = &((LZ4_stream_t*)state)->internal_donotuse;
+    LZ4_stream_t_internal* const ctx = &((LZ4_stream_t*)state)->internal_donotuse;
    if (acceleration < 1) acceleration = LZ4_ACCELERATION_DEFAULT;
    if (acceleration > LZ4_ACCELERATION_MAX) acceleration = LZ4_ACCELERATION_MAX;
+    assert(ctx != NULL);

    if (dstCapacity >= LZ4_compressBound(srcSize)) {
        if (srcSize < LZ4_64Klimit) {
@@ -1413,17 +1424,17 @@ int LZ4_compress_fast_extState_fastReset(void* state, const char* src, char* dst
 }


-int LZ4_compress_fast(const char* source, char* dest, int inputSize, int maxOutputSize, int acceleration)
+int LZ4_compress_fast(const char* src, char* dest, int srcSize, int dstCapacity, int acceleration)
 {
    int result;
 #if (LZ4_HEAPMODE)
-    LZ4_stream_t* ctxPtr = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t));   /* malloc-calloc always properly aligned */
+    LZ4_stream_t* const ctxPtr = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t));   /* malloc-calloc always properly aligned */
    if (ctxPtr == NULL) return 0;
 #else
    LZ4_stream_t ctx;
    LZ4_stream_t* const ctxPtr = &ctx;
 #endif
-    result = LZ4_compress_fast_extState(ctxPtr, source, dest, inputSize, maxOutputSize, acceleration);
+    result = LZ4_compress_fast_extState(ctxPtr, src, dest, srcSize, dstCapacity, acceleration);

 #if (LZ4_HEAPMODE)
    FREEMEM(ctxPtr);
@@ -1432,9 +1443,9 @@ int LZ4_compress_fast(const char* source, char* dest, int inputSize, int maxOutp
 }


-int LZ4_compress_default(const char* src, char* dst, int srcSize, int maxOutputSize)
+int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity)
 {
-    return LZ4_compress_fast(src, dst, srcSize, maxOutputSize, 1);
+    return LZ4_compress_fast(src, dst, srcSize, dstCapacity, 1);
 }


@@ -1461,11 +1472,11 @@ static int LZ4_compress_destSize_extState (LZ4_stream_t* state, const char* src,
 int LZ4_compress_destSize(const char* src, char* dst, int* srcSizePtr, int targetDstSize)
 {
 #if (LZ4_HEAPMODE)
-    LZ4_stream_t* ctx = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t));   /* malloc-calloc always properly aligned */
+    LZ4_stream_t* const ctx = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t));   /* malloc-calloc always properly aligned */
    if (ctx == NULL) return 0;
 #else
    LZ4_stream_t ctxBody;
-    LZ4_stream_t* ctx = &ctxBody;
+    LZ4_stream_t* const ctx = &ctxBody;
 #endif

    int result = LZ4_compress_destSize_extState(ctx, src, dst, srcSizePtr, targetDstSize);
@@ -1540,11 +1551,11 @@ int LZ4_freeStream (LZ4_stream_t* LZ4_stream)
 #define HASH_UNIT sizeof(reg_t)
 int LZ4_loadDict (LZ4_stream_t* LZ4_dict, const char* dictionary, int dictSize)
 {
-    LZ4_stream_t_internal* dict = &LZ4_dict->internal_donotuse;
+    LZ4_stream_t_internal* const dict = &LZ4_dict->internal_donotuse;
    const tableType_t tableType = byU32;
    const BYTE* p = (const BYTE*)dictionary;
    const BYTE* const dictEnd = p + dictSize;
-    const BYTE* base;
+    U32 idx32;

    DEBUGLOG(4, "LZ4_loadDict (%i bytes from %p into %p)", dictSize, dictionary, LZ4_dict);

@@ -1567,14 +1578,15 @@ int LZ4_loadDict (LZ4_stream_t* LZ4_dict, const char* dictionary, int dictSize)
    }

    if ((dictEnd - p) > 64 KB) p = dictEnd - 64 KB;
-    base = dictEnd - dict->currentOffset;
    dict->dictionary = p;
    dict->dictSize = (U32)(dictEnd - p);
    dict->tableType = (U32)tableType;
+    idx32 = dict->currentOffset - dict->dictSize;

    while (p <= dictEnd-HASH_UNIT) {
-        LZ4_putPosition(p, dict->hashTable, tableType, base);
-        p+=3;
+        U32 const h = LZ4_hashPosition(p, tableType);
+        LZ4_putIndexOnHash(idx32, h, dict->hashTable, tableType);
+        p+=3; idx32+=3;
    }

    return (int)dict->dictSize;
@@ -1711,7 +1723,7 @@ int LZ4_compress_fast_continue (LZ4_stream_t* LZ4_stream,
 /* Hidden debug function, to force-test external dictionary mode */
 int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source, char* dest, int srcSize)
 {
-    LZ4_stream_t_internal* streamPtr = &LZ4_dict->internal_donotuse;
+    LZ4_stream_t_internal* const streamPtr = &LZ4_dict->internal_donotuse;
    int result;

    LZ4_renormDictT(streamPtr, srcSize);
@@ -1774,7 +1786,7 @@ typedef enum { decode_full_block = 0, partial_decode = 1 } earlyEnd_directive;
 * does not know end of input
 * presumes input is well formed
 * note : will consume at least one byte */
-size_t read_long_length_no_check(const BYTE** pp)
+static size_t read_long_length_no_check(const BYTE** pp)
 {
    size_t b, l = 0;
    do { b = **pp; (*pp)++; l += b; } while (b==255);
@@ -1993,6 +2005,7 @@ LZ4_decompress_generic(
        }

        /* Fast loop : decode sequences as long as output < oend-FASTLOOP_SAFE_DISTANCE */
+        DEBUGLOG(6, "using fast decode loop");
        while (1) {
            /* Main fastloop assertion: We can always wildcopy FASTLOOP_SAFE_DISTANCE */
            assert(oend - op >= FASTLOOP_SAFE_DISTANCE);
@@ -2003,7 +2016,10 @@ LZ4_decompress_generic(
            /* decode literal length */
            if (length == RUN_MASK) {
                size_t const addl = read_variable_length(&ip, iend-RUN_MASK, 1);
-                if (addl == rvl_error) { goto _output_error; }
+                if (addl == rvl_error) {
+                    DEBUGLOG(6, "error reading long literal length");
+                    goto _output_error;
+                }
                length += addl;
                if (unlikely((uptrval)(op)+length<(uptrval)(op))) { goto _output_error; } /* overflow detection */
                if (unlikely((uptrval)(ip)+length<(uptrval)(ip))) { goto _output_error; } /* overflow detection */
@@ -2026,6 +2042,7 @@ LZ4_decompress_generic(

            /* get offset */
            offset = LZ4_readLE16(ip); ip+=2;
+            DEBUGLOG(6, " offset = %zu", offset);
            match = op - offset;
            assert(match <= op);  /* overflow check */

@@ -2034,11 +2051,17 @@ LZ4_decompress_generic(

            if (length == ML_MASK) {
                size_t const addl = read_variable_length(&ip, iend - LASTLITERALS + 1, 0);
-                if (addl == rvl_error) { goto _output_error; }
+                if (addl == rvl_error) {
+                    DEBUGLOG(6, "error reading long match length");
+                    goto _output_error;
+                }
                length += addl;
                length += MINMATCH;
                if (unlikely((uptrval)(op)+length<(uptrval)op)) { goto _output_error; } /* overflow detection */
-                if ((checkOffset) && (unlikely(match + dictSize < lowPrefix))) { goto _output_error; } /* Error : offset outside buffers */
+                if ((checkOffset) && (unlikely(match + dictSize < lowPrefix))) {
+                    DEBUGLOG(6, "Error : offset outside buffers");
+                    goto _output_error;
+                }
                if (op + length >= oend - FASTLOOP_SAFE_DISTANCE) {
                    goto safe_match_copy;
                }
@@ -2062,7 +2085,10 @@ LZ4_decompress_generic(
                        continue;
            }   }   }

-            if (checkOffset && (unlikely(match + dictSize < lowPrefix))) { goto _output_error; } /* Error : offset outside buffers */
+            if ( checkOffset && (unlikely(match + dictSize < lowPrefix)) ) {
+                DEBUGLOG(6, "Error : pos=%zi, offset=%zi => outside buffers", op-lowPrefix, op-match);
+                goto _output_error;
+            }
            /* match starting within external dictionary */
            if ((dict==usingExtDict) && (match < lowPrefix)) {
                assert(dictEnd != NULL);
@@ -2071,7 +2097,8 @@ LZ4_decompress_generic(
                        DEBUGLOG(7, "partialDecoding: dictionary match, close to dstEnd");
                        length = MIN(length, (size_t)(oend-op));
                    } else {
-                        goto _output_error;  /* end-of-block condition violated */
+                        DEBUGLOG(6, "end-of-block condition violated")
+                        goto _output_error;
                }   }

                if (length <= (size_t)(lowPrefix-match)) {
@@ -2111,6 +2138,7 @@ LZ4_decompress_generic(
 #endif

        /* Main Loop : decode remaining sequences where output < FASTLOOP_SAFE_DISTANCE */
+        DEBUGLOG(6, "using safe decode loop");
        while (1) {
            assert(ip < iend);
            token = *ip++;
@@ -2418,6 +2446,7 @@ int LZ4_decompress_safe_forceExtDict(const char* source, char* dest,
                                     int compressedSize, int maxOutputSize,
                                     const void* dictStart, size_t dictSize)
 {
+    DEBUGLOG(5, "LZ4_decompress_safe_forceExtDict");
    return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
                                  decode_full_block, usingExtDict,
                                  (BYTE*)dest, (const BYTE*)dictStart, dictSize);
@@ -189,8 +189,9 @@ LZ4LIB_API const char* LZ4_versionString (void);   /**< library version string;
 LZ4LIB_API int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity);

 /*! LZ4_decompress_safe() :
- *  compressedSize : is the exact complete size of the compressed block.
- *  dstCapacity : is the size of destination buffer (which must be already allocated), presumed an upper bound of decompressed size.
+ * @compressedSize : is the exact complete size of the compressed block.
+ * @dstCapacity : is the size of destination buffer (which must be already allocated),
+ *                is an upper bound of decompressed size.
 * @return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
 *           If destination buffer is not large enough, decoding will stop and output an error code (negative value).
 *           If the source stream is detected malformed, the function will stop decoding and return a negative result.
@@ -443,11 +444,24 @@ LZ4LIB_API int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const
 LZ4LIB_API int LZ4_decoderRingBufferSize(int maxBlockSize);
 #define LZ4_DECODER_RING_BUFFER_SIZE(maxBlockSize) (65536 + 14 + (maxBlockSize))  /* for static allocation; maxBlockSize presumed valid */

-/*! LZ4_decompress_*_continue() :
- *  These decoding functions allow decompression of consecutive blocks in "streaming" mode.
- *  A block is an unsplittable entity, it must be presented entirely to a decompression function.
- *  Decompression functions only accepts one block at a time.
- *  The last 64KB of previously decoded data *must* remain available and unmodified at the memory position where they were decoded.
+/*! LZ4_decompress_safe_continue() :
+ *  This decoding function allows decompression of consecutive blocks in "streaming" mode.
+ *  The difference with the usual independent blocks is that
+ *  new blocks are allowed to find references into former blocks.
+ *  A block is an unsplittable entity, and must be presented entirely to the decompression function.
+ *  LZ4_decompress_safe_continue() only accepts one block at a time.
+ *  It's modeled after `LZ4_decompress_safe()` and behaves similarly.
+ *
+ * @LZ4_streamDecode : decompression state, tracking the position in memory of past data
+ * @compressedSize : exact complete size of one compressed block.
+ * @dstCapacity : size of destination buffer (which must be already allocated),
+ *                must be an upper bound of decompressed size.
+ * @return : number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
+ *           If destination buffer is not large enough, decoding will stop and output an error code (negative value).
+ *           If the source stream is detected malformed, the function will stop decoding and return a negative result.
+ *
+ *  The last 64KB of previously decoded data *must* remain available and unmodified
+ *  at the memory position where they were previously decoded.
 *  If less than 64KB of data has been decoded, all the data must be present.
 *
 *  Special : if decompression side sets a ring buffer, it must respect one of the following conditions :
@@ -474,10 +488,10 @@ LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode,
                        int srcSize, int dstCapacity);


-/*! LZ4_decompress_*_usingDict() :
- *  These decoding functions work the same as
- *  a combination of LZ4_setStreamDecode() followed by LZ4_decompress_*_continue()
- *  They are stand-alone, and don't need an LZ4_streamDecode_t structure.
+/*! LZ4_decompress_safe_usingDict() :
+ *  Works the same as
+ *  a combination of LZ4_setStreamDecode() followed by LZ4_decompress_safe_continue()
+ *  However, it's stateless: it doesn't need any LZ4_streamDecode_t state.
 *  Dictionary is presumed stable : it must remain accessible and unmodified during decompression.
 *  Performance tip : Decompression speed can be substantially increased
 *                    when dst == dictStart + dictSize.
@@ -487,6 +501,12 @@ LZ4_decompress_safe_usingDict(const char* src, char* dst,
                              int srcSize, int dstCapacity,
                              const char* dictStart, int dictSize);

+/*! LZ4_decompress_safe_partial_usingDict() :
+ *  Behaves the same as LZ4_decompress_safe_partial()
+ *  with the added ability to specify a memory segment for past data.
+ *  Performance tip : Decompression speed can be substantially increased
+ *                    when dst == dictStart + dictSize.
+ */
 LZ4LIB_API int
 LZ4_decompress_safe_partial_usingDict(const char* src, char* dst,
                                      int compressedSize,
@@ -247,9 +247,9 @@ LZ4F_errorCode_t LZ4F_writeOpen(LZ4_writeFile_t** lz4fWrite, FILE* fp, const LZ4
  return LZ4F_OK_NoError;
 }

-size_t LZ4F_write(LZ4_writeFile_t* lz4fWrite, void* buf, size_t size)
+size_t LZ4F_write(LZ4_writeFile_t* lz4fWrite, const void* buf, size_t size)
 {
-  LZ4_byte* p = (LZ4_byte*)buf;
+  const LZ4_byte* p = (const LZ4_byte*)buf;
  size_t remain = size;
  size_t chunk;
  size_t ret;
@@ -78,7 +78,7 @@ LZ4FLIB_STATIC_API LZ4F_errorCode_t LZ4F_writeOpen(LZ4_writeFile_t** lz4fWrite,
 * `buf` write data buffer.
 * `size` write data buffer size.
 */
-LZ4FLIB_STATIC_API size_t LZ4F_write(LZ4_writeFile_t* lz4fWrite, void* buf, size_t size);
+LZ4FLIB_STATIC_API size_t LZ4F_write(LZ4_writeFile_t* lz4fWrite, const void* buf, size_t size);

 /*! LZ4F_writeClose() :
 * Close lz4file handle.
@@ -54,8 +54,8 @@
 **************************************/
 /*
 * LZ4F_HEAPMODE :
- * Select how default compression functions will allocate memory for their hash table,
- * in memory stack (0:default, fastest), or in memory heap (1:requires malloc()).
+ * Control how LZ4F_compressFrame allocates the Compression State,
+ * either on stack (0:default, fastest), or in memory heap (1:requires malloc()).
 */
 #ifndef LZ4F_HEAPMODE
 #  define LZ4F_HEAPMODE 0
@@ -264,7 +264,7 @@ typedef struct LZ4F_cctx_s
    LZ4F_CustomMem cmem;
    LZ4F_preferences_t prefs;
    U32    version;
-    U32    cStage;
+    U32    cStage;     /* 0 : compression uninitialized ; 1 : initialized, can compress */
    const LZ4F_CDict* cdict;
    size_t maxBlockSize;
    size_t maxBufferSize;
@@ -732,7 +732,7 @@ size_t LZ4F_compressBegin_usingCDict(LZ4F_cctx* cctxPtr,
        if (cctxPtr->maxBufferSize < requiredBuffSize) {
            cctxPtr->maxBufferSize = 0;
            LZ4F_free(cctxPtr->tmpBuff, cctxPtr->cmem);
-            cctxPtr->tmpBuff = (BYTE*)LZ4F_calloc(requiredBuffSize, cctxPtr->cmem);
+            cctxPtr->tmpBuff = (BYTE*)LZ4F_malloc(requiredBuffSize, cctxPtr->cmem);
            RETURN_ERROR_IF(cctxPtr->tmpBuff == NULL, allocation_failed);
            cctxPtr->maxBufferSize = requiredBuffSize;
    }   }
@@ -1085,7 +1085,6 @@ size_t LZ4F_uncompressedUpdate(LZ4F_cctx* cctxPtr,
                               void* dstBuffer, size_t dstCapacity,
                         const void* srcBuffer, size_t srcSize,
                         const LZ4F_compressOptions_t* compressOptionsPtr) {
-    RETURN_ERROR_IF(cctxPtr->prefs.frameInfo.blockMode != LZ4F_blockIndependent, blockMode_invalid);
    return LZ4F_compressUpdateImpl(cctxPtr,
                                   dstBuffer, dstCapacity,
                                   srcBuffer, srcSize,
@@ -1176,7 +1175,6 @@ size_t LZ4F_compressEnd(LZ4F_cctx* cctxPtr,
    }

    cctxPtr->cStage = 0;   /* state is now re-usable (with identical preferences) */
-    cctxPtr->maxBufferSize = 0;  /* reuse HC context */

    if (cctxPtr->prefs.frameInfo.contentSize) {
        if (cctxPtr->prefs.frameInfo.contentSize != cctxPtr->totalInSize)
@@ -1722,10 +1720,10 @@ size_t LZ4F_decompress(LZ4F_dctx* dctx,
                    /* history management (linked blocks only)*/
                    if (dctx->frameInfo.blockMode == LZ4F_blockLinked) {
                        LZ4F_updateDict(dctx, dstPtr, sizeToCopy, dstStart, 0);
-                }   }
-
-                srcPtr += sizeToCopy;
-                dstPtr += sizeToCopy;
+                    }
+                    srcPtr += sizeToCopy;
+                    dstPtr += sizeToCopy;
+                }
                if (sizeToCopy == dctx->tmpInTarget) {   /* all done */
                    if (dctx->frameInfo.blockChecksumFlag) {
                        dctx->tmpInSize = 0;
@@ -215,9 +215,19 @@ LZ4FLIB_API int LZ4F_compressionLevel_max(void);   /* v1.8.0+ */
 LZ4FLIB_API size_t LZ4F_compressFrameBound(size_t srcSize, const LZ4F_preferences_t* preferencesPtr);

 /*! LZ4F_compressFrame() :
- *  Compress an entire srcBuffer into a valid LZ4 frame.
- *  dstCapacity MUST be >= LZ4F_compressFrameBound(srcSize, preferencesPtr).
- *  The LZ4F_preferences_t structure is optional : you can provide NULL as argument. All preferences will be set to default.
+ *  Compress srcBuffer content into an LZ4-compressed frame.
+ *  It's a one shot operation, all input content is consumed, and all output is generated.
+ *
+ *  Note : it's a stateless operation (no LZ4F_cctx state needed).
+ *  In order to reduce load on the allocator, LZ4F_compressFrame(), by default,
+ *  uses the stack to allocate space for the compression state and some table.
+ *  If this usage of the stack is too much for your application,
+ *  consider compiling `lz4frame.c` with compile-time macro LZ4F_HEAPMODE set to 1 instead.
+ *  All state allocations will use the Heap.
+ *  It also means each invocation of LZ4F_compressFrame() will trigger several internal alloc/free invocations.
+ *
+ * @dstCapacity MUST be >= LZ4F_compressFrameBound(srcSize, preferencesPtr).
+ * @preferencesPtr is optional : one can provide NULL, in which case all preferences are set to default.
 * @return : number of bytes written into dstBuffer.
 *           or an error code if it fails (can be tested using LZ4F_isError())
 */
@@ -278,7 +288,7 @@ LZ4FLIB_API LZ4F_errorCode_t LZ4F_freeCompressionContext(LZ4F_cctx* cctx);
 /*! LZ4F_compressBegin() :
 *  will write the frame header into dstBuffer.
 *  dstCapacity must be >= LZ4F_HEADER_SIZE_MAX bytes.
- * `prefsPtr` is optional : you can provide NULL as argument, all preferences will then be set to default.
+ * `prefsPtr` is optional : NULL can be provided to set all preferences to default.
 * @return : number of bytes written into dstBuffer for the header
 *           or an error code (which can be tested using LZ4F_isError())
 */
@@ -463,6 +473,11 @@ LZ4F_getFrameInfo(LZ4F_dctx* dctx,
 * `dstBuffer` can freely change between each consecutive function invocation.
 * `dstBuffer` content will be overwritten.
 *
+ *  Note: if `LZ4F_getFrameInfo()` is called before `LZ4F_decompress()`, srcBuffer must be updated to reflect
+ *  the number of bytes consumed after reading the frame header. Failure to update srcBuffer before calling
+ *  `LZ4F_decompress()` will cause decompression failure or, even worse, successful but incorrect decompression.
+ *  See the `LZ4F_getFrameInfo()` docs for details.
+ *
 * @return : an hint of how many `srcSize` bytes LZ4F_decompress() expects for next call.
 *  Schematically, it's the size of the current (or remaining) compressed block + header of next block.
 *  Respecting the hint provides some small speed benefit, because it skips intermediate buffers.
@@ -660,7 +675,7 @@ LZ4F_decompress_usingDict(LZ4F_dctx* dctxPtr,
                    const LZ4F_decompressOptions_t* decompressOptionsPtr);


-/*! Custom memory allocation :
+/*! Custom memory allocation : v1.9.4+
 *  These prototypes make it possible to pass custom allocation/free functions.
 *  LZ4F_customMem is provided at state creation time, using LZ4F_create*_advanced() listed below.
 *  All allocation/free operations will be completed using these custom variants instead of regular <stdlib.h> ones.
@@ -39,9 +39,10 @@
 ***************************************/

 /*! HEAPMODE :
- *  Select how default compression function will allocate workplace memory,
- *  in stack (0:fastest), or in heap (1:requires malloc()).
- *  Since workplace is rather large, heap mode is recommended.
+ *  Select how stateless HC compression functions like `LZ4_compress_HC()`
+ *  allocate memory for their workspace:
+ *  in stack (0:fastest), or in heap (1:default, requires malloc()).
+ *  Since workspace is rather large, heap mode is recommended.
 **/
 #ifndef LZ4HC_HEAPMODE
 #  define LZ4HC_HEAPMODE 1
@@ -85,6 +86,7 @@ typedef enum { noDictCtx, usingDictCtxHc } dictCtx_directive;
 /* Make fields passed to, and updated by LZ4HC_encodeSequence explicit */
 #define UPDATABLE(ip, op, anchor) &ip, &op, &anchor

+#define LZ4HC_HASHSIZE 4
 static U32 LZ4HC_hashPtr(const void* ptr) { return HASH_FUNCTION(LZ4_read32(ptr)); }


@@ -101,6 +103,7 @@ static void LZ4HC_init_internal (LZ4HC_CCtx_internal* hc4, const BYTE* start)
 {
    size_t const bufferSize = (size_t)(hc4->end - hc4->prefixStart);
    size_t newStartingOffset = bufferSize + hc4->dictLimit;
+    DEBUGLOG(5, "LZ4HC_init_internal");
    assert(newStartingOffset >= bufferSize);  /* check overflow */
    if (newStartingOffset > 1 GB) {
        LZ4HC_clearTables(hc4);
@@ -236,13 +239,17 @@ static int LZ4HC_protectDictEnd(U32 const dictLimit, U32 const matchIndex)
 typedef enum { rep_untested, rep_not, rep_confirmed } repeat_state_e;
 typedef enum { favorCompressionRatio=0, favorDecompressionSpeed } HCfavor_e;

-LZ4_FORCE_INLINE int
+typedef struct {
+    int off;
+    int len;
+} LZ4HC_match_t;
+
+LZ4_FORCE_INLINE LZ4HC_match_t
 LZ4HC_InsertAndGetWiderMatch (
        LZ4HC_CCtx_internal* const hc4,
        const BYTE* const ip,
        const BYTE* const iLowLimit, const BYTE* const iHighLimit,
        int longest,
-        const BYTE** matchpos,
        const BYTE** startpos,
        const int maxNbAttempts,
        const int patternAnalysis, const int chainSwap,
@@ -250,7 +257,7 @@ LZ4HC_InsertAndGetWiderMatch (
        const HCfavor_e favorDecSpeed)
 {
    U16* const chainTable = hc4->chainTable;
-    U32* const HashTable = hc4->hashTable;
+    U32* const hashTable = hc4->hashTable;
    const LZ4HC_CCtx_internal * const dictCtx = hc4->dictCtx;
    const BYTE* const prefixPtr = hc4->prefixStart;
    const U32 prefixIdx = hc4->dictLimit;
@@ -267,22 +274,26 @@ LZ4HC_InsertAndGetWiderMatch (
    U32 matchIndex;
    repeat_state_e repeat = rep_untested;
    size_t srcPatternLength = 0;
+    int offset = 0;

    DEBUGLOG(7, "LZ4HC_InsertAndGetWiderMatch");
+    assert(startpos != NULL);
+    *startpos = ip;  /* in case there is no solution */
    /* First Match */
-    LZ4HC_Insert(hc4, ip);
-    matchIndex = HashTable[LZ4HC_hashPtr(ip)];
-    DEBUGLOG(7, "First match at index %u / %u (lowestMatchIndex)",
-                matchIndex, lowestMatchIndex);
+    LZ4HC_Insert(hc4, ip);  /* insert all prior positions up to ip (excluded) */
+    matchIndex = hashTable[LZ4HC_hashPtr(ip)];
+    DEBUGLOG(7, "First candidate match for pos %u found at index %u / %u (lowestMatchIndex)",
+                ipIndex, matchIndex, lowestMatchIndex);

    while ((matchIndex>=lowestMatchIndex) && (nbAttempts>0)) {
        int matchLength=0;
        nbAttempts--;
        assert(matchIndex < ipIndex);
        if (favorDecSpeed && (ipIndex - matchIndex < 8)) {
-            /* do nothing */
+            /* do nothing:
+             * favorDecSpeed intentionally skips matches with offset < 8 */
        } else if (matchIndex >= prefixIdx) {   /* within current Prefix */
-            const BYTE* const matchPtr = prefixPtr + matchIndex - prefixIdx;
+            const BYTE* const matchPtr = prefixPtr + (matchIndex - prefixIdx);
            assert(matchPtr < ip);
            assert(longest >= 1);
            if (LZ4_read16(iLowLimit + longest - 1) == LZ4_read16(matchPtr - lookBackLength + longest - 1)) {
@@ -292,10 +303,11 @@ LZ4HC_InsertAndGetWiderMatch (
                    matchLength -= back;
                    if (matchLength > longest) {
                        longest = matchLength;
-                        *matchpos = matchPtr + back;
+                        offset = (int)(ipIndex - matchIndex);
                        *startpos = ip + back;
+                        DEBUGLOG(7, "Found match of len=%i within prefix, offset=%i, back=%i", longest, offset, -back);
            }   }   }
-        } else {   /* lowestMatchIndex <= matchIndex < dictLimit */
+        } else {   /* lowestMatchIndex <= matchIndex < dictLimit : within Ext Dict */
            const BYTE* const matchPtr = dictStart + (matchIndex - dictIdx);
            assert(matchIndex >= dictIdx);
            if ( likely(matchIndex <= prefixIdx - 4)
@@ -310,8 +322,9 @@ LZ4HC_InsertAndGetWiderMatch (
                matchLength -= back;
                if (matchLength > longest) {
                    longest = matchLength;
-                    *matchpos = prefixPtr - prefixIdx + matchIndex + back;   /* virtual pos, relative to ip, to retrieve offset */
+                    offset = (int)(ipIndex - matchIndex);
                    *startpos = ip + back;
+                    DEBUGLOG(7, "Found match of len=%i within dict, offset=%i, back=%i", longest, offset, -back);
        }   }   }

        if (chainSwap && matchLength==longest) {   /* better match => select a better chain */
@@ -344,6 +357,7 @@ LZ4HC_InsertAndGetWiderMatch (
                if (repeat == rep_untested) {
                    if ( ((pattern & 0xFFFF) == (pattern >> 16))
                      &  ((pattern & 0xFF)   == (pattern >> 24)) ) {
+                        DEBUGLOG(7, "Repeat pattern detected, char %02X", pattern >> 24);
                        repeat = rep_confirmed;
                        srcPatternLength = LZ4HC_countPattern(ip+sizeof(pattern), iHighLimit, pattern) + sizeof(pattern);
                    } else {
@@ -352,7 +366,7 @@ LZ4HC_InsertAndGetWiderMatch (
                if ( (repeat == rep_confirmed) && (matchCandidateIdx >= lowestMatchIndex)
                  && LZ4HC_protectDictEnd(prefixIdx, matchCandidateIdx) ) {
                    const int extDict = matchCandidateIdx < prefixIdx;
-                    const BYTE* const matchPtr = (extDict ? dictStart - dictIdx : prefixPtr - prefixIdx) + matchCandidateIdx;
+                    const BYTE* const matchPtr = extDict ? dictStart + (matchCandidateIdx - dictIdx) : prefixPtr + (matchCandidateIdx - prefixIdx);
                    if (LZ4_read32(matchPtr) == pattern) {  /* good candidate */
                        const BYTE* const iLimit = extDict ? dictEnd : iHighLimit;
                        size_t forwardPatternLength = LZ4HC_countPattern(matchPtr+sizeof(pattern), iLimit, pattern) + sizeof(pattern);
@@ -398,8 +412,9 @@ LZ4HC_InsertAndGetWiderMatch (
                                            if ((size_t)(ip - prefixPtr) + prefixIdx - matchIndex > LZ4_DISTANCE_MAX) break;
                                            assert(maxML < 2 GB);
                                            longest = (int)maxML;
-                                            *matchpos = prefixPtr - prefixIdx + matchIndex;   /* virtual pos, relative to ip, to retrieve offset */
+                                            offset = (int)(ipIndex - matchIndex);
                                            *startpos = ip;
+                                            DEBUGLOG(7, "Found repeat pattern match of len=%i, offset=%i", longest, offset);
                                        }
                                        {   U32 const distToNextPattern = DELTANEXTU16(chainTable, matchIndex);
                                            if (distToNextPattern > matchIndex) break;  /* avoid overflow */
@@ -421,6 +436,7 @@ LZ4HC_InsertAndGetWiderMatch (
        U32 dictMatchIndex = dictCtx->hashTable[LZ4HC_hashPtr(ip)];
        assert(dictEndOffset <= 1 GB);
        matchIndex = dictMatchIndex + lowestMatchIndex - (U32)dictEndOffset;
+        if (dictMatchIndex>0) DEBUGLOG(7, "dictEndOffset = %zu, dictMatchIndex = %u => relative matchIndex = %i", dictEndOffset, dictMatchIndex, (int)dictMatchIndex - (int)dictEndOffset);
        while (ipIndex - matchIndex <= LZ4_DISTANCE_MAX && nbAttempts--) {
            const BYTE* const matchPtr = dictCtx->prefixStart - dictCtx->dictLimit + dictMatchIndex;

@@ -434,8 +450,9 @@ LZ4HC_InsertAndGetWiderMatch (
                mlt -= back;
                if (mlt > longest) {
                    longest = mlt;
-                    *matchpos = prefixPtr - prefixIdx + matchIndex + back;
+                    offset = (int)(ipIndex - matchIndex);
                    *startpos = ip + back;
+                    DEBUGLOG(7, "found match of length %i within extDictCtx", longest);
            }   }

            {   U32 const nextOffset = DELTANEXTU16(dictCtx->chainTable, dictMatchIndex);
@@ -443,22 +460,27 @@ LZ4HC_InsertAndGetWiderMatch (
                matchIndex -= nextOffset;
    }   }   }

-    return longest;
+    {   LZ4HC_match_t md;
+        assert(longest >= 0);
+        md.len = longest;
+        md.off = offset;
+        return md;
+    }
 }

-LZ4_FORCE_INLINE int
+LZ4_FORCE_INLINE LZ4HC_match_t
 LZ4HC_InsertAndFindBestMatch(LZ4HC_CCtx_internal* const hc4,   /* Index table will be updated */
                       const BYTE* const ip, const BYTE* const iLimit,
-                       const BYTE** matchpos,
                       const int maxNbAttempts,
                       const int patternAnalysis,
                       const dictCtx_directive dict)
 {
    const BYTE* uselessPtr = ip;
+    DEBUGLOG(7, "LZ4HC_InsertAndFindBestMatch");
    /* note : LZ4HC_InsertAndGetWiderMatch() is able to modify the starting position of a match (*startpos),
     * but this won't be the case here, as we define iLowLimit==ip,
     * so LZ4HC_InsertAndGetWiderMatch() won't be allowed to search past ip */
-    return LZ4HC_InsertAndGetWiderMatch(hc4, ip, ip, iLimit, MINMATCH-1, matchpos, &uselessPtr, maxNbAttempts, patternAnalysis, 0 /*chainSwap*/, dict, favorCompressionRatio);
+    return LZ4HC_InsertAndGetWiderMatch(hc4, ip, ip, iLimit, MINMATCH-1, &uselessPtr, maxNbAttempts, patternAnalysis, 0 /*chainSwap*/, dict, favorCompressionRatio);
 }

 /* LZ4HC_encodeSequence() :
@@ -469,7 +491,7 @@ LZ4_FORCE_INLINE int LZ4HC_encodeSequence (
    BYTE** _op,
    const BYTE** _anchor,
    int matchLength,
-    const BYTE* const match,
+    int offset,
    limitedOutput_directive limit,
    BYTE* oend)
 {
@@ -490,9 +512,9 @@ LZ4_FORCE_INLINE int LZ4HC_encodeSequence (
    U32 const cost = 1 + llAdd + ll + 2 + mlAdd;
    if (start==NULL) start = anchor;  /* only works for single segment */
    /* g_debuglog_enable = (pos >= 2228) & (pos <= 2262); */
-    DEBUGLOG(6, "pos:%7u -- literals:%4u, match:%4i, offset:%5u, cost:%4u + %5u",
+    DEBUGLOG(6, "pos:%7u -- literals:%4u, match:%4i, offset:%5i, cost:%4u + %5u",
                pos,
-                (U32)(ip - anchor), matchLength, (U32)(ip-match),
+                (U32)(ip - anchor), matchLength, offset,
                cost, totalCost);
    totalCost += cost;
 #endif
@@ -520,8 +542,9 @@ LZ4_FORCE_INLINE int LZ4HC_encodeSequence (
    op += length;

    /* Encode Offset */
-    assert( (ip - match) <= LZ4_DISTANCE_MAX );   /* note : consider providing offset as a value, rather than as a pointer difference */
-    LZ4_writeLE16(op, (U16)(ip - match)); op += 2;
+    assert(offset <= LZ4_DISTANCE_MAX );
+    assert(offset > 0);
+    LZ4_writeLE16(op, (U16)(offset)); op += 2;

    /* Encode MatchLength */
    assert(matchLength >= MINMATCH);
@@ -574,127 +597,116 @@ LZ4_FORCE_INLINE int LZ4HC_compress_hashChain (
    BYTE* op = (BYTE*) dest;
    BYTE* oend = op + maxOutputSize;

-    int   ml0, ml, ml2, ml3;
    const BYTE* start0;
-    const BYTE* ref0;
-    const BYTE* ref = NULL;
    const BYTE* start2 = NULL;
-    const BYTE* ref2 = NULL;
    const BYTE* start3 = NULL;
-    const BYTE* ref3 = NULL;
+    LZ4HC_match_t m0, m1, m2, m3;
+    const LZ4HC_match_t nomatch = {0, 0};

    /* init */
+    DEBUGLOG(5, "LZ4HC_compress_hashChain (dict?=>%i)", dict);
    *srcSizePtr = 0;
    if (limit == fillOutput) oend -= LASTLITERALS;                  /* Hack for support LZ4 format restriction */
    if (inputSize < LZ4_minLength) goto _last_literals;             /* Input too small, no compression (all literals) */

    /* Main Loop */
    while (ip <= mflimit) {
-        ml = LZ4HC_InsertAndFindBestMatch(ctx, ip, matchlimit, &ref, maxNbAttempts, patternAnalysis, dict);
-        if (ml<MINMATCH) { ip++; continue; }
+        m1 = LZ4HC_InsertAndFindBestMatch(ctx, ip, matchlimit, maxNbAttempts, patternAnalysis, dict);
+        if (m1.len<MINMATCH) { ip++; continue; }

        /* saved, in case we would skip too much */
-        start0 = ip; ref0 = ref; ml0 = ml;
+        start0 = ip; m0 = m1;

 _Search2:
-        if (ip+ml <= mflimit) {
-            ml2 = LZ4HC_InsertAndGetWiderMatch(ctx,
-                            ip + ml - 2, ip + 0, matchlimit, ml, &ref2, &start2,
+        DEBUGLOG(7, "_Search2 (currently found match of size %i)", m1.len);
+        if (ip+m1.len <= mflimit) {
+            m2 = LZ4HC_InsertAndGetWiderMatch(ctx,
+                            ip + m1.len - 2, ip + 0, matchlimit, m1.len, &start2,
                            maxNbAttempts, patternAnalysis, 0, dict, favorCompressionRatio);
        } else {
-            ml2 = ml;
+            m2 = nomatch;  /* do not search further */
        }

-        if (ml2 == ml) { /* No better match => encode ML1 */
+        if (m2.len <= m1.len) { /* No better match => encode ML1 immediately */
            optr = op;
-            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, limit, oend)) goto _dest_overflow;
+            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, limit, oend)) goto _dest_overflow;
            continue;
        }

        if (start0 < ip) {   /* first match was skipped at least once */
-            if (start2 < ip + ml0) {  /* squeezing ML1 between ML0(original ML1) and ML2 */
-                ip = start0; ref = ref0; ml = ml0;  /* restore initial ML1 */
+            if (start2 < ip + m0.len) {  /* squeezing ML1 between ML0(original ML1) and ML2 */
+                ip = start0; m1 = m0;  /* restore initial Match1 */
        }   }

        /* Here, start0==ip */
        if ((start2 - ip) < 3) {  /* First Match too small : removed */
-            ml = ml2;
            ip = start2;
-            ref =ref2;
+            m1 = m2;
            goto _Search2;
        }

 _Search3:
-        /* At this stage, we have :
-        *  ml2 > ml1, and
-        *  ip1+3 <= ip2 (usually < ip1+ml1) */
        if ((start2 - ip) < OPTIMAL_ML) {
            int correction;
-            int new_ml = ml;
+            int new_ml = m1.len;
            if (new_ml > OPTIMAL_ML) new_ml = OPTIMAL_ML;
-            if (ip+new_ml > start2 + ml2 - MINMATCH) new_ml = (int)(start2 - ip) + ml2 - MINMATCH;
+            if (ip+new_ml > start2 + m2.len - MINMATCH)
+                new_ml = (int)(start2 - ip) + m2.len - MINMATCH;
            correction = new_ml - (int)(start2 - ip);
            if (correction > 0) {
                start2 += correction;
-                ref2 += correction;
-                ml2 -= correction;
+                m2.len -= correction;
            }
        }
-        /* Now, we have start2 = ip+new_ml, with new_ml = min(ml, OPTIMAL_ML=18) */

-        if (start2 + ml2 <= mflimit) {
-            ml3 = LZ4HC_InsertAndGetWiderMatch(ctx,
-                            start2 + ml2 - 3, start2, matchlimit, ml2, &ref3, &start3,
+        if (start2 + m2.len <= mflimit) {
+            m3 = LZ4HC_InsertAndGetWiderMatch(ctx,
+                            start2 + m2.len - 3, start2, matchlimit, m2.len, &start3,
                            maxNbAttempts, patternAnalysis, 0, dict, favorCompressionRatio);
        } else {
-            ml3 = ml2;
+            m3 = nomatch;  /* do not search further */
        }

-        if (ml3 == ml2) {  /* No better match => encode ML1 and ML2 */
+        if (m3.len <= m2.len) {  /* No better match => encode ML1 and ML2 */
            /* ip & ref are known; Now for ml */
-            if (start2 < ip+ml)  ml = (int)(start2 - ip);
+            if (start2 < ip+m1.len) m1.len = (int)(start2 - ip);
            /* Now, encode 2 sequences */
            optr = op;
-            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, limit, oend)) goto _dest_overflow;
+            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, limit, oend))
+                goto _dest_overflow;
            ip = start2;
            optr = op;
-            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml2, ref2, limit, oend)) {
-                ml  = ml2;
-                ref = ref2;
+            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m2.len, m2.off, limit, oend)) {
+                m1 = m2;
                goto _dest_overflow;
            }
            continue;
        }

-        if (start3 < ip+ml+3) {  /* Not enough space for match 2 : remove it */
-            if (start3 >= (ip+ml)) {  /* can write Seq1 immediately ==> Seq2 is removed, so Seq3 becomes Seq1 */
-                if (start2 < ip+ml) {
-                    int correction = (int)(ip+ml - start2);
+        if (start3 < ip+m1.len+3) {  /* Not enough space for match 2 : remove it */
+            if (start3 >= (ip+m1.len)) {  /* can write Seq1 immediately ==> Seq2 is removed, so Seq3 becomes Seq1 */
+                if (start2 < ip+m1.len) {
+                    int correction = (int)(ip+m1.len - start2);
                    start2 += correction;
-                    ref2 += correction;
-                    ml2 -= correction;
-                    if (ml2 < MINMATCH) {
+                    m2.len -= correction;
+                    if (m2.len < MINMATCH) {
                        start2 = start3;
-                        ref2 = ref3;
-                        ml2 = ml3;
+                        m2 = m3;
                    }
                }

                optr = op;
-                if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, limit, oend)) goto _dest_overflow;
+                if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, limit, oend)) goto _dest_overflow;
                ip  = start3;
-                ref = ref3;
-                ml  = ml3;
+                m1 = m3;

                start0 = start2;
-                ref0 = ref2;
-                ml0 = ml2;
+                m0 = m2;
                goto _Search2;
            }

            start2 = start3;
-            ref2 = ref3;
-            ml2 = ml3;
+            m2 = m3;
            goto _Search3;
        }

@@ -703,29 +715,29 @@ _Search3:
        * let's write the first one ML1.
        * ip & ref are known; Now decide ml.
        */
-        if (start2 < ip+ml) {
+        if (start2 < ip+m1.len) {
            if ((start2 - ip) < OPTIMAL_ML) {
                int correction;
-                if (ml > OPTIMAL_ML) ml = OPTIMAL_ML;
-                if (ip + ml > start2 + ml2 - MINMATCH) ml = (int)(start2 - ip) + ml2 - MINMATCH;
-                correction = ml - (int)(start2 - ip);
+                if (m1.len > OPTIMAL_ML) m1.len = OPTIMAL_ML;
+                if (ip + m1.len > start2 + m2.len - MINMATCH)
+                    m1.len = (int)(start2 - ip) + m2.len - MINMATCH;
+                correction = m1.len - (int)(start2 - ip);
                if (correction > 0) {
                    start2 += correction;
-                    ref2 += correction;
-                    ml2 -= correction;
+                    m2.len -= correction;
                }
            } else {
-                ml = (int)(start2 - ip);
+                m1.len = (int)(start2 - ip);
            }
        }
        optr = op;
-        if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, limit, oend)) goto _dest_overflow;
+        if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, limit, oend)) goto _dest_overflow;

        /* ML2 becomes ML1 */
-        ip = start2; ref = ref2; ml = ml2;
+        ip = start2; m1 = m2;

        /* ML3 becomes ML2 */
-        start2 = start3; ref2 = ref3; ml2 = ml3;
+        start2 = start3; m2 = m3;

        /* let's find a new ML3 */
        goto _Search3;
@@ -776,10 +788,10 @@ _dest_overflow:
            /* ll validated; now adjust match length */
            size_t const bytesLeftForMl = (size_t)(maxLitPos - (op+ll_totalCost));
            size_t const maxMlSize = MINMATCH + (ML_MASK-1) + (bytesLeftForMl * 255);
-            assert(maxMlSize < INT_MAX); assert(ml >= 0);
-            if ((size_t)ml > maxMlSize) ml = (int)maxMlSize;
-            if ((oend + LASTLITERALS) - (op + ll_totalCost + 2) - 1 + ml >= MFLIMIT) {
-                LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, notLimited, oend);
+            assert(maxMlSize < INT_MAX); assert(m1.len >= 0);
+            if ((size_t)m1.len > maxMlSize) m1.len = (int)maxMlSize;
+            if ((oend + LASTLITERALS) - (op + ll_totalCost + 2) - 1 + m1.len >= MFLIMIT) {
+                LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, notLimited, oend);
        }   }
        goto _last_literals;
    }
@@ -797,16 +809,17 @@ static int LZ4HC_compress_optimal( LZ4HC_CCtx_internal* ctx,
    const HCfavor_e favorDecSpeed);


-LZ4_FORCE_INLINE int LZ4HC_compress_generic_internal (
-    LZ4HC_CCtx_internal* const ctx,
-    const char* const src,
-    char* const dst,
-    int* const srcSizePtr,
-    int const dstCapacity,
-    int cLevel,
-    const limitedOutput_directive limit,
-    const dictCtx_directive dict
-    )
+LZ4_FORCE_INLINE int
+LZ4HC_compress_generic_internal (
+            LZ4HC_CCtx_internal* const ctx,
+            const char* const src,
+            char* const dst,
+            int* const srcSizePtr,
+            int const dstCapacity,
+            int cLevel,
+            const limitedOutput_directive limit,
+            const dictCtx_directive dict
+            )
 {
    typedef enum { lz4hc, lz4opt } lz4hc_strat_e;
    typedef struct {
@@ -830,8 +843,8 @@ LZ4_FORCE_INLINE int LZ4HC_compress_generic_internal (
        { lz4opt,16384,LZ4_OPT_NUM },  /* 12==LZ4HC_CLEVEL_MAX */
    };

-    DEBUGLOG(4, "LZ4HC_compress_generic(ctx=%p, src=%p, srcSize=%d, limit=%d)",
-                ctx, src, *srcSizePtr, limit);
+    DEBUGLOG(5, "LZ4HC_compress_generic_internal(src=%p, srcSize=%d)",
+                src, *srcSizePtr);

    if (limit == fillOutput && dstCapacity < 1) return 0;   /* Impossible to store anything */
    if ((U32)*srcSizePtr > (U32)LZ4_MAX_INPUT_SIZE) return 0;    /* Unsupported input size (too large or negative) */
@@ -965,6 +978,7 @@ int LZ4_compress_HC(const char* src, char* dst, int srcSize, int dstCapacity, in
    LZ4_streamHC_t state;
    LZ4_streamHC_t* const statePtr = &state;
 #endif
+    DEBUGLOG(5, "LZ4_compress_HC")
    cSize = LZ4_compress_HC_extStateHC(statePtr, src, dst, srcSize, dstCapacity, compressionLevel);
 #if defined(LZ4HC_HEAPMODE) && LZ4HC_HEAPMODE==1
    FREEMEM(statePtr);
@@ -1032,18 +1046,16 @@ void LZ4_resetStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr, int compressionLevel)

 void LZ4_resetStreamHC_fast (LZ4_streamHC_t* LZ4_streamHCPtr, int compressionLevel)
 {
-    DEBUGLOG(4, "LZ4_resetStreamHC_fast(%p, %d)", LZ4_streamHCPtr, compressionLevel);
-    if (LZ4_streamHCPtr->internal_donotuse.dirty) {
+    LZ4HC_CCtx_internal* const s = &LZ4_streamHCPtr->internal_donotuse;
+    DEBUGLOG(5, "LZ4_resetStreamHC_fast(%p, %d)", LZ4_streamHCPtr, compressionLevel);
+    if (s->dirty) {
        LZ4_initStreamHC(LZ4_streamHCPtr, sizeof(*LZ4_streamHCPtr));
    } else {
-        /* preserve end - prefixStart : can trigger clearTable's threshold */
-        if (LZ4_streamHCPtr->internal_donotuse.end != NULL) {
-            LZ4_streamHCPtr->internal_donotuse.end -= (uptrval)LZ4_streamHCPtr->internal_donotuse.prefixStart;
-        } else {
-            assert(LZ4_streamHCPtr->internal_donotuse.prefixStart == NULL);
-        }
-        LZ4_streamHCPtr->internal_donotuse.prefixStart = NULL;
-        LZ4_streamHCPtr->internal_donotuse.dictCtx = NULL;
+        assert(s->end >= s->prefixStart);
+        s->dictLimit += (U32)(s->end - s->prefixStart);
+        s->prefixStart = NULL;
+        s->end = NULL;
+        s->dictCtx = NULL;
    }
    LZ4_setCompressionLevel(LZ4_streamHCPtr, compressionLevel);
 }
@@ -1080,7 +1092,7 @@ int LZ4_loadDictHC (LZ4_streamHC_t* LZ4_streamHCPtr,
    }
    LZ4HC_init_internal (ctxPtr, (const BYTE*)dictionary);
    ctxPtr->end = (const BYTE*)dictionary + dictSize;
-    if (dictSize >= 4) LZ4HC_Insert (ctxPtr, ctxPtr->end-3);
+    if (dictSize >= LZ4HC_HASHSIZE) LZ4HC_Insert (ctxPtr, ctxPtr->end-3);
    return dictSize;
 }

@@ -1140,7 +1152,8 @@ LZ4_compressHC_continue_generic (LZ4_streamHC_t* LZ4_streamHCPtr,
            if (sourceEnd > dictEnd) sourceEnd = dictEnd;
            ctxPtr->lowLimit += (U32)(sourceEnd - ctxPtr->dictStart);
            ctxPtr->dictStart += (U32)(sourceEnd - ctxPtr->dictStart);
-            if (ctxPtr->dictLimit - ctxPtr->lowLimit < 4) {
+            /* invalidate dictionary is it's too small */
+            if (ctxPtr->dictLimit - ctxPtr->lowLimit < LZ4HC_HASHSIZE) {
                ctxPtr->lowLimit = ctxPtr->dictLimit;
                ctxPtr->dictStart = ctxPtr->prefixStart;
    }   }   }
@@ -1150,6 +1163,7 @@ LZ4_compressHC_continue_generic (LZ4_streamHC_t* LZ4_streamHCPtr,

 int LZ4_compress_HC_continue (LZ4_streamHC_t* LZ4_streamHCPtr, const char* src, char* dst, int srcSize, int dstCapacity)
 {
+    DEBUGLOG(5, "LZ4_compress_HC_continue");
    if (dstCapacity < LZ4_compressBound(srcSize))
        return LZ4_compressHC_continue_generic (LZ4_streamHCPtr, src, dst, &srcSize, dstCapacity, limitedOutput);
    else
@@ -1179,10 +1193,10 @@ int LZ4_saveDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, char* safeBuffer, int dictS
    if (dictSize > prefixSize) dictSize = prefixSize;
    if (safeBuffer == NULL) assert(dictSize == 0);
    if (dictSize > 0)
-        LZ4_memmove(safeBuffer, streamPtr->end - dictSize, dictSize);
+        LZ4_memmove(safeBuffer, streamPtr->end - dictSize, (size_t)dictSize);
    {   U32 const endIndex = (U32)(streamPtr->end - streamPtr->prefixStart) + streamPtr->dictLimit;
-        streamPtr->end = (const BYTE*)safeBuffer + dictSize;
-        streamPtr->prefixStart = streamPtr->end - dictSize;
+        streamPtr->end = (safeBuffer == NULL) ? NULL : (const BYTE*)safeBuffer + dictSize;
+        streamPtr->prefixStart = (const BYTE*)safeBuffer;
        streamPtr->dictLimit = endIndex - (U32)dictSize;
        streamPtr->lowLimit = endIndex - (U32)dictSize;
        streamPtr->dictStart = streamPtr->prefixStart;
@@ -1254,10 +1268,10 @@ int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char* src, c

 char* LZ4_slideInputBufferHC(void* LZ4HC_Data)
 {
-    LZ4_streamHC_t* const ctx = (LZ4_streamHC_t*)LZ4HC_Data;
-    const BYTE* bufferStart = ctx->internal_donotuse.prefixStart - ctx->internal_donotuse.dictLimit + ctx->internal_donotuse.lowLimit;
-    LZ4_resetStreamHC_fast(ctx, ctx->internal_donotuse.compressionLevel);
-    /* avoid const char * -> char * conversion warning :( */
+    LZ4HC_CCtx_internal* const s = &((LZ4_streamHC_t*)LZ4HC_Data)->internal_donotuse;
+    const BYTE* const bufferStart = s->prefixStart - s->dictLimit + s->lowLimit;
+    LZ4_resetStreamHC_fast((LZ4_streamHC_t*)LZ4HC_Data, s->compressionLevel);
+    /* ugly conversion trick, required to evade (const char*) -> (char*) cast-qual warning :( */
    return (char*)(uptrval)bufferStart;
 }

@@ -1299,10 +1313,6 @@ LZ4_FORCE_INLINE int LZ4HC_sequencePrice(int litlen, int mlen)
 }


-typedef struct {
-    int off;
-    int len;
-} LZ4HC_match_t;

 LZ4_FORCE_INLINE LZ4HC_match_t
 LZ4HC_FindLongerMatch(LZ4HC_CCtx_internal* const ctx,
@@ -1311,19 +1321,16 @@ LZ4HC_FindLongerMatch(LZ4HC_CCtx_internal* const ctx,
                      const dictCtx_directive dict,
                      const HCfavor_e favorDecSpeed)
 {
-    LZ4HC_match_t match = { 0 , 0 };
-    const BYTE* matchPtr = NULL;
+    LZ4HC_match_t const match0 = { 0 , 0 };
    /* note : LZ4HC_InsertAndGetWiderMatch() is able to modify the starting position of a match (*startpos),
     * but this won't be the case here, as we define iLowLimit==ip,
-     * so LZ4HC_InsertAndGetWiderMatch() won't be allowed to search past ip */
-    int matchLength = LZ4HC_InsertAndGetWiderMatch(ctx, ip, ip, iHighLimit, minLen, &matchPtr, &ip, nbSearches, 1 /*patternAnalysis*/, 1 /*chainSwap*/, dict, favorDecSpeed);
-    if (matchLength <= minLen) return match;
+    ** so LZ4HC_InsertAndGetWiderMatch() won't be allowed to search past ip */
+    LZ4HC_match_t md = LZ4HC_InsertAndGetWiderMatch(ctx, ip, ip, iHighLimit, minLen, &ip, nbSearches, 1 /*patternAnalysis*/, 1 /*chainSwap*/, dict, favorDecSpeed);
+    if (md.len <= minLen) return match0;
    if (favorDecSpeed) {
-        if ((matchLength>18) & (matchLength<=36)) matchLength=18;   /* favor shortcut */
+        if ((md.len>18) & (md.len<=36)) md.len=18;   /* favor shortcut */
    }
-    match.len = matchLength;
-    match.off = (int)(ip-matchPtr);
-    return match;
+    return md;
 }


@@ -1356,7 +1363,7 @@ static int LZ4HC_compress_optimal ( LZ4HC_CCtx_internal* ctx,
    BYTE* opSaved = (BYTE*) dst;
    BYTE* oend = op + dstCapacity;
    int ovml = MINMATCH;  /* overflow - last sequence */
-    const BYTE* ovref = NULL;
+    int ovoff = 0;

    /* init */
 #if defined(LZ4HC_HEAPMODE) && LZ4HC_HEAPMODE==1
@@ -1379,11 +1386,10 @@ static int LZ4HC_compress_optimal ( LZ4HC_CCtx_internal* ctx,
         if ((size_t)firstMatch.len > sufficient_len) {
             /* good enough solution : immediate encoding */
             int const firstML = firstMatch.len;
-             const BYTE* const matchPos = ip - firstMatch.off;
             opSaved = op;
-             if ( LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), firstML, matchPos, limit, oend) ) {  /* updates ip, op and anchor */
+             if ( LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), firstML, firstMatch.off, limit, oend) ) {  /* updates ip, op and anchor */
                 ovml = firstML;
-                 ovref = matchPos;
+                 ovoff = firstMatch.off;
                 goto _dest_overflow;
             }
             continue;
@@ -1557,9 +1563,9 @@ encode: /* cur, last_match_pos, best_mlen, best_off must be set */
                 assert(ml >= MINMATCH);
                 assert((offset >= 1) && (offset <= LZ4_DISTANCE_MAX));
                 opSaved = op;
-                 if ( LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ip - offset, limit, oend) ) {  /* updates ip, op and anchor */
+                 if ( LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, offset, limit, oend) ) {  /* updates ip, op and anchor */
                     ovml = ml;
-                     ovref = ip - offset;
+                     ovoff = offset;
                     goto _dest_overflow;
         }   }   }
     }  /* while (ip <= mflimit) */
@@ -1618,7 +1624,7 @@ if (limit == fillOutput) {
         if ((oend + LASTLITERALS) - (op + ll_totalCost + 2) - 1 + ovml >= MFLIMIT) {
             DEBUGLOG(6, "Space to end : %i + ml (%i)", (int)((oend + LASTLITERALS) - (op + ll_totalCost + 2) - 1), ovml);
             DEBUGLOG(6, "Before : ip = %p, anchor = %p", ip, anchor);
-             LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ovml, ovref, notLimited, oend);
+             LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ovml, ovoff, notLimited, oend);
             DEBUGLOG(6, "After : ip = %p, anchor = %p", ip, anchor);
     }   }
     goto _last_literals;
@@ -67,7 +67,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
    /* Partial decompression with small prefix. */
    LZ4_decompress_safe_partial_usingDict((char const*)dataAfterDict, dst, size,
                                  dstCapacity, dstCapacity, smallDict, smallDictSize);
-    /* Partial decompression wtih large prefix. */
+    /* Partial decompression with large prefix. */
    LZ4_decompress_safe_partial_usingDict((char const*)dataAfterDict, dst, size,
                                  dstCapacity, dstCapacity, largeDict, largeDictSize);
    free(dst);
@@ -242,6 +242,7 @@ LZ4_compressBlockStream(const struct compressionParameters* pThis,
                        int srcSize, int dstSize)
 {
    int const acceleration = (pThis->cLevel < 0) ? -pThis->cLevel + 1 : 1;
+    LZ4_compressResetStream(pThis);
    return LZ4_compress_fast_continue(pThis->LZ4_stream, src, dst, srcSize, dstSize, acceleration);
 }

@@ -250,6 +251,7 @@ LZ4_compressBlockStreamHC(const struct compressionParameters* pThis,
                          const char* src, char* dst,
                          int srcSize, int dstSize)
 {
+    LZ4_compressResetStreamHC(pThis);
    return LZ4_compress_HC_continue(pThis->LZ4_streamHC, src, dst, srcSize, dstSize);
 }

@@ -367,6 +369,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
    size_t const maxInSize = (size_t)LZ4_MAX_INPUT_SIZE / decMultiplier;
    size_t const maxDecSize = srcSize < maxInSize ? srcSize * decMultiplier : LZ4_MAX_INPUT_SIZE;
    void* const resultBuffer = malloc(maxDecSize);
+    int benchError = 0;
    U32 nbBlocks;
    struct compressionParameters compP;

@@ -463,7 +466,10 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                            &compP,
                            blockTable[blockNb].srcPtr, blockTable[blockNb].cPtr,
                            (int)blockTable[blockNb].srcSize, (int)blockTable[blockNb].cRoom);
-                        if (LZ4_isError(rSize)) END_PROCESS(1, "LZ4 compression failed");
+                        if (LZ4_isError(rSize)) {
+                            DISPLAY("LZ4 compression failed on block %u", blockNb);
+                            benchError =1 ;
+                        }
                        blockTable[blockNb].cSize = rSize;
                }   }
                {   U64 const clockSpan = UTIL_clockSpanNano(clockStart);
@@ -522,7 +528,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                                decString, blockNb, (unsigned)blockTable[blockNb].srcSize);
                            if (g_decodeOnly)
                                DISPLAY("Is input using LZ4 Frame format ? \n");
-                            END_PROCESS(2, "error during decoding");
+                            benchError = 1;
                            break;
                        }
                        blockTable[blockNb].resSize = (size_t)regenSize;
@@ -560,6 +566,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                if (crcOrig!=crcCheck) {
                    size_t u;
                    DISPLAY("\n!!! WARNING !!! %17s : Invalid Checksum : %x != %x   \n", displayName, (unsigned)crcOrig, (unsigned)crcCheck);
+                    benchError = 1;
                    for (u=0; u<srcSize; u++) {
                        if (((const BYTE*)srcBuffer)[u] != ((const BYTE*)resultBuffer)[u]) {
                            U32 segNb, bNb, pos;
@@ -598,7 +605,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
    free(blockTable);
    free(compressedBuffer);
    free(resultBuffer);
-    return 0;
+    return benchError;
 }


@@ -626,13 +633,13 @@ static size_t BMK_findMaxMem(U64 requiredMem)
 }


-static void BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
+static int BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
                            const char* displayName, int cLevel, int cLevelLast,
                            const size_t* fileSizes, unsigned nbFiles,
                            const char* dictBuf, int dictSize)
 {
    int l;
-
+    int benchError = 0;
    const char* pch = strrchr(displayName, '\\'); /* Windows */
    if (!pch) pch = strrchr(displayName, '/'); /* Linux */
    if (pch) displayName = pch+1;
@@ -645,11 +652,13 @@ static void BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
    if (cLevelLast < cLevel) cLevelLast = cLevel;

    for (l=cLevel; l <= cLevelLast; l++) {
-        BMK_benchMem(srcBuffer, benchedSize,
-                     displayName, l,
-                     fileSizes, nbFiles,
-                     dictBuf, dictSize);
+        benchError |= BMK_benchMem(
+                            srcBuffer, benchedSize,
+                            displayName, l,
+                            fileSizes, nbFiles,
+                            dictBuf, dictSize);
    }
+    return benchError;
 }


@@ -688,12 +697,13 @@ static void BMK_loadFiles(void* buffer, size_t bufferSize,
    if (totalSize == 0) END_PROCESS(12, "no data to bench");
 }

-static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
-                               int cLevel, int cLevelLast,
-                               const char* dictBuf, int dictSize)
+static int BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
+                              int cLevel, int cLevelLast,
+                              const char* dictBuf, int dictSize)
 {
    void* srcBuffer;
    size_t benchedSize;
+    int benchError = 0;
    size_t* fileSizes = (size_t*)malloc(nbFiles * sizeof(size_t));
    U64 const totalSizeToLoad = UTIL_getTotalFileSize(fileNamesTable, nbFiles);
    char mfName[20] = {0};
@@ -720,7 +730,7 @@ static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
    /* Bench */
    snprintf (mfName, sizeof(mfName), " %u files", nbFiles);
    {   const char* displayName = (nbFiles > 1) ? mfName : fileNamesTable[0];
-        BMK_benchCLevel(srcBuffer, benchedSize,
+        benchError = BMK_benchCLevel(srcBuffer, benchedSize,
                        displayName, cLevel, cLevelLast,
                        fileSizes, nbFiles,
                        dictBuf, dictSize);
@@ -729,12 +739,14 @@ static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
    /* clean up */
    free(srcBuffer);
    free(fileSizes);
+    return benchError;
 }


-static void BMK_syntheticTest(int cLevel, int cLevelLast, double compressibility,
-                              const char* dictBuf, int dictSize)
+static int BMK_syntheticTest(int cLevel, int cLevelLast, double compressibility,
+                             const char* dictBuf, int dictSize)
 {
+    int benchError = 0;
    char name[20] = {0};
    size_t benchedSize = 10000000;
    void* const srcBuffer = malloc(benchedSize);
@@ -747,10 +759,12 @@ static void BMK_syntheticTest(int cLevel, int cLevelLast, double compressibility

    /* Bench */
    snprintf (name, sizeof(name), "Synthetic %2u%%", (unsigned)(compressibility*100));
-    BMK_benchCLevel(srcBuffer, benchedSize, name, cLevel, cLevelLast, &benchedSize, 1, dictBuf, dictSize);
+    benchError = BMK_benchCLevel(srcBuffer, benchedSize, name, cLevel, cLevelLast, &benchedSize, 1, dictBuf, dictSize);

    /* clean up */
    free(srcBuffer);
+
+    return benchError;
 }


@@ -759,15 +773,16 @@ BMK_benchFilesSeparately(const char** fileNamesTable, unsigned nbFiles,
                   int cLevel, int cLevelLast,
                   const char* dictBuf, int dictSize)
 {
+    int benchError = 0;
    unsigned fileNb;
    if (cLevel > LZ4HC_CLEVEL_MAX) cLevel = LZ4HC_CLEVEL_MAX;
    if (cLevelLast > LZ4HC_CLEVEL_MAX) cLevelLast = LZ4HC_CLEVEL_MAX;
    if (cLevelLast < cLevel) cLevelLast = cLevel;

    for (fileNb=0; fileNb<nbFiles; fileNb++)
-        BMK_benchFileTable(fileNamesTable+fileNb, 1, cLevel, cLevelLast, dictBuf, dictSize);
+        benchError |= BMK_benchFileTable(fileNamesTable+fileNb, 1, cLevel, cLevelLast, dictBuf, dictSize);

-    return 0;
+    return benchError;
 }


@@ -775,6 +790,7 @@ int BMK_benchFiles(const char** fileNamesTable, unsigned nbFiles,
                   int cLevel, int cLevelLast,
                   const char* dictFileName)
 {
+    int benchError = 0;
    double const compressibility = (double)g_compressibilityDefault / 100;
    char* dictBuf = NULL;
    size_t dictSize = 0;
@@ -824,14 +840,14 @@ int BMK_benchFiles(const char** fileNamesTable, unsigned nbFiles,
    }

    if (nbFiles == 0)
-        BMK_syntheticTest(cLevel, cLevelLast, compressibility, dictBuf, (int)dictSize);
+        benchError = BMK_syntheticTest(cLevel, cLevelLast, compressibility, dictBuf, (int)dictSize);
    else {
        if (g_benchSeparately)
-            BMK_benchFilesSeparately(fileNamesTable, nbFiles, cLevel, cLevelLast, dictBuf, (int)dictSize);
+            benchError = BMK_benchFilesSeparately(fileNamesTable, nbFiles, cLevel, cLevelLast, dictBuf, (int)dictSize);
        else
-            BMK_benchFileTable(fileNamesTable, nbFiles, cLevel, cLevelLast, dictBuf, (int)dictSize);
+            benchError = BMK_benchFileTable(fileNamesTable, nbFiles, cLevel, cLevelLast, dictBuf, (int)dictSize);
    }

    free(dictBuf);
-    return 0;
+    return benchError;
 }
@@ -325,11 +325,16 @@ static FILE* LZ4IO_openSrcFile(const char* srcFileName)
        DISPLAYLEVEL(4,"Using stdin for input \n");
        f = stdin;
        SET_BINARY_MODE(stdin);
-    } else {
-        f = fopen(srcFileName, "rb");
-        if (f==NULL) DISPLAYLEVEL(1, "%s: %s \n", srcFileName, strerror(errno));
+        return f;
    }

+    if (UTIL_isDirectory(srcFileName)) {
+        DISPLAYLEVEL(1, "lz4: %s is a directory -- ignored \n", srcFileName);
+        return NULL;
+    }
+
+    f = fopen(srcFileName, "rb");
+    if (f==NULL) DISPLAYLEVEL(1, "%s: %s \n", srcFileName, strerror(errno));
    return f;
 }

@@ -337,7 +342,8 @@ static FILE* LZ4IO_openSrcFile(const char* srcFileName)
 *  prefs is writable, because sparseFileSupport might be updated.
 *  condition : `dstFileName` must be non-NULL.
 * @result : FILE* to `dstFileName`, or NULL if it fails */
-static FILE* LZ4IO_openDstFile(const char* dstFileName, const LZ4IO_prefs_t* const prefs)
+static FILE*
+LZ4IO_openDstFile(const char* dstFileName, const LZ4IO_prefs_t* const prefs)
 {
    FILE* f;
    assert(dstFileName != NULL);
@@ -381,7 +387,6 @@ static FILE* LZ4IO_openDstFile(const char* dstFileName, const LZ4IO_prefs_t* con
 }


-
 /***************************************
 *   Legacy Compression
 ***************************************/
@@ -409,8 +414,10 @@ static int LZ4IO_LZ4_compress(const char* src, char* dst, int srcSize, int dstSi
 /* LZ4IO_compressFilename_Legacy :
 * This function is intentionally "hidden" (not published in .h)
 * It generates compressed streams using the old 'legacy' format */
-int LZ4IO_compressFilename_Legacy(const char* input_filename, const char* output_filename,
-                                  int compressionlevel, const LZ4IO_prefs_t* prefs)
+int LZ4IO_compressFilename_Legacy(const char* input_filename,
+                                  const char* output_filename,
+                                  int compressionlevel,
+                                  const LZ4IO_prefs_t* prefs)
 {
    typedef int (*compress_f)(const char* src, char* dst, int srcSize, int dstSize, int cLevel);
    compress_f const compressionFunction = (compressionlevel < 3) ? LZ4IO_LZ4_compress : LZ4_compress_HC;
@@ -563,11 +570,14 @@ static void* LZ4IO_createDict(size_t* dictSize, const char* const dictFilename)
    char*  dictBuf;
    FILE* dictFile;

-    if (!circularBuf) END_PROCESS(25, "Allocation error : not enough memory for circular buffer");
-    if (!dictFilename) END_PROCESS(26, "Dictionary error : no filename provided");
+    if (!dictFilename)
+        END_PROCESS(26, "Dictionary error : no filename provided");
+    if (!circularBuf)
+        END_PROCESS(25, "Allocation error : not enough memory for circular buffer");

    dictFile = LZ4IO_openSrcFile(dictFilename);
-    if (!dictFile) END_PROCESS(27, "Dictionary error : could not open dictionary file");
+    if (!dictFile)
+        END_PROCESS(27, "Dictionary error : could not open dictionary file");

    /* opportunistically seek to the part of the file we care about.
     * If this fails it's not a problem since we'll just read everything anyways. */
@@ -1311,7 +1321,8 @@ LZ4IO_decompressSrcFile(dRess_t ress,

 static int
 LZ4IO_decompressDstFile(dRess_t ress,
-                        const char* input_filename, const char* output_filename,
+                        const char* input_filename,
+                        const char* output_filename,
                        const LZ4IO_prefs_t* const prefs)
 {
    int result;
@@ -195,13 +195,10 @@ UTIL_STATIC void* UTIL_realloc(void* ptr, size_t size)
 /*-****************************************
 *  String functions
 ******************************************/
-/*
- * A modified version of realloc().
- * If UTIL_realloc() fails the original block is freed.
-*/
+/* supports a==NULL or b==NULL */
 UTIL_STATIC int UTIL_sameString(const char* a, const char* b)
 {
-    assert(a!=NULL && b!=NULL);  /* unsupported scenario */
+    assert(a != NULL || b != NULL);  /* unsupported scenario */
    if (a==NULL) return 0;
    if (b==NULL) return 0;
    return !strcmp(a,b);
@@ -412,7 +409,6 @@ UTIL_STATIC int UTIL_getFileStat(const char* infilename, stat_t *statbuf)
    return 1;
 }

-
 UTIL_STATIC int UTIL_isRegFD(int fd)
 {
    stat_t statbuf;
@@ -423,26 +419,25 @@ UTIL_STATIC int UTIL_isRegFD(int fd)
    return UTIL_getFDStat(fd, &statbuf); /* Only need to know whether it is a regular file */
 }

-
 UTIL_STATIC int UTIL_isRegFile(const char* infilename)
 {
    stat_t statbuf;
    return UTIL_getFileStat(infilename, &statbuf); /* Only need to know whether it is a regular file */
 }

-
-UTIL_STATIC U32 UTIL_isDirectory(const char* infilename)
+UTIL_STATIC int UTIL_isDirectory(const char* infilename)
 {
-    int r;
    stat_t statbuf;
+    int r;
 #if defined(_MSC_VER)
    r = _stat64(infilename, &statbuf);
-    if (!r && (statbuf.st_mode & _S_IFDIR)) return 1;
+    if (r) return 0;
+    return (statbuf.st_mode & S_IFDIR);
 #else
    r = stat(infilename, &statbuf);
-    if (!r && S_ISDIR(statbuf.st_mode)) return 1;
+    if (r) return 0;
+    return (S_ISDIR(statbuf.st_mode));
 #endif
-    return 0;
 }


@@ -115,9 +115,6 @@ decompress-partial: lz4.o decompress-partial.c
 decompress-partial-usingDict: lz4.o decompress-partial-usingDict.c
 	$(CC) $(FLAGS) $^ -o $@$(EXT)

-freestanding: freestanding.c
-	$(CC) -ffreestanding -nostdlib $^ -o $@$(EXT)
-
 .PHONY: clean
 clean:
 	@$(MAKE) -C $(LZ4DIR) $@ > $(VOID)
@@ -173,6 +170,7 @@ endif
 CAT:=cat
 DD:=dd
 DATAGEN:=./datagen
+PATH:=../programs:$(shell pwd):$(PATH)

 .PHONY: list
 list:
@@ -211,312 +209,55 @@ test-compile-with-lz4-memory-usage:
 # All FPREFIX must start with `tmp`, for `make clean`
 # All tests must clean their temporary test files on successful completion,
 # and only their test files : do not employ sweeping statements such `rm tmp*` or `rm *.lz4`
-test-lz4-sparse: FPREFIX = tmp-tls
 test-lz4-sparse: lz4 datagen
 	@echo "\n ---- test sparse file support ----"
-	$(DATAGEN) -g5M  -P100 > $(FPREFIX)dg5M
-	$(LZ4) -B4D $(FPREFIX)dg5M -c | $(LZ4) -dv --sparse > $(FPREFIX)cB4
-	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)cB4
-	$(LZ4) -B5D $(FPREFIX)dg5M -c | $(LZ4) -dv --sparse > $(FPREFIX)cB5
-	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)cB5
-	$(LZ4) -B6D $(FPREFIX)dg5M -c | $(LZ4) -dv --sparse > $(FPREFIX)cB6
-	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)cB6
-	$(LZ4) -B7D $(FPREFIX)dg5M -c | $(LZ4) -dv --sparse > $(FPREFIX)cB7
-	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)cB7
-	$(LZ4) $(FPREFIX)dg5M -c | $(LZ4) -dv --no-sparse > $(FPREFIX)nosparse
-	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)nosparse
-	ls -ls $(FPREFIX)*
-	$(DATAGEN) -s1 -g1200007 -P100 | $(LZ4) | $(LZ4) -dv --sparse > $(FPREFIX)odd   # Odd size file (to generate non-full last block)
-	$(DATAGEN) -s1 -g1200007 -P100 | $(DIFF) -s - $(FPREFIX)odd
-	ls -ls $(FPREFIX)odd
-	@$(RM) $(FPREFIX)*
-	@echo "\n Compatibility with Console :"
-	echo "Hello World 1 !" | $(LZ4) | $(LZ4) -d -c
-	echo "Hello World 2 !" | $(LZ4) | $(LZ4) -d | $(CAT)
-	echo "Hello World 3 !" | $(LZ4) --no-frame-crc | $(LZ4) -d -c
-	@echo "\n Compatibility with Append :"
-	$(DATAGEN) -P100 -g1M > $(FPREFIX)dg1M
-	$(CAT) $(FPREFIX)dg1M $(FPREFIX)dg1M > $(FPREFIX)2M
-	$(LZ4) -B5 -v $(FPREFIX)dg1M $(FPREFIX)c
-	$(LZ4) -d -v $(FPREFIX)c $(FPREFIX)r
-	$(LZ4) -d -v $(FPREFIX)c -c >> $(FPREFIX)r
-	ls -ls $(FPREFIX)*
-	$(DIFF) $(FPREFIX)2M $(FPREFIX)r
-	@$(RM) $(FPREFIX)*
+	./test-lz4-sparse.sh

-test-lz4-contentSize: FPREFIX = tmp-lzc
 test-lz4-contentSize: lz4 datagen
 	@echo "\n ---- test original size support ----"
-	$(DATAGEN) -g15M > $(FPREFIX)
-	$(LZ4) -v $(FPREFIX) -c | $(LZ4) -t
-	$(LZ4) -v --content-size $(FPREFIX) -c | $(LZ4) -d > $(FPREFIX)-dup
-	$(DIFF) $(FPREFIX) $(FPREFIX)-dup
-	$(LZ4) -f $(FPREFIX) -c > $(FPREFIX).lz4 # compressed with content size
-	$(LZ4) --content-size $(FPREFIX) -c > $(FPREFIX)-wcz.lz4
-	! $(DIFF) $(FPREFIX).lz4 $(FPREFIX)-wcz.lz4  # must differ, due to content size
-	$(LZ4) --content-size < $(FPREFIX) > $(FPREFIX)-wcz2.lz4 # can determine content size because stdin is just a file
-	$(DIFF) $(FPREFIX)-wcz.lz4 $(FPREFIX)-wcz2.lz4  # both must contain content size
-	$(CAT) $(FPREFIX) | $(LZ4) > $(FPREFIX)-ncz.lz4
-	$(DIFF) $(FPREFIX).lz4 $(FPREFIX)-ncz.lz4  # both don't have content size
-	$(CAT) $(FPREFIX) | $(LZ4) --content-size > $(FPREFIX)-ncz2.lz4 # can't determine content size
-	$(DIFF) $(FPREFIX).lz4 $(FPREFIX)-ncz2.lz4  # both don't have content size
-	@$(RM) $(FPREFIX)*
+	./test-lz4-contentSize.sh

-test-lz4-frame-concatenation: FPREFIX = tmp-lfc
 test-lz4-frame-concatenation: lz4 datagen
 	@echo "\n ---- test frame concatenation ----"
-	@echo -n > $(FPREFIX)-empty
-	@echo hi > $(FPREFIX)-nonempty
-	$(CAT) $(FPREFIX)-nonempty $(FPREFIX)-empty $(FPREFIX)-nonempty > $(FPREFIX)-src
-	$(LZ4) -zq $(FPREFIX)-empty -c > $(FPREFIX)-empty.lz4
-	$(LZ4) -zq $(FPREFIX)-nonempty -c > $(FPREFIX)-nonempty.lz4
-	$(CAT) $(FPREFIX)-nonempty.lz4 $(FPREFIX)-empty.lz4 $(FPREFIX)-nonempty.lz4 > $(FPREFIX)-concat.lz4
-	$(LZ4) -d $(FPREFIX)-concat.lz4 -c > $(FPREFIX)-result
-	$(CMP) $(FPREFIX)-src $(FPREFIX)-result
-	@$(RM) $(FPREFIX)*
-	@echo frame concatenation test completed
+	./test-lz4-frame-concatenation.sh

-test-lz4-multiple: FPREFIX = tmp-tml
 test-lz4-multiple: lz4 datagen
 	@echo "\n ---- test multiple files ----"
-	@$(DATAGEN) -s1        > $(FPREFIX)1 2> $(VOID)
-	@$(DATAGEN) -s2 -g100K > $(FPREFIX)2 2> $(VOID)
-	@$(DATAGEN) -s3 -g200K > $(FPREFIX)3 2> $(VOID)
-	# compress multiple files : one .lz4 per source file
-	$(LZ4) -f -m $(FPREFIX)*
-	test -f $(FPREFIX)1.lz4
-	test -f $(FPREFIX)2.lz4
-	test -f $(FPREFIX)3.lz4
-	# decompress multiple files : one output file per .lz4
-	mv $(FPREFIX)1 $(FPREFIX)1-orig
-	mv $(FPREFIX)2 $(FPREFIX)2-orig
-	mv $(FPREFIX)3 $(FPREFIX)3-orig
-	$(LZ4) -d -f -m $(FPREFIX)*.lz4
-	$(CMP) $(FPREFIX)1 $(FPREFIX)1-orig   # must be identical
-	$(CMP) $(FPREFIX)2 $(FPREFIX)2-orig
-	$(CMP) $(FPREFIX)3 $(FPREFIX)3-orig
-	# compress multiple files into stdout
-	$(CAT) $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 > $(FPREFIX)-concat1
-	$(RM) $(FPREFIX)*.lz4
-	$(LZ4) -m $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3 -c > $(FPREFIX)-concat2
-	test ! -f $(FPREFIX)1.lz4  # must not create .lz4 artefact
-	$(CMP) $(FPREFIX)-concat1 $(FPREFIX)-concat2  # must be equivalent
-	# decompress multiple files into stdout
-	$(RM) $(FPREFIX)-concat1 $(FPREFIX)-concat2
-	$(LZ4) -f -m $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3   # generate .lz4 to decompress
-	$(CAT) $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3 > $(FPREFIX)-concat1   # create concatenated reference
-	$(RM) $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3
-	$(LZ4) -d -m $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 -c > $(FPREFIX)-concat2
-	test ! -f $(FPREFIX)1  # must not create file artefact
-	$(CMP) $(FPREFIX)-concat1 $(FPREFIX)-concat2  # must be equivalent
-	# compress multiple files, one of which is absent (must fail)
-	! $(LZ4) -f -m $(FPREFIX)-concat1 notHere $(FPREFIX)-concat2  # must fail : notHere not present
-	# test lz4-compressed file
-	$(LZ4) -tm $(FPREFIX)-concat1.lz4
-	$(LZ4) -tm $(FPREFIX)-concat1.lz4 $(FPREFIX)-concat2.lz4
-	# test multiple lz4 files, one of which is absent (must fail)
-	! $(LZ4) -tm $(FPREFIX)-concat1.lz4 notHere.lz4 $(FPREFIX)-concat2.lz4
-	@$(RM) $(FPREFIX)*
+	./test-lz4-multiple.sh

-test-lz4-multiple-legacy: FPREFIX = tmp-lml
 test-lz4-multiple-legacy: lz4 datagen
 	@echo "\n ---- test multiple files (Legacy format) ----"
-	@$(DATAGEN) -s1        > $(FPREFIX)1 2> $(VOID)
-	@$(DATAGEN) -s2 -g100K > $(FPREFIX)2 2> $(VOID)
-	@$(DATAGEN) -s3 -g200K > $(FPREFIX)3 2> $(VOID)
-	# compress multiple files using legacy format: one .lz4 per source file
-	$(LZ4) -f -l -m $(FPREFIX)*
-	test -f $(FPREFIX)1.lz4
-	test -f $(FPREFIX)2.lz4
-	test -f $(FPREFIX)3.lz4
-	# decompress multiple files compressed using legacy format: one output file per .lz4
-	mv $(FPREFIX)1 $(FPREFIX)1-orig
-	mv $(FPREFIX)2 $(FPREFIX)2-orig
-	mv $(FPREFIX)3 $(FPREFIX)3-orig
-	$(LZ4) -d -f -m $(FPREFIX)*.lz4
-	$(LZ4) -l -d -f -m $(FPREFIX)*.lz4 # -l mustn't impact -d option
-	$(CMP) $(FPREFIX)1 $(FPREFIX)1-orig   # must be identical
-	$(CMP) $(FPREFIX)2 $(FPREFIX)2-orig
-	$(CMP) $(FPREFIX)3 $(FPREFIX)3-orig
-	# compress multiple files into stdout using legacy format
-	$(CAT) $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 > $(FPREFIX)-concat1
-	$(RM) $(FPREFIX)*.lz4
-	$(LZ4) -l -m $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3 -c > $(FPREFIX)-concat2
-	test ! -f $(FPREFIX)1.lz4  # must not create .lz4 artefact
-	$(CMP) $(FPREFIX)-concat1 $(FPREFIX)-concat2  # must be equivalent
-	# # # decompress multiple files into stdout using legacy format
-	$(RM) $(FPREFIX)-concat1 $(FPREFIX)-concat2
-	$(LZ4) -l -f -m $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3   # generate .lz4 to decompress
-	$(CAT) $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3 > $(FPREFIX)-concat1   # create concatenated reference
-	$(RM) $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3
-	$(LZ4) -d -m $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 -c > $(FPREFIX)-concat2
-	$(LZ4) -d -l -m $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 -c > $(FPREFIX)-concat2 # -l mustn't impact option -d
-	test ! -f $(FPREFIX)1  # must not create file artefact
-	$(CMP) $(FPREFIX)-concat1 $(FPREFIX)-concat2  # must be equivalent
-	# # # compress multiple files, one of which is absent (must fail)
-	! $(LZ4) -f -l -m $(FPREFIX)-concat1 notHere-legacy $(FPREFIX)-concat2  # must fail : notHere-legacy not present
-	@$(RM) $(FPREFIX)*
+	./test-lz4-multiple-legacy.sh

-SKIPFILE = goldenSamples/skip.bin
-test-lz4-skippable: FPREFIX = tmp-lsk
-test-lz4-skippable: lz4 datagen
+test-lz4-skippable: lz4
 	@echo "\n ---- test lz4 with skippable frames ----"
-	$(LZ4) -dc $(SKIPFILE)
-	$(LZ4) -dc < $(SKIPFILE)
-	cat $(SKIPFILE) | $(LZ4) -dc
-	echo "Hello from Valid Frame!\n" | $(LZ4) -c > $(FPREFIX).lz4
-	cat $(SKIPFILE) $(FPREFIX).lz4 $(SKIPFILE) | $(LZ4) -dc
-	$(RM) $(FPREFIX)*
+	./test-lz4-skippable.sh

-test-lz4-basic: FPREFIX = tmp-tlb
 test-lz4-basic: lz4 datagen unlz4 lz4cat
 	@echo "\n ---- test lz4 basic compression/decompression ----"
-	$(DATAGEN) -g0       | $(LZ4) -v     | $(LZ4) -t
-	$(DATAGEN) -g16KB    | $(LZ4) -9     | $(LZ4) -t
-	$(DATAGEN) -g20KB > $(FPREFIX)-dg20k
-	$(LZ4) < $(FPREFIX)-dg20k | $(LZ4) -d > $(FPREFIX)-dec
-	$(DIFF) -q $(FPREFIX)-dg20k $(FPREFIX)-dec
-	$(LZ4) --no-frame-crc < $(FPREFIX)-dg20k | $(LZ4) -d > $(FPREFIX)-dec
-	$(DIFF) -q $(FPREFIX)-dg20k $(FPREFIX)-dec
-	$(DATAGEN)           | $(LZ4) -BI    | $(LZ4) -t
-	$(DATAGEN)           | $(LZ4) --no-crc | $(LZ4) -t
-	$(DATAGEN) -g6M -P99 | $(LZ4) -9BD   | $(LZ4) -t
-	$(DATAGEN) -g17M     | $(LZ4) -9v    | $(LZ4) -qt
-	$(DATAGEN) -g33M     | $(LZ4) --no-frame-crc | $(LZ4) -t
-	$(DATAGEN) -g256MB   | $(LZ4) -vqB4D | $(LZ4) -t --no-crc
-	@echo "hello world" > $(FPREFIX)-hw
-	$(LZ4) --rm -f $(FPREFIX)-hw $(FPREFIX)-hw.lz4
-	test ! -f $(FPREFIX)-hw                      # must fail (--rm)
-	test   -f $(FPREFIX)-hw.lz4
-	$(PRGDIR)/lz4cat $(FPREFIX)-hw.lz4 | $(GREP) "hello world"
-	$(PRGDIR)/unlz4 --rm $(FPREFIX)-hw.lz4 $(FPREFIX)-hw
-	test   -f $(FPREFIX)-hw
-	test ! -f $(FPREFIX)-hw.lz4                  # must fail (--rm)
-	test ! -f $(FPREFIX)-hw.lz4.lz4              # must fail (unlz4)
-	$(PRGDIR)/lz4cat $(FPREFIX)-hw               # pass-through mode
-	test   -f $(FPREFIX)-hw
-	test ! -f $(FPREFIX)-hw.lz4                  # must fail (lz4cat)
-	$(LZ4) $(FPREFIX)-hw $(FPREFIX)-hw.lz4          # creates $(FPREFIX)-hw.lz4
-	$(PRGDIR)/lz4cat < $(FPREFIX)-hw.lz4 > $(FPREFIX)3  # checks lz4cat works with stdin (#285)
-	$(DIFF) -q $(FPREFIX)-hw $(FPREFIX)3
-	$(PRGDIR)/lz4cat < $(FPREFIX)-hw > $(FPREFIX)2      # checks lz4cat works in pass-through mode
-	$(DIFF) -q $(FPREFIX)-hw $(FPREFIX)2
-	cp $(FPREFIX)-hw ./-d
-	$(LZ4) --rm -- -d -d.lz4               # compresses ./d into ./-d.lz4
-	test   -f ./-d.lz4
-	test ! -f ./-d
-	mv ./-d.lz4 ./-z
-	$(LZ4) -d --rm -- -z $(FPREFIX)4          # uncompresses ./-z into $(FPREFIX)4
-	test ! -f ./-z
-	$(DIFF) -q $(FPREFIX)-hw $(FPREFIX)4
-	! $(LZ4) $(FPREFIX)2 $(FPREFIX)3 $(FPREFIX)4    # must fail: refuse to handle 3+ file names
-	$(LZ4) -f $(FPREFIX)-hw                   # create $(FPREFIX)-hw.lz4, for next tests
-	$(LZ4) --list $(FPREFIX)-hw.lz4           # test --list on valid single-frame file
-	$(LZ4) --list < $(FPREFIX)-hw.lz4         # test --list from stdin (file only)
-	$(CAT) $(FPREFIX)-hw >> $(FPREFIX)-hw.lz4
-	! $(LZ4) -f $(FPREFIX)-hw.lz4             # uncompress valid frame followed by invalid data (must fail now)
-	$(LZ4) -BX $(FPREFIX)-hw -c -q | $(LZ4) -tv  # test block checksum
-	# $(DATAGEN) -g20KB generates the same file every single time
-	# cannot save output of $(DATAGEN) -g20KB as input file to lz4 because the following shell commands are run before $(DATAGEN) -g20KB
-	test "$(shell $(DATAGEN) -g20KB | $(LZ4) -c --fast | wc -c)" -lt "$(shell $(DATAGEN) -g20KB | $(LZ4) -c --fast=9 | wc -c)" # -1 vs -9
-	test "$(shell $(DATAGEN) -g20KB | $(LZ4) -c -1 | wc -c)" -lt "$(shell $(DATAGEN) -g20KB| $(LZ4) -c --fast=1 | wc -c)" # 1 vs -1
-	test "$(shell $(DATAGEN) -g20KB | $(LZ4) -c --fast=1 | wc -c)" -eq "$(shell $(DATAGEN) -g20KB| $(LZ4) -c --fast| wc -c)" # checks default fast compression is -1
-	! $(LZ4) -c --fast=0 $(FPREFIX)-dg20K # lz4 should fail when fast=0
-	! $(LZ4) -c --fast=-1 $(FPREFIX)-dg20K # lz4 should fail when fast=-1
-	# High --fast values can result in out-of-bound dereferences #876
-	$(DATAGEN) -g1M | $(LZ4) -c --fast=999999999 > /dev/null
-	# Test for #596
-	@echo "TEST" > $(FPREFIX)-test
-	$(LZ4) -m $(FPREFIX)-test
-	$(LZ4) $(FPREFIX)-test.lz4 $(FPREFIX)-test2
-	$(DIFF) -q $(FPREFIX)-test $(FPREFIX)-test2
-	@$(RM) $(FPREFIX)*
+	./test-lz4-basic.sh

-
-test-lz4-dict: FPREFIX = tmp-dict
 test-lz4-dict: lz4 datagen
 	@echo "\n ---- test lz4 compression/decompression with dictionary ----"
-	$(DATAGEN) -g16KB > $(FPREFIX)
-	$(DATAGEN) -g32KB > $(FPREFIX)-sample-32k
-	< $(FPREFIX)-sample-32k $(LZ4) -D $(FPREFIX) | $(LZ4) -dD $(FPREFIX) | diff - $(FPREFIX)-sample-32k
-	$(DATAGEN) -g128MB > $(FPREFIX)-sample-128m
-	< $(FPREFIX)-sample-128m $(LZ4) -D $(FPREFIX) | $(LZ4) -dD $(FPREFIX) | diff - $(FPREFIX)-sample-128m
-	touch $(FPREFIX)-sample-0
-	< $(FPREFIX)-sample-0 $(LZ4) -D $(FPREFIX) | $(LZ4) -dD $(FPREFIX) | diff - $(FPREFIX)-sample-0
-
-	< $(FPREFIX)-sample-32k $(LZ4) -D $(FPREFIX)-sample-0 | $(LZ4) -dD $(FPREFIX)-sample-0 | diff - $(FPREFIX)-sample-32k
-	< $(FPREFIX)-sample-0 $(LZ4) -D $(FPREFIX)-sample-0 | $(LZ4) -dD $(FPREFIX)-sample-0 | diff - $(FPREFIX)-sample-0
-
-	@echo "\n ---- test lz4 dictionary loading ----"
-	$(DATAGEN) -g128KB > $(FPREFIX)-data-128KB
-	set -e; \
-	for l in 0 1 4 128 32767 32768 32769 65535 65536 65537 98303 98304 98305 131071 131072 131073; do \
-		$(DATAGEN) -g$$l > $(FPREFIX)-$$l; \
-		$(DD) if=$(FPREFIX)-$$l of=$(FPREFIX)-$$l-tail bs=1 count=65536 skip=$$((l > 65536 ? l - 65536 : 0)); \
-		< $(FPREFIX)-$$l      $(LZ4) -D stdin $(FPREFIX)-data-128KB -c | $(LZ4) -dD $(FPREFIX)-$$l-tail | $(DIFF) - $(FPREFIX)-data-128KB; \
-		< $(FPREFIX)-$$l-tail $(LZ4) -D stdin $(FPREFIX)-data-128KB -c | $(LZ4) -dD $(FPREFIX)-$$l      | $(DIFF) - $(FPREFIX)-data-128KB; \
-	done
-	@$(RM) $(FPREFIX)*
+	./test-lz4-dict.sh

 test-lz4hc-hugefile: lz4 datagen
 	@echo "\n ---- test HC compression/decompression of huge files ----"
-	$(DATAGEN) -g4200MB | $(LZ4) -v3BD | $(LZ4) -qt
+	./test-lz4hc-hugefile.sh

-test-lz4-fast-hugefile: FPREFIX = tmp-lfh
 test-lz4-fast-hugefile: lz4 datagen
 	@echo "\n ---- test huge files compression/decompression ----"
-	$(DATAGEN) -g6GB    | $(LZ4) -vB5D | $(LZ4) -qt
-	# test large file size [2-4] GB
-	@$(DATAGEN) -g3G -P100 | $(LZ4) -vv | $(LZ4) --decompress --force --sparse - $(FPREFIX)1
-	@ls -ls $(FPREFIX)1
-	@$(DATAGEN) -g3G -P100 | $(LZ4) --quiet --content-size | $(LZ4) --verbose --decompress --force --sparse - $(FPREFIX)2
-	@ls -ls $(FPREFIX)2
-	$(DIFF) -s $(FPREFIX)1 $(FPREFIX)2
-	@$(RM) $(FPREFIX)*
+	./test-lz4-fast-hugefile.sh

 test-lz4-hugefile: test-lz4-fast-hugefile test-lz4hc-hugefile

-test-lz4-testmode: FPREFIX = tmp-ltm
 test-lz4-testmode: lz4 datagen
 	@echo "\n ---- bench mode ----"
-	$(LZ4) -bi0
-	$(DATAGEN) > $(FPREFIX)
-	$(LZ4) -f $(FPREFIX) -c > $(FPREFIX).lz4
-	$(LZ4) -bdi0 $(FPREFIX).lz4 # test benchmark decode-only mode
-	$(LZ4) -bdi0 --no-crc $(FPREFIX).lz4 # test benchmark decode-only mode
-	@echo "\n ---- test mode ----"
-	! $(DATAGEN) | $(LZ4) -t
-	! $(DATAGEN) | $(LZ4) -tf
-	@echo "\n ---- pass-through mode ----"
-	@echo "Why hello there " > $(FPREFIX)2.lz4
-	! $(LZ4) -f $(FPREFIX)2.lz4 > $(VOID)
-	! $(DATAGEN) | $(LZ4) -dc  > $(VOID)
-	! $(DATAGEN) | $(LZ4) -df > $(VOID)
-	$(DATAGEN) | $(LZ4) -dcf > $(VOID)
-	@echo "Hello World !" > $(FPREFIX)1
-	$(LZ4) -dcf $(FPREFIX)1
-	@echo "from underground..." > $(FPREFIX)2
-	$(LZ4) -dcfm $(FPREFIX)1 $(FPREFIX)2
-	@echo "\n ---- non-existing source (must fail cleanly) ----"
-	! $(LZ4)     file-does-not-exist
-	! $(LZ4) -f  file-does-not-exist
-	! $(LZ4) -t  file-does-not-exist
-	! $(LZ4) -fm file1-dne file2-dne
-	@$(RM) $(FPREFIX)*
+	./test-lz4-testmode.sh

 test-lz4-opt-parser: lz4 datagen
 	@echo "\n ---- test opt-parser ----"
-	$(DATAGEN) -g16KB      | $(LZ4) -12      | $(LZ4) -t
-	$(DATAGEN) -P10        | $(LZ4) -12B4    | $(LZ4) -t
-	$(DATAGEN) -g256K      | $(LZ4) -12B4D   | $(LZ4) -t
-	$(DATAGEN) -g512K -P25 | $(LZ4) -12BD    | $(LZ4) -t
-	$(DATAGEN) -g1M        | $(LZ4) -12B5    | $(LZ4) -t
-	$(DATAGEN) -g1M -s2    | $(LZ4) -12B4D   | $(LZ4) -t
-	$(DATAGEN) -g2M -P99   | $(LZ4) -11B4D   | $(LZ4) -t
-	$(DATAGEN) -g4M        | $(LZ4) -11vq    | $(LZ4) -qt
-	$(DATAGEN) -g8M        | $(LZ4) -11B4    | $(LZ4) -t
-	$(DATAGEN) -g16M -P90  | $(LZ4) -11B5    | $(LZ4) -t
-	$(DATAGEN) -g32M -P10  | $(LZ4) -11B5D   | $(LZ4) -t
+	./test-lz4-opt-parser.sh

 test-lz4-essentials : lz4 datagen test-lz4-basic test-lz4-multiple test-lz4-multiple-legacy \
                      test-lz4-frame-concatenation test-lz4-testmode \
@@ -609,10 +350,44 @@ test-decompress-partial : decompress-partial decompress-partial-usingDict
 	@echo "\n ---- test decompress-partial-usingDict ----"
 	./decompress-partial-usingDict$(EXT)

+
+#-----------------------------------------------------------------------------
+# freestanding test only for Linux x86_64
+#-----------------------------------------------------------------------------
+ifeq ($(OS),Windows_NT)
+  UNAME_S := Windows
+else
+  UNAME_S := $(shell uname -s)
+endif
+
+ifeq ($(OS),Windows_NT)
+  UNAME_P := Unknown
+else
+  UNAME_P := $(shell uname -p)
+endif
+
+FREESTANDING_CFLAGS := -ffreestanding -nostdlib
+
+ifneq ($(UNAME_S), Linux)
+  FREESTANDING_CFLAGS :=
+endif
+
+ifneq ($(UNAME_P), x86_64)
+  FREESTANDING_CFLAGS :=
+endif
+
+freestanding: freestanding.c
+	$(CC) $(FREESTANDING_CFLAGS) $^ -o $@$(EXT)
+
 test-freestanding: freestanding
 	@echo "\n ---- test freestanding ----"
+ifeq ($(FREESTANDING_CFLAGS),)
+	@echo "\n (skip)"
+else
 	./freestanding$(EXT)
 	-strace ./freestanding$(EXT)
 	-ltrace ./freestanding$(EXT)
+endif
+

 endif
@@ -30,7 +30,6 @@


 #if !defined(__x86_64__) || !defined(__linux__)
-EXTERN_C void _start(void) { }
 int main(int argc, char** argv) { return 0; }
 #else

@@ -371,6 +371,33 @@ static int local_LZ4F_compressFrame(const char* in, char* out, int inSize)
    return (int)LZ4F_compressFrame(out, LZ4F_compressFrameBound((size_t)inSize, NULL), in, (size_t)inSize, NULL);
 }

+LZ4F_cctx* g_cctx = NULL;
+static int local_LZ4F_compress(const char* in, char* out, int inSize)
+{
+    /* output buffer size is assumed */
+    size_t const outSize = LZ4F_compressFrameBound((size_t)inSize, NULL);
+    size_t cSize = 0;
+    assert(inSize >= 0);
+    if (g_cctx == NULL) {
+        /* create and initialize LZ4F compression context the first time */
+        LZ4F_createCompressionContext(&g_cctx, LZ4F_VERSION);
+        assert(g_cctx != NULL);
+    } /* re-use existing compression context otherwise */
+    {   size_t const cbSize = LZ4F_compressBegin(g_cctx, out, outSize, NULL);
+        assert(!LZ4F_isError(cbSize));
+        cSize += cbSize;
+    }
+    {   size_t const cuSize = LZ4F_compressUpdate(g_cctx, out+cSize, outSize-cSize, in, (size_t)inSize, NULL);
+        assert(!LZ4F_isError(cuSize));
+        cSize += cuSize;
+    }
+    {   size_t const ceSize = LZ4F_compressEnd(g_cctx, out+cSize, outSize-cSize, NULL);
+        assert(!LZ4F_isError(ceSize));
+        cSize += ceSize;
+    }
+    return (int)cSize;
+}
+
 static LZ4F_decompressionContext_t g_dCtx;

 static int local_LZ4F_decompress(const char* in, char* out, int inSize, int outSize)
@@ -586,6 +613,9 @@ int fullSpeedBench(const char** fileNamesTable, int nbFiles)
            case 30: compressionFunction = local_LZ4F_compressFrame; compressorName = "LZ4F_compressFrame";
                        chunkP[0].origSize = (int)benchedSize; nbChunks=1;
                        break;
+            case 31: compressionFunction = local_LZ4F_compress; compressorName = "LZ4F_compressUpdate";
+                        chunkP[0].origSize = (int)benchedSize; nbChunks=1;
+                        break;
            case 40: compressionFunction = local_LZ4_saveDict; compressorName = "LZ4_saveDict";
                        if (chunkP[0].origSize < 8) { DISPLAY(" cannot bench %s with less then 8 bytes \n", compressorName); continue; }
                        LZ4_loadDict(&LZ4_stream, chunkP[0].origBuffer, chunkP[0].origSize);
@@ -0,0 +1,76 @@
+#!/bin/sh
+
+FPREFIX="tmp-tlb"
+
+set -e
+
+remove () {
+    rm -rf $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+datagen -g0       | lz4 -v     | lz4 -t
+datagen -g16KB    | lz4 -9     | lz4 -t
+datagen -g20KB > $FPREFIX-dg20k
+lz4 < $FPREFIX-dg20k | lz4 -d > $FPREFIX-dec
+diff -q $FPREFIX-dg20k $FPREFIX-dec
+lz4 --no-frame-crc < $FPREFIX-dg20k | lz4 -d > $FPREFIX-dec
+diff -q $FPREFIX-dg20k $FPREFIX-dec
+datagen           | lz4 -BI    | lz4 -t
+datagen           | lz4 --no-crc | lz4 -t
+datagen -g6M -P99 | lz4 -9BD   | lz4 -t
+datagen -g17M     | lz4 -9v    | lz4 -qt
+datagen -g33M     | lz4 --no-frame-crc | lz4 -t
+datagen -g256MB   | lz4 -vqB4D | lz4 -t --no-crc
+echo "hello world" > $FPREFIX-hw
+lz4 --rm -f $FPREFIX-hw $FPREFIX-hw.lz4
+test ! -f $FPREFIX-hw                   # must fail (--rm)
+test   -f $FPREFIX-hw.lz4
+lz4cat $FPREFIX-hw.lz4 | grep "hello world"
+unlz4 --rm $FPREFIX-hw.lz4 $FPREFIX-hw
+test   -f $FPREFIX-hw
+test ! -f $FPREFIX-hw.lz4               # must fail (--rm)
+test ! -f $FPREFIX-hw.lz4.lz4           # must fail (unlz4)
+lz4cat $FPREFIX-hw                      # pass-through mode
+test   -f $FPREFIX-hw
+test ! -f $FPREFIX-hw.lz4               # must fail (lz4cat)
+lz4 $FPREFIX-hw $FPREFIX-hw.lz4         # creates $FPREFIX-hw.lz4
+lz4cat < $FPREFIX-hw.lz4 > ${FPREFIX}3  # checks lz4cat works with stdin (#285)
+diff -q $FPREFIX-hw ${FPREFIX}3
+lz4cat < $FPREFIX-hw > ${FPREFIX}2      # checks lz4cat works in pass-through mode
+diff -q $FPREFIX-hw ${FPREFIX}2
+cp $FPREFIX-hw ./-d
+lz4 --rm -- -d -d.lz4                   # compresses ./d into ./-d.lz4
+test   -f ./-d.lz4
+test ! -f ./-d
+mv ./-d.lz4 ./-z
+lz4 -d --rm -- -z ${FPREFIX}4           # uncompresses ./-z into $FPREFIX4
+test ! -f ./-z
+diff -q $FPREFIX-hw ${FPREFIX}4
+lz4 ${FPREFIX}2 ${FPREFIX}3 ${FPREFIX}4 && exit 1  # must fail: refuse to handle 3+ file names
+mkdir -p ${FPREFIX}-dir
+lz4 ${FPREFIX}-dir && exit 1          # must fail: refuse to compress directory
+test ! -f ${FPREFIX}-dir.lz4          # must not create artifact (#1211)
+lz4 -f $FPREFIX-hw                    # create $FPREFIX-hw.lz4, for next tests
+lz4 --list $FPREFIX-hw.lz4            # test --list on valid single-frame file
+lz4 --list < $FPREFIX-hw.lz4          # test --list from stdin (file only)
+cat $FPREFIX-hw >> $FPREFIX-hw.lz4
+lz4 -f $FPREFIX-hw.lz4 && exit 1      # uncompress valid frame followed by invalid data (must fail now)
+lz4 -BX $FPREFIX-hw -c -q | lz4 -tv   # test block checksum
+# datagen -g20KB generates the same file every single time
+# cannot save output of datagen -g20KB as input file to lz4 because the following shell commands are run before datagen -g20KB
+test "$(datagen -g20KB | lz4 -c --fast | wc -c)" -lt "$(datagen -g20KB | lz4 -c --fast=9 | wc -c)" # -1 vs -9
+test "$(datagen -g20KB | lz4 -c -1 | wc -c)" -lt "$(datagen -g20KB| lz4 -c --fast=1 | wc -c)" # 1 vs -1
+test "$(datagen -g20KB | lz4 -c --fast=1 | wc -c)" -eq "$(datagen -g20KB| lz4 -c --fast| wc -c)" # checks default fast compression is -1
+lz4 -c --fast=0 $FPREFIX-dg20K && exit 1  # lz4 should fail when fast=0
+lz4 -c --fast=-1 $FPREFIX-dg20K && exit 1 # lz4 should fail when fast=-1
+# High --fast values can result in out-of-bound dereferences #876
+datagen -g1M | lz4 -c --fast=999999999 > $FPREFIX-trash
+# Test for #596
+echo "TEST" > $FPREFIX-test
+lz4 -m $FPREFIX-test
+lz4 $FPREFIX-test.lz4 $FPREFIX-test2
+diff -q $FPREFIX-test $FPREFIX-test2
@@ -0,0 +1,27 @@
+#!/bin/sh
+
+FPREFIX="tmp-lzc"
+
+set -e
+
+remove () {
+    rm $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+datagen -g15M > $FPREFIX
+lz4 -v $FPREFIX -c | lz4 -t
+lz4 -v --content-size $FPREFIX -c | lz4 -d > $FPREFIX-dup
+diff $FPREFIX $FPREFIX-dup
+lz4 -f $FPREFIX -c > $FPREFIX.lz4 # compressed with content size
+lz4 --content-size $FPREFIX -c > $FPREFIX-wcz.lz4
+diff $FPREFIX.lz4 $FPREFIX-wcz.lz4 && exit 1 # must differ, due to content size
+lz4 --content-size < $FPREFIX > $FPREFIX-wcz2.lz4 # can determine content size because stdin is just a file
+diff $FPREFIX-wcz.lz4 $FPREFIX-wcz2.lz4  # both must contain content size
+cat $FPREFIX | lz4 > $FPREFIX-ncz.lz4
+diff $FPREFIX.lz4 $FPREFIX-ncz.lz4  # both don't have content size
+cat $FPREFIX | lz4 --content-size > $FPREFIX-ncz2.lz4 # can't determine content size
+diff $FPREFIX.lz4 $FPREFIX-ncz2.lz4  # both don't have content size
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+FPREFIX="tmp-dict"
+
+set -e
+
+remove () {
+    rm $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+datagen -g16KB > $FPREFIX
+datagen -g32KB > $FPREFIX-sample-32k
+< $FPREFIX-sample-32k lz4 -D $FPREFIX | lz4 -dD $FPREFIX | diff - $FPREFIX-sample-32k
+datagen -g128MB > $FPREFIX-sample-128m
+< $FPREFIX-sample-128m lz4 -D $FPREFIX | lz4 -dD $FPREFIX | diff - $FPREFIX-sample-128m
+touch $FPREFIX-sample-0
+< $FPREFIX-sample-0 lz4 -D $FPREFIX | lz4 -dD $FPREFIX | diff - $FPREFIX-sample-0
+
+< $FPREFIX-sample-32k lz4 -D $FPREFIX-sample-0 | lz4 -dD $FPREFIX-sample-0 | diff - $FPREFIX-sample-32k
+< $FPREFIX-sample-0 lz4 -D $FPREFIX-sample-0 | lz4 -dD $FPREFIX-sample-0 | diff - $FPREFIX-sample-0
+lz4 -bi0 -D $FPREFIX $FPREFIX-sample-32k $FPREFIX-sample-32k
+
+echo "---- test lz4 dictionary loading ----"
+datagen -g128KB > $FPREFIX-data-128KB
+set -e; \
+for l in 0 1 4 128 32767 32768 32769 65535 65536 65537 98303 98304 98305 131071 131072 131073; do \
+    datagen -g$$l > $FPREFIX-$$l; \
+    dd if=$FPREFIX-$$l of=$FPREFIX-$$l-tail bs=1 count=65536 skip=$((l > 65536 ? l - 65536 : 0)); \
+    < $FPREFIX-$$l      lz4 -D stdin $FPREFIX-data-128KB -c | lz4 -dD $FPREFIX-$$l-tail | diff - $FPREFIX-data-128KB; \
+    < $FPREFIX-$$l-tail lz4 -D stdin $FPREFIX-data-128KB -c | lz4 -dD $FPREFIX-$$l      | diff - $FPREFIX-data-128KB; \
+done
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+FPREFIX="tmp-lfh"
+
+set -e
+
+remove () {
+    rm $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+datagen -g6GB    | lz4 -vB5D | lz4 -qt
+# test large file size [2-4] GB
+datagen -g3G -P100 | lz4 -vv | lz4 --decompress --force --sparse - ${FPREFIX}1
+ls -ls ${FPREFIX}1
+datagen -g3G -P100 | lz4 --quiet --content-size | lz4 --verbose --decompress --force --sparse - ${FPREFIX}2
+ls -ls ${FPREFIX}2
+diff -s ${FPREFIX}1 ${FPREFIX}2
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+FPREFIX="tmp-lfc"
+
+set -e
+remove () {
+    rm $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+echo > $FPREFIX-empty
+echo hi > $FPREFIX-nonempty
+cat $FPREFIX-nonempty $FPREFIX-empty $FPREFIX-nonempty > $FPREFIX-src
+lz4 -zq $FPREFIX-empty -c > $FPREFIX-empty.lz4
+lz4 -zq $FPREFIX-nonempty -c > $FPREFIX-nonempty.lz4
+cat $FPREFIX-nonempty.lz4 $FPREFIX-empty.lz4 $FPREFIX-nonempty.lz4 > $FPREFIX-concat.lz4
+lz4 -d $FPREFIX-concat.lz4 -c > $FPREFIX-result
+cmp $FPREFIX-src $FPREFIX-result
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+FPREFIX="tmp-lml"
+
+set -e
+
+remove () {
+    rm $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+datagen -s1        > "${FPREFIX}1" 2> $FPREFIX-trash
+datagen -s2 -g100K > "${FPREFIX}2" 2> $FPREFIX-trash
+datagen -s3 -g200K > "${FPREFIX}3" 2> $FPREFIX-trash
+# compress multiple files using legacy format: one .lz4 per source file
+lz4 -f -l -m $FPREFIX*
+test -f ${FPREFIX}1.lz4
+test -f ${FPREFIX}2.lz4
+test -f ${FPREFIX}3.lz4
+# decompress multiple files compressed using legacy format: one output file per .lz4
+mv ${FPREFIX}1 ${FPREFIX}1-orig
+mv ${FPREFIX}2 ${FPREFIX}2-orig
+mv ${FPREFIX}3 ${FPREFIX}3-orig
+lz4 -d -f -m $FPREFIX*.lz4
+lz4 -l -d -f -m $FPREFIX*.lz4 # -l mustn't impact -d option
+cmp ${FPREFIX}1 ${FPREFIX}1-orig   # must be identical
+cmp ${FPREFIX}2 ${FPREFIX}2-orig
+cmp ${FPREFIX}3 ${FPREFIX}3-orig
+# compress multiple files into stdout using legacy format
+cat ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 > $FPREFIX-concat1
+rm $FPREFIX*.lz4
+lz4 -l -m ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3 -c > $FPREFIX-concat2
+test ! -f ${FPREFIX}1.lz4  # must not create .lz4 artefact
+cmp $FPREFIX-concat1 $FPREFIX-concat2  # must be equivalent
+# # # decompress multiple files into stdout using legacy format
+rm $FPREFIX-concat1 $FPREFIX-concat2
+lz4 -l -f -m ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3   # generate .lz4 to decompress
+cat ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3 > ${FPREFIX}-concat1   # create concatenated reference
+rm ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3
+lz4 -d -m ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 -c > $FPREFIX-concat2
+lz4 -d -l -m ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 -c > $FPREFIX-concat2 # -l mustn't impact option -d
+test ! -f ${FPREFIX}1  # must not create file artefact
+cmp $FPREFIX-concat1 $FPREFIX-concat2  # must be equivalent
+# # # compress multiple files, one of which is absent (must fail)
+lz4 -f -l -m $FPREFIX-concat1 notHere-legacy $FPREFIX-concat2 && exit 1 # must fail : notHere-legacy not present
+true
@@ -0,0 +1,58 @@
+#!/bin/sh
+
+FPREFIX="tmp-tml"
+
+set -e
+
+remove () {
+    rm $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+datagen -s1        > ${FPREFIX}1 2> $FPREFIX-trash
+datagen -s2 -g100K > ${FPREFIX}2 2> $FPREFIX-trash
+datagen -s3 -g200K > ${FPREFIX}3 2> $FPREFIX-trash
+# compress multiple files : one .lz4 per source file
+lz4 -f -m $FPREFIX*
+test -f ${FPREFIX}1.lz4
+test -f ${FPREFIX}2.lz4
+test -f ${FPREFIX}3.lz4
+# decompress multiple files : one output file per .lz4
+mv ${FPREFIX}1 ${FPREFIX}1-orig
+mv ${FPREFIX}2 ${FPREFIX}2-orig
+mv ${FPREFIX}3 ${FPREFIX}3-orig
+lz4 -d -f -m $FPREFIX*.lz4
+cmp ${FPREFIX}1 ${FPREFIX}1-orig   # must be identical
+cmp ${FPREFIX}2 ${FPREFIX}2-orig
+cmp ${FPREFIX}3 ${FPREFIX}3-orig
+# compress multiple files into stdout
+cat ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 > $FPREFIX-concat1
+rm $FPREFIX*.lz4
+lz4 -m ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3 -c > $FPREFIX-concat2
+test ! -f ${FPREFIX}1.lz4  # must not create .lz4 artefact
+cmp $FPREFIX-concat1 $FPREFIX-concat2  # must be equivalent
+# decompress multiple files into stdout
+rm $FPREFIX-concat1 $FPREFIX-concat2
+lz4 -f -m ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3   # generate .lz4 to decompress
+cat ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3 > $FPREFIX-concat1   # create concatenated reference
+rm ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3
+lz4 -d -m ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 -c > $FPREFIX-concat2
+test ! -f ${FPREFIX}1  # must not create file artefact
+cmp $FPREFIX-concat1 $FPREFIX-concat2  # must be equivalent
+# compress multiple files, one of which is absent (must fail)
+lz4 -f -m $FPREFIX-concat1 notHere $FPREFIX-concat2 && exit 1 # must fail : notHere not present
+# test lz4-compressed file
+rm $FPREFIX-concat1 $FPREFIX-concat2
+lz4 -tm $FPREFIX-concat1.lz4
+# ensure the test doesn't create artifact
+test ! -f $FPREFIX-concat1  # must not create file artefact
+# test multiple lz4-compressed file
+lz4 -tm $FPREFIX-concat1.lz4 $FPREFIX-concat2.lz4
+test ! -f $FPREFIX-concat1  # must not create file artefact
+test ! -f $FPREFIX-concat2  # must not create file artefact
+# test multiple lz4 files, one of which is absent (must fail)
+lz4 -tm $FPREFIX-concat1.lz4 notHere.lz4 $FPREFIX-concat2.lz4 && exit 1
+true
@@ -0,0 +1,16 @@
+#!/bin/sh
+
+set -e
+set -x
+
+datagen -g16KB      | lz4 -12      | lz4 -t
+datagen -P10        | lz4 -12B4    | lz4 -t
+datagen -g256K      | lz4 -12B4D   | lz4 -t
+datagen -g512K -P25 | lz4 -12BD    | lz4 -t
+datagen -g1M        | lz4 -12B5    | lz4 -t
+datagen -g1M -s2    | lz4 -12B4D   | lz4 -t
+datagen -g2M -P99   | lz4 -11B4D   | lz4 -t
+datagen -g4M        | lz4 -11vq    | lz4 -qt
+datagen -g8M        | lz4 -11B4    | lz4 -t
+datagen -g16M -P90  | lz4 -11B5    | lz4 -t
+datagen -g32M -P10  | lz4 -11B5D   | lz4 -t
@@ -0,0 +1,19 @@
+#!/bin/sh
+
+SKIPFILE="goldenSamples/skip.bin"
+FPREFIX="tmp-lsk"
+
+set -e
+
+remove () {
+    rm "$FPREFIX"*
+}
+
+trap remove EXIT
+
+set -x
+
+lz4 -dc $SKIPFILE
+lz4 -dc < $SKIPFILE
+printf "Hello from Valid Frame!\n" | lz4 -c > $FPREFIX.lz4
+cat $SKIPFILE $FPREFIX.lz4 $SKIPFILE | lz4 -dc
@@ -0,0 +1,42 @@
+#!/bin/sh
+
+FPREFIX="tmp-tls"
+
+set -e
+
+remove () {
+    rm $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+datagen -g5M  -P100 > ${FPREFIX}dg5M
+lz4 -B4D ${FPREFIX}dg5M -c | lz4 -dv --sparse > ${FPREFIX}cB4
+diff -s ${FPREFIX}dg5M ${FPREFIX}cB4
+lz4 -B5D ${FPREFIX}dg5M -c | lz4 -dv --sparse > ${FPREFIX}cB5
+diff -s ${FPREFIX}dg5M ${FPREFIX}cB5
+lz4 -B6D ${FPREFIX}dg5M -c | lz4 -dv --sparse > ${FPREFIX}cB6
+diff -s ${FPREFIX}dg5M ${FPREFIX}cB6
+lz4 -B7D ${FPREFIX}dg5M -c | lz4 -dv --sparse > ${FPREFIX}cB7
+diff -s ${FPREFIX}dg5M ${FPREFIX}cB7
+lz4 ${FPREFIX}dg5M -c | lz4 -dv --no-sparse > ${FPREFIX}nosparse
+diff -s ${FPREFIX}dg5M ${FPREFIX}nosparse
+ls -ls $FPREFIX*
+datagen -s1 -g1200007 -P100 | lz4 | lz4 -dv --sparse > ${FPREFIX}odd   # Odd size file (to generate non-full last block)
+datagen -s1 -g1200007 -P100 | diff -s - ${FPREFIX}odd
+ls -ls ${FPREFIX}odd
+rm $FPREFIX*
+printf "\n Compatibility with Console :"
+echo "Hello World 1 !" | lz4 | lz4 -d -c
+echo "Hello World 2 !" | lz4 | lz4 -d | cat
+echo "Hello World 3 !" | lz4 --no-frame-crc | lz4 -d -c
+printf "\n Compatibility with Append :"
+datagen -P100 -g1M > ${FPREFIX}dg1M
+cat ${FPREFIX}dg1M ${FPREFIX}dg1M > ${FPREFIX}2M
+lz4 -B5 -v ${FPREFIX}dg1M ${FPREFIX}c
+lz4 -d -v ${FPREFIX}c ${FPREFIX}r
+lz4 -d -v ${FPREFIX}c -c >> ${FPREFIX}r
+ls -ls $FPREFIX*
+diff ${FPREFIX}2M ${FPREFIX}r
@@ -0,0 +1,38 @@
+#!/bin/sh
+
+FPREFIX="tmp-ltm"
+
+set -e
+
+remove () {
+    rm $FPREFIX*
+}
+
+trap remove EXIT
+
+set -x
+
+lz4 -bi0
+datagen > $FPREFIX
+lz4 -f $FPREFIX -c > $FPREFIX.lz4
+lz4 -bdi0 $FPREFIX.lz4 # test benchmark decode-only mode
+lz4 -bdi0 --no-crc $FPREFIX.lz4 # test benchmark decode-only mode
+echo "---- test mode ----"
+datagen | lz4 -t && exit 1
+datagen | lz4 -tf && exit 1
+echo "---- pass-through mode ----"
+echo "Why hello there " > ${FPREFIX}2.lz4
+lz4 -f ${FPREFIX}2.lz4 > $FPREFIX-trash && exit 1
+datagen | lz4 -dc  > $FPREFIX-trash && exit 1
+datagen | lz4 -df > $FPREFIX-trash && exit 1
+datagen | lz4 -dcf > $FPREFIX-trash
+echo "Hello World !" > ${FPREFIX}1
+lz4 -dcf ${FPREFIX}1
+echo "from underground..." > ${FPREFIX}2
+lz4 -dcfm ${FPREFIX}1 ${FPREFIX}2
+echo "---- non-existing source (must fail cleanly) ----"
+lz4     file-does-not-exist && exit 1
+lz4 -f  file-does-not-exist && exit 1
+lz4 -t  file-does-not-exist && exit 1
+lz4 -fm file1-dne file2-dne && exit 1
+true
@@ -0,0 +1,6 @@
+#!/bin/sh
+
+set -e
+set -x
+
+datagen -g4200MB | lz4 -v3BD | lz4 -qt
Author	SHA1	Message	Date
Yann Collet	a51521916d	merge into a single UTIL_isDirectory() method	2023-02-26 20:12:59 -08:00
Yann Collet	45a4880b9c	refuse to compress directories fix #1211, reported by @imba-tjd	2023-02-26 10:40:28 -08:00
Yann Collet	ecf92d0897	Merge pull request #1199 from embg/docs Improve LZ4F_decompress() docs	2023-02-07 14:59:26 -08:00
Yann Collet	185cf42020	Merge pull request #1207 from eli-schwartz/meson build: move meson files from contrib, to go alongside other build systems	2023-02-07 14:59:06 -08:00
Elliot Gorokhovsky	c3addfea0f	improve LZ4F_decompress() documentation	2023-02-03 12:51:05 -08:00
Eli Schwartz	7ab223b7fa	build: move meson files from contrib, to go alongside other build systems	2023-01-30 12:55:03 -05:00
Yann Collet	1f3adea1c6	Merge pull request #1206 from tristan957/meson Clean up generation of internal static library	2023-01-30 09:50:30 -08:00
Tristan Partin	ab8328bc8e	Clean up generation of internal static library No sense in relinking if the default_library indicates that a static library was already generated.	2023-01-10 12:55:46 -06:00
Yann Collet	266359f7a6	Merge pull request #1204 from DimitriPapadopoulos/codespell Fix typo found by codespell	2023-01-08 11:26:23 -08:00
Dimitri Papadopoulos	b1fd838cdb	Fix typo found by codespell	2023-01-08 09:44:22 +01:00
Yann Collet	fe389cab10	version note	2022-12-19 13:16:08 -08:00
Yann Collet	299daf03ac	Merge pull request #1184 from tristan957/meson Meson updates	2022-12-04 19:57:14 -08:00
Yann Collet	e7e7b7ef9e	Merge pull request #1175 from lz4/usan32 fixed usan32 tests	2022-12-02 17:01:02 -08:00
Yann Collet	8f2fd69708	Merge pull request #1178 from alexmohr/allow-uncompressed-block-for-all-modes uncompressed-blocks: Allow uncompressed blocks for all modes	2022-12-02 17:00:29 -08:00
Yann Collet	002a2756e5	Merge pull request #1188 from x4m/static_function Declare read_long_length_no_check() static	2022-12-02 16:59:51 -08:00
Tristan Partin	2fc9a85bf8	Install lz4file.h only when default_library isn't shared	2022-12-02 17:42:41 -06:00
Tristan Partin	3301f3110f	Only build the freestanding test on Linux x86_64	2022-12-02 17:42:41 -06:00
Tristan Partin	3946e3daed	Add Meson override for the library	2022-12-02 17:42:38 -06:00
Yann Collet	d879e7f15a	Merge pull request #1187 from t-mat/pr-issue-1186 Add environment check for freestanding test : resolves #1186	2022-12-02 14:46:24 -08:00
Yann Collet	3e213fdf8e	Merge pull request #1196 from t-mat/fix-issue-1195 Remove PATH=$(PATH) prefix from all shell script invocation in tests/Makefile	2022-11-29 16:09:31 -08:00
Yann Collet	8f2fad801b	Merge pull request #1197 from t-mat/fix-test-clang-11-12-mx32 Change the version of lib[x]gcc for clang-(11\|12) -mx32	2022-11-29 09:25:32 -08:00
Takayuki Matsuoka	5b83db474a	Change the version of lib[x]gcc for clang-(11\|12) -mx32	2022-11-29 23:15:31 +09:00
Takayuki Matsuoka	95d703ae41	Remove PATH=$(PATH) prefix from all shell script invocation When $(PATH) contains ' ' (space, 0x20), this prefix causes error. Also we don't need reconfiguring PATH for each shell script, we don't need them. This changeset fixes issue #1195	2022-11-29 09:25:42 +09:00
Yann Collet	8a31e6402d	Merge pull request #1191 from vsolontsov-volant/install_lz4file_h Add copying lz4file.h to make install	2022-11-23 16:15:10 -08:00
Yann Collet	f4f072ead8	Merge pull request #1192 from vsolontsov-volant/lz4f_write_const_src Fixed const-ness of src data pointer in lz4file and install lz4file.h	2022-11-22 06:51:46 -08:00
Vladimir Solontsov	e4ea198b44	Fixed const-ness of src data pointer in lz4file and install lz4file.h	2022-11-22 16:26:18 +03:00
Vladimir Solontsov	2a782cc328	Add copying lz4file.h to make install	2022-11-22 16:10:08 +03:00
Andrey Borodin	812c4b13d6	Declare read_long_length_no_check() static Fix #1176	2022-11-10 09:27:30 -08:00
Takayuki Matsuoka	08f1483b0a	Add environment check for freestanding test Freestanding test is intended to run on only Linux x86_64 platform. Because it uses Linux x86_64 syscall directly to avoid any dependency to the standard library. This changeset adds platform checking code to tests/Makefile to avoid unintended error in non-Linux x86_64 platforms. This changeset resolves issue #1186	2022-11-09 23:42:03 +09:00
Yann Collet	214bfb325b	Merge pull request #1139 from tristan957/meson Update Meson build to 1.9.4	2022-10-24 23:47:31 -07:00
Tristan Partin	198e532300	Update Meson build to 1.9.4 Specifically this adds support for the following options: - LZ4_ALIGN_TEST - LZ4_STATIC_LINKING_ONLY_DISABLE_MEMORY_ALLOCATION - LZ4_DISTANCE_MAX - LZ4_FAST_DEC_LOOP - LZ4_FORCE_SW_BITCOUNT - LZ4_FREESTANDING - LZ4_USER_MEMORY_FUNCTIONS - compiling ossfuzz targets - compiling more test targets - registering some tests	2022-10-19 23:27:05 -05:00
Alexander Mohr	7213a321ad	uncompressed-blocks: Allow uncompressed blocks for all modes this commit changes that uncompressed blocks are only available for independent blocks. Building and updating the dictionary from an uncompressed block has no adverse side effects. Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>	2022-10-10 19:16:51 +02:00
Yann Collet	4dafb855a6	fixed usan32 tests the sanitizer was not enabled, due to environment variables not being passed.	2022-10-01 00:36:24 -07:00
Yann Collet	e312412c60	Merge pull request #1174 from lz4/lz4_ubnull very minor refactor of lz4.c	2022-10-01 00:11:35 -07:00
Yann Collet	e3974e5a14	minor refactor of lz4.c almost no change, functionally equivalent replaced one test by an assert() (since it should always be true)	2022-09-30 22:14:04 -07:00
Yann Collet	b9117c2b20	Merge pull request #1173 from lz4/md_off LZ4 HC match finder and parsers use direct offset values	2022-09-30 22:13:44 -07:00
Yann Collet	68848ec601	fix another ubsan warning in lz4hc because ubsan complains even about intermediate pointer arithmetic results (in a simple linear formula with 3 factors for example). use parenthesis to make sure calculations go directly from one valid memory address to another valid memory address value.	2022-09-30 18:06:53 -07:00
Yann Collet	a3d1762092	use LZ4HC_match_t structure directly to store match candidates this formalized better the coupling between match length and match offset which were 2 separated variables before.	2022-09-30 18:00:28 -07:00
Yann Collet	2fefb1da61	removed virtual pointer from optimal parser replaced by direct offset value. this virtual pointer was only used in rare _dstSize scenario.	2022-09-29 18:48:03 -07:00
Yann Collet	0a2e406d82	removed virtual match pointer from HC parser replaced by direct offset values.	2022-09-29 18:42:46 -07:00
Yann Collet	a0adc616cd	sequence encoder accepts offset as a value instead of a virtual pointer	2022-09-29 18:36:22 -07:00
Yann Collet	952942d032	LZ4 HC matchfinder returns an offset value instead of a virtual past pointer.	2022-09-29 18:29:03 -07:00
Yann Collet	4a555363ba	Merge pull request #1171 from dosaboy/snap-1.9.4 Update snapcraft.yaml to reflect build of v1.9.4	2022-09-27 16:16:03 -07:00
Yann Collet	586e9a4c6d	added code documentation on heap mode and also on decompress_continue variants.	2022-09-27 06:30:40 -07:00
Edward Hope-Morley	1ae9a50d4d	Update snapcraft.yaml to reflect build of v1.9.4	2022-09-21 19:47:11 +01:00
Yann Collet	b21490232d	added notes about LZ4_compressFrame() and stack/heap memory usage	2022-09-20 12:49:51 -07:00
Yann Collet	f39c0810bc	Merge pull request #1169 from lz4/fix_ub485 fix rare ub	2022-09-16 20:42:42 -07:00
Yann Collet	dc944197e6	fix rare ub apparently, accessing the short member of a union still requires enough space for its largest member.	2022-09-16 17:45:51 -07:00
Yann Collet	27c9530b75	Merge pull request #1163 from lz4/dict_fillOnIndex remove usages of `base` pointer	2022-09-16 12:20:34 -07:00
Yann Collet	7f54a56413	fixed minor UB warning now, even intermediate ptr arithmetic results can be UB ??	2022-09-16 09:34:15 -07:00
Yann Collet	2c8fd1142a	removed a few more usages of base ptr by making LZ4_putPosition() specific to byPtr strategy. byU32 and byU16 use LZ4_putIndexOnHash() instead. In both cases, it makes it irrelevant to pass` base as reference ptr.	2022-09-16 09:34:15 -07:00
Yann Collet	2620c0924b	remove another usage of base within `LZ4_loadDict()` function : it's possible to only employ Indexes directly.	2022-09-16 09:34:15 -07:00
Yann Collet	e84f375421	Merge pull request #1168 from lz4/benchD fix benchmark more using Dictionary	2022-09-15 22:37:40 -07:00
Yann Collet	251e04a9c1	added test able to catch bug #1167 also : errors during benchmark mode return an error code.	2022-09-15 16:48:28 -07:00
Yann Collet	ec0d3e6ec9	fix benchmark more using Dictionary benchmark dictionary mode, implemented in #808, is incorrect. It would desynchronize compression and decompression as soon as 2+ files are dictionary compressed. Also : slightly improved traces system, to also include __LINE__ number	2022-09-15 16:25:00 -07:00
Yann Collet	c619263b17	Merge pull request #1165 from lz4/testm add a test to catch issue #1164	2022-09-14 17:45:44 -07:00
Yann Collet	3c1d5812af	add a test to catch issue #1164 where `--test -m files.lz4` would create a bunch of `files` files. `--test` should not create any file. It turns out the bug was already fixed when #1164 was filled, but it also shows that the CI system was blind to such issues when it happened. The new test was run with `v1.9.3` and triggered an error, as expected.	2022-09-14 11:13:07 -07:00
Yann Collet	fdfbe3a20a	update v1.9.4 NEWS one fix was forgotten	2022-09-14 11:11:40 -07:00
Yann Collet	5799b2d4f3	document Makefile variables	2022-09-13 16:12:17 -07:00
Yann Collet	851ef4b23c	Merge pull request #1162 from foxeng/fix-static-only build: Support BUILD_SHARED=no	2022-09-13 16:06:01 -07:00
Fotis Xenakis	5ccbd38277	build: Support BUILD_SHARED=no Since `e585a438c7`, the BUILD_SHARED Makefile variable only takes effect for the install target (i.e. the shared libraries always built). This restores the original behaviour.	2022-09-13 23:55:54 +03:00
Yann Collet	72b9348f02	Clarifiy documentation for LZ4F_HEAPMODE Now it's only about LZ4F_compressFrame(), and it's only about the Compression Context (not the hash tables, which used to be integrated).	2022-09-12 18:34:48 -07:00
Yann Collet	32bfb209d3	clarify Data Block in the Frame format documentation as requested in #1156	2022-09-12 18:24:26 -07:00
Yann Collet	520330eade	Merge pull request #1161 from lz4/byPtr simplify getPosition	2022-09-12 17:59:42 -07:00
Yann Collet	f6c18481ce	simplify getPosition it's only used in byPtr mode. This removes the need to transfer `base` ptr value.	2022-09-12 16:57:42 -07:00
Yann Collet	d91d16b48a	updated documentation : no more issue with 32-bit compilation on recent compilers	2022-09-12 14:28:24 -07:00
Yann Collet	9bed6b56e9	updated Github Actions tests documentation notably regarding ubsan test status (now enabled)	2022-09-09 11:05:32 -07:00
Yann Collet	46ecb595d8	Merge pull request #1160 from lz4/ubsan_hc fixed a few remaining ubsan warnings in lz4hc	2022-09-09 10:38:15 -07:00
Yann Collet	fbd2f9f05e	fixed a few ubsan warnings in lz4hc mostly related to pointer arithmetic involving NULL ptr.	2022-09-08 22:44:12 -07:00
Yann Collet	e94d096e0e	Merge pull request #1159 from lz4/ubsan2 attempt to enable ubsan tests in CI	2022-09-08 18:39:58 -07:00
Yann Collet	8b7c57a8b1	attempt to enable ubsan tests in CI	2022-09-08 15:43:28 -07:00
Yann Collet	72997c5a91	Merge pull request #1158 from lz4/fix1157 allocation optimization for lz4frame compression	2022-09-08 07:28:54 -07:00
Yann Collet	2822825845	added LZ4F_compressUpdate() in fullbench	2022-09-07 23:51:30 -07:00
Yann Collet	cedf9cd57f	allocation optimization for lz4frame compression as noted by @yixiutt, the temporary buffer inside lz4frame compression is being invalidated at end of compression, forcing it to be re-allocated at next compression job. This shouldn't be necessary. This change was introduced in #236, as a way to fix #232, but neither the issue is explained, nor why the patch fixes it. This patch revert to previous behavior, where temporary buffer is kept between compression calls. This results in a net reduction of allocation workload. Additionally, the temporary buffer should only need malloc(), not calloc(), thus saving initialization. This diff implements both changes. Long fuzzer tests will be run to ensure that no sanitizer warning get triggered. Additionally : fixed a minor ubsan warning in LZ4F_decompress().	2022-09-07 22:50:28 -07:00
Yann Collet	4de3e98ad2	Merge pull request #1150 from lz4/dependabot/github_actions/actions/upload-artifact-3 Bump actions/upload-artifact from 1 to 3	2022-08-19 14:51:40 -07:00
dependabot[bot]	36de5a515f	Bump actions/upload-artifact from 1 to 3 Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 1 to 3. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v1...v3) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>	2022-08-19 20:48:41 +00:00
Yann Collet	3e28dde8e5	Merge pull request #1149 from lz4/dependabot/github_actions/actions/checkout-3 Bump actions/checkout from 2 to 3	2022-08-19 13:47:39 -07:00
dependabot[bot]	cd64053729	Bump actions/checkout from 2 to 3 Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>	2022-08-19 18:51:07 +00:00
Yann Collet	c327cc47c7	Merge pull request #1148 from lz4/dependabot/github_actions/actions/setup-python-4 Bump actions/setup-python from 2 to 4	2022-08-19 11:50:43 -07:00
dependabot[bot]	19532ca0df	Bump actions/setup-python from 2 to 4 Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v2...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>	2022-08-19 18:13:27 +00:00
Yann Collet	7d00e2267f	Merge pull request #1147 from tristan957/dependabot Add dependabot	2022-08-19 11:13:04 -07:00
Tristan Partin	78b30782ad	Add dependabot Dependabot is configured to update GitHub actions. It will check weekly.	2022-08-18 23:34:09 -05:00
Yann Collet	3847e5751b	Merge pull request #1142 from tristan957/ci Cancel in-progress CI if a new commit workflow supplants it	2022-08-17 02:59:45 -07:00
Tristan Partin	c871a28367	Cancel in-progress CI if a new commit workflow supplants it This is useful when repushing to a PR very quickly and when merging to dev very quickly.	2022-08-16 23:58:19 -05:00