Merge pull request #1138 from lz4/dev

stage v1.9.4
Merge branch 'dev' into release
2022-08-15 15:45:31 -07:00 · 2020-11-17 19:42:24 -08:00 · 2020-11-15 20:59:35 -08:00 · 2020-11-15 15:53:42 -08:00
51 changed files with 801 additions and 1385 deletions
@@ -1,7 +0,0 @@
-version: 2
-
-updates:
-  - package-ecosystem: github-actions
-    directory: /
-    schedule:
-      interval: weekly
@@ -4,19 +4,27 @@ This directory contains [GitHub Actions](https://github.com/features/actions) wo

 ## USAN, ASAN (`lz4-ubsan-x64`, `lz4-ubsan-x86`, `lz4-asan-x64`)

-For now, `lz4-ubsan-*` uses the `-fsanitize-recover=pointer-overflow` flag:
-there are known cases of pointer overflow arithmetic within `lz4.c` fast compression.
-These cases are not dangerous with known architecture,
-but they are not guaranteed to work by the C standard,
-which means that, in some future, some new architecture or some new compiler
-may decide to do something funny that could break this behavior.
-Hence, in anticipation, it's better to remove them.
-This has been already achieved in `lz4hc.c`.
-However, the same attempt in `lz4.c` resulted in massive speed losses,
-which is not an acceptable cost for preemptively solving a "potential future" problem
-not active anywhere today.
-Therefore, a more acceptable work-around will have to be found first.
+For now, `lz4-ubsan-*` ignores the exit code of `make usan` and `make usan32`.
+Because there are several issues which may take relatively long time to resolve.

+We'll fully enable it when we ensure `make usan` is ready for all commits and PRs.
+
+See [#983](https://github.com/lz4/lz4/pull/983) for details.
+
+
+## C Compilers (`lz4-c-compilers`)
+
+- Our test doesn't use `gcc-4.5` due to installation issue of its package.  (`apt-get install gcc-4.5` fails on GH-Actions VM)
+
+- Currently, the following 32bit executable tests fail with all versions of `clang`.
+  - `CC=clang-X CFLAGS='-O3' make V=1 -C tests clean test-lz4c32`
+  - `CC=clang-X CFLAGS='-O3 -mx32' make V=1 -C tests clean test-lz4c32`
+  - See [#991](https://github.com/lz4/lz4/issues/991) for details.
+
+- Currently, the following 32bit executable tests fail with `gcc-11`
+  - `CC=gcc-11 CFLAGS='-O3' make V=1 -C tests clean test-lz4c32`
+  - `CC=gcc-11 CFLAGS='-O3 -mx32' make V=1 -C tests clean test-lz4c32`
+  - See [#991](https://github.com/lz4/lz4/issues/991) for details.


 ## cppcheck (`lz4-cppcheck`)
@@ -42,7 +50,11 @@ Also sometimes it reports false positives.

 # Difference with `.travis.yml`

-The following tests are not be included due to limitation of GH-Actions.
+The following tests are not included yet.
+
+- name: Compile OSS-Fuzz targets
+
+The following tests will not be included due to limitation of GH-Actions.

 - name: aarch64 real-hw tests
 - name: PPC64LE real-hw tests
@@ -16,10 +16,6 @@ on: [push, pull_request]
 permissions:
  contents: read

-concurrency:
-  cancel-in-progress: true
-  group: ${{ github.workflow }}-${{ github.head_ref }}
-
 jobs:
  lz4-c-compilers:
    name: CC=${{ matrix.cc }}, ${{ matrix.os }}
@@ -60,8 +56,8 @@ jobs:
          { pkgs: 'lib32gcc-11-dev libx32gcc-11-dev',                   cc: clang,     cxx: clang++,     x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-latest, },
          { pkgs: 'clang-14  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-14,  cxx: clang++-14,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
          { pkgs: 'clang-13  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-13,  cxx: clang++-13,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
-          { pkgs: 'clang-12  lib32gcc-12-dev libx32gcc-12-dev',         cc: clang-12,  cxx: clang++-12,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
-          { pkgs: 'clang-11  lib32gcc-12-dev libx32gcc-12-dev',         cc: clang-11,  cxx: clang++-11,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
+          { pkgs: 'clang-12  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-12,  cxx: clang++-12,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
+          { pkgs: 'clang-11  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-11,  cxx: clang++-11,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-22.04,  },
          { pkgs: 'clang-10  lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-10,  cxx: clang++-10,  x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-20.04,  },
          { pkgs: 'clang-9   lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-9,   cxx: clang++-9,   x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-20.04,  },
          { pkgs: 'clang-8   lib32gcc-11-dev libx32gcc-11-dev',         cc: clang-8,   cxx: clang++-8,   x32: 'true', x86: 'true', cxxtest: 'true',  freestanding: 'false', os: ubuntu-20.04,  },
@@ -79,7 +75,7 @@ jobs:
      CXX: ${{ matrix.cxx }}
      FIXME__LZ4_CI_IGNORE : ' echo Error.  But we ignore it for now.'
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -180,7 +176,7 @@ jobs:
    name: Benchmark
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -207,7 +203,7 @@ jobs:
    name: Fuzzer test
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -228,7 +224,7 @@ jobs:
    name: LZ4 Makefile - support for standard variables
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: make standard_variables
      run: make V=1 standard_variables
@@ -238,7 +234,7 @@ jobs:
    name: LZ4 versions test
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -253,7 +249,7 @@ jobs:
    name: LZ4 inter-versions ABI test
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -268,7 +264,7 @@ jobs:
    name: LZ4 frame test
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -285,7 +281,7 @@ jobs:
    name: test different values of LZ4_MEMORY_USAGE
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: LZ4_MEMORY_USAGE
      run: make V=1 -C tests test-compile-with-lz4-memory-usage

@@ -294,7 +290,7 @@ jobs:
    name: Custom LZ4_DISTANCE_MAX
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: custom LZ4_DISTANCE_MAX; test LZ4_USER_MEMORY_FUNCTIONS
      run: |
        MOREFLAGS='-DLZ4_DISTANCE_MAX=8000' make V=1 check
@@ -310,7 +306,7 @@ jobs:
    name: Test lz4 compression on a block device
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: create a block device, compress it with lz4 # alternative : blindly use /dev/loop0, seems to always exist
      run: |
        make lz4
@@ -337,7 +333,7 @@ jobs:
    name: make cppcheck
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: apt-get install
      run: |
        sudo apt-get update
@@ -356,7 +352,7 @@ jobs:
    name: make staticAnalyze
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: apt-get install
      run: |
        sudo apt-get update
@@ -378,7 +374,7 @@ jobs:
    name: valgrind
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: apt-get install
      run: |
        sudo apt-get update
@@ -396,20 +392,27 @@ jobs:
  lz4-ubsan-x64:
    name: Linux x64 ubsan
    runs-on: ubuntu-latest
+    env:                        # Set environment variables
+      FIXME__LZ4_CI_IGNORE : ' echo Error.  But we ignore it for now.'
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: ubsan
-      run: |
-        make clean
-        make V=1 usan
+      #########################################################
+      # For now, we ignore the exit code of `make usan`.
+      # See "Known issues / lz4-ubsan-x64" in README.md
+      # When we'll resolve this issue, remove "|| $FIXME__LZ4_CI_IGNORE"
+      #########################################################
+      run: make V=1 clean usan MOREFLAGS='-Wcomma -Werror' || $FIXME__LZ4_CI_IGNORE


  lz4-ubsan-x86:
    name: Linux x86 ubsan
    runs-on: ubuntu-latest
+    env:                        # Set environment variables
+      FIXME__LZ4_CI_IGNORE : ' echo Error.  But we ignore it for now.'
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -418,16 +421,21 @@ jobs:
        sudo apt-get install lib32gcc-11-dev

    - name: ubsan32
-      run: |
-        make clean
-        CC=clang make V=1 usan32
+      #########################################################
+      # For now, we ignore the exit code of `make usan32`.
+      # See "Known issues / lz4-ubsaan-x86" in README.md.
+      # When we'll resolve this issue, remove "|| $FIXME__LZ4_CI_IGNORE"
+      #########################################################
+      run: CC=clang make V=1 clean usan32 MOREFLAGS='-Wcomma -Werror' || $FIXME__LZ4_CI_IGNORE


  lz4-asan-x64:
    name: Linux x64 ASAN
    runs-on: ubuntu-latest
+    env:                        # Set environment variables
+      FIXME__LZ4_CI_IGNORE : ' echo Error.  But we ignore it for now.'
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: setup
      run: sudo sysctl -w vm.mmap_min_addr=4096
@@ -442,7 +450,7 @@ jobs:
    name: lint unicode in ./lib/, ./tests/ and ./programs/
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: unicode lint
      run: bash ./tests/unicode_lint.sh

@@ -450,7 +458,7 @@ jobs:
    name: make examples
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: apt-get install
      run: |
        sudo apt-get update
@@ -489,7 +497,7 @@ jobs:
        dry-run: false
        sanitizer: ${{ matrix.sanitizer }}
    - name: Upload Crash
-      uses: actions/upload-artifact@v3
+      uses: actions/upload-artifact@v1
      if: failure() && steps.build.outcome == 'success'
      with:
        name: ${{ matrix.sanitizer }}-artifacts
@@ -541,7 +549,7 @@ jobs:
      XCC: ${{ matrix.xcc }}
      XEMU: ${{ matrix.xemu }}
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: apt-get install
      run: |
@@ -582,7 +590,7 @@ jobs:
    name: macOS
    runs-on: macos-latest
    steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v2

    - name: Environment info
      run: |
@@ -614,7 +622,7 @@ jobs:
    name: make
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: Environment info
      run: |
@@ -629,7 +637,7 @@ jobs:
    name: make travis-install
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: travis-install
      run: make V=1 clean travis-install
@@ -645,7 +653,7 @@ jobs:
    name: cmake
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout

    - name: Environment info
      run: |
@@ -666,7 +674,7 @@ jobs:
    name: make cmake
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
    - name: make cmake
      # V=1 for lz4 Makefile, VERBOSE=1 for cmake Makefile.
      run: make V=1 VERBOSE=1 clean cmake
@@ -677,8 +685,8 @@ jobs:
    name: Meson + Ninja
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3 # https://github.com/actions/checkout
-    - uses: actions/setup-python@v4 # https://github.com/actions/setup-python
+    - uses: actions/checkout@v2 # https://github.com/actions/checkout
+    - uses: actions/setup-python@v2 # https://github.com/actions/setup-python
      with:
        python-version: '3.x'

@@ -695,11 +703,10 @@ jobs:
        echo && type python && which python && python --version
        echo && type meson && which meson && meson --version

-    - name: setup
+    - name: meson
      # 'run: >' replaces all newlines in the following block with spaces
      run: >
        meson setup
-        --fatal-meson-warnings
        --buildtype=debug
        -Db_lundef=false
        -Dauto_features=enabled
@@ -707,15 +714,11 @@ jobs:
        -Dcontrib=true
        -Dtests=true
        -Dexamples=true
-        build/meson builddir
-
-    - name: test
-      run: |
-        meson test -C builddir
+        contrib/meson build

    - name: staging
      run: |
-        cd builddir
+        cd build
        DESTDIR=./staging ninja install
        tree ./staging

@@ -728,7 +731,7 @@ jobs:
    name: git version tag checking for release
    runs-on: ubuntu-latest
    steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v2
    - name: make -C tests checkTag
      if: startsWith(github.ref, 'refs/tags/v')   # If git tag name starts with 'v'
      run: |
@@ -754,7 +757,7 @@ jobs:

    runs-on: ${{ matrix.os }}
    steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v2

    - name: init
      run: |
@@ -162,10 +162,10 @@ usan: clean
 	CC=$(CC) CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) test FUZZER_TIME="-T30s" NB_LOOPS=-i1

 .PHONY: usan32
-usan32: CFLAGS = -m32 -O3 -g -fsanitize=undefined -fno-sanitize-recover=undefined -fsanitize-recover=pointer-overflow
+usan32: CFLAGS = -m32 -O3 -g -fsanitize=undefined
 usan32: LDFLAGS = $(CFLAGS)
 usan32: clean
-	CFLAGS='$(CFLAGS)' LDFLAGS='$(LDFLAGS)' $(MAKE) V=1 test FUZZER_TIME="-T30s" NB_LOOPS=-i1
+	$(MAKE) test FUZZER_TIME="-T30s" NB_LOOPS=-i1

 SCANBUILD ?= scan-build
 SCANBUILD_FLAGS += --status-bugs -v --force-analyze-debug-code
@@ -9,7 +9,6 @@ api  : new experimental function `LZ4F_uncompressedUpdate()`, by @alexmohr
 cli  : `--list` works on `stdin` input, by @Low-power
 cli  : `--no-crc` does not produce (compression) nor check (decompression) checksums
 cli  : fix: `--test` and `--list` produce an error code when parsing invalid input
-cli  : fix: `--test -m` does no longer create decompressed file artifacts
 cli  : fix: support skippable frames when passed via `stdin`, reported by @davidmankin
 build: fix: Makefile respects CFLAGS directives passed via environment variable
 build: `LZ4_FREESTANDING`, new build macro for freestanding environments, by @t-mat
@@ -5,7 +5,6 @@ Projects for various integrated development environments (IDE)

 The following projects are included with the lz4 distribution:
 - `cmake` - CMake project
- `meson` - Meson project
 - `VS2010` - Visual Studio 2010 project (which also works well with Visual Studio 2012, 2013, 2015)
 - `VS2017` - Visual Studio 2017 project

@@ -1,126 +0,0 @@
-# #############################################################################
-# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
-# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
-# All rights reserved.
-#
-# This source code is licensed under both the BSD-style license (found in the
-# LICENSE file in the root directory of this source tree) and the GPLv2 (found
-# in the COPYING file in the root directory of this source tree).
-# #############################################################################
-
-cc = meson.get_compiler('c')
-
-fs = import('fs')
-pkgconfig = import('pkgconfig')
-
-lz4_source_root = '../../..'
-
-add_project_arguments('-DXXH_NAMESPACE=LZ4_', language: 'c')
-
-if get_option('debug')
-  add_project_arguments(cc.get_supported_arguments(
-      '-Wcast-qual',
-      '-Wcast-align',
-      '-Wshadow',
-      '-Wswitch-enum',
-      '-Wdeclaration-after-statement',
-      '-Wstrict-prototypes',
-      '-Wundef',
-      '-Wpointer-arith',
-      '-Wstrict-aliasing=1',
-      '-DLZ4_DEBUG=@0@'.format(get_option('debug-level'))
-    ),
-    language: 'c'
-  )
-endif
-
-compile_args = []
-
-if not get_option('align-test')
-  add_project_arguments('-DLZ4_ALIGN_TEST=0', language: 'c')
-endif
-
-if get_option('disable-memory-allocation')
-  if get_option('default_library') != 'static'
-    error('Memory allocation can only be disabled in static builds')
-  endif
-
-  add_project_arguments('-DLZ4_STATIC_LINKING_ONLY_DISABLE_MEMORY_ALLOCATION')
-  compile_args += '-DLZ4_STATIC_LINKING_ONLY_DISABLE_MEMORY_ALLOCATION'
-endif
-
-add_project_arguments(
-  '-DLZ4_DISTANCE_MAX=@0@'.format(get_option('distance-max')),
-  language: 'c'
-)
-compile_args += '-DLZ4_DISTANCE_MAX=@0@'.format(get_option('distance-max'))
-
-if not get_option('fast-dec-loop').auto()
-  add_project_arguments(
-    '-DLZ4_FAST_DEC_LOOP=@0@'.format(
-      get_option('fast-dec-loop').enabled() ? 1 : 0
-    ),
-    language: 'c'
-  )
-endif
-
-if get_option('force-sw-bitcount')
-  add_project_arguments('-DLZ4_FORCE_SW_BITCOUNT', language: 'c')
-endif
-
-if get_option('freestanding')
-  add_project_arguments('-DLZ4_FREESTANDING=1', language: 'c')
-  compile_args += '-DLZ4_FREESTANDING=1'
-endif
-
-if get_option('memory-usage') > 0
-  add_project_arguments(
-    '-DLZ4_MEMORY_USAGE=@0@'.format(get_option('memory-usage')),
-    language: 'c'
-  )
-  compile_args += '-DLZ4_MEMORY_USAGE=@0@'.format(get_option('memory-usage'))
-endif
-
-if get_option('unstable')
-  add_project_arguments('-DLZ4_STATIC_LINKING_ONLY', language: 'c')
-  compile_args += '-DLZ4_STATIC_LINKING_ONLY'
-  if get_option('default_library') != 'static'
-    add_project_arguments('-DLZ4_PUBLISH_STATIC_FUNCTIONS', language: 'c')
-    compile_args += '-DLZ4_PUBLISH_STATIC_FUNCTIONS'
-
-    add_project_arguments('-DLZ4F_PUBLISH_STATIC_FUNCTIONS', language: 'c')
-    compile_args += '-DLZ4F_PUBLISH_STATIC_FUNCTIONS'
-  endif
-endif
-
-if get_option('user-memory-functions')
-  add_project_arguments('-DLZ4_USER_MEMORY_FUNCTIONS', language: 'c')
-endif
-
-run_env = environment()
-
-subdir('lib')
-
-if get_option('programs')
-  subdir('programs')
-else
-  lz4 = disabler()
-  lz4cat = disabler()
-  unlz4 = disabler()
-endif
-
-if get_option('tests')
-  subdir('tests')
-endif
-
-if get_option('contrib')
-  subdir('contrib')
-endif
-
-if get_option('examples')
-  subdir('examples')
-endif
-
-if get_option('ossfuzz')
-  subdir('ossfuzz')
-endif
@@ -1,35 +0,0 @@
-fuzzers = [
-  'compress_frame_fuzzer',
-  'compress_fuzzer',
-  'compress_hc_fuzzer',
-  'decompress_frame_fuzzer',
-  'decompress_fuzzer',
-  'round_trip_frame_uncompressed_fuzzer',
-  'round_trip_fuzzer',
-  'round_trip_hc_fuzzer',
-  'round_trip_stream_fuzzer'
-]
-
-c_args = cc.get_supported_arguments(
-  '-Wno-unused-function',
-  '-Wno-sign-compare',
-  '-Wno-declaration-after-statement'
-)
-
-foreach f : fuzzers
-  lib = static_library(
-    f,
-    lz4_source_root / 'ossfuzz/@0@.c'.format(f),
-    lz4_source_root / 'ossfuzz/lz4_helpers.c',
-    lz4_source_root / 'ossfuzz/fuzz_data_producer.c',
-    c_args: c_args,
-    dependencies: [liblz4_internal_dep]
-  )
-
-  executable(
-    f,
-    lz4_source_root / 'ossfuzz/standaloneengine.c',
-    link_with: lib,
-    dependencies: [liblz4_internal_dep]
-  )
-endforeach
@@ -1,158 +0,0 @@
-# #############################################################################
-# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
-# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
-# All rights reserved.
-#
-# This source code is licensed under both the BSD-style license (found in the
-# LICENSE file in the root directory of this source tree) and the GPLv2 (found
-# in the COPYING file in the root directory of this source tree).
-# #############################################################################
-
-lz4_source_root = '../../../..'
-
-fuzzer_time = 90
-test_exes = {
-  'abiTest': {
-    'sources': files(lz4_source_root / 'tests/abiTest.c'),
-    'test': false,
-  },
-  'checkFrame': {
-    'sources': files(lz4_source_root / 'tests/checkFrame.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-  },
-  'checkTag': {
-    'sources': files(lz4_source_root / 'tests/checkTag.c'),
-    'test': false,
-  },
-  'datagen': {
-    'sources': files(lz4_source_root / 'tests/datagencli.c'),
-    'objects': lz4.extract_objects(lz4_source_root / 'programs/datagen.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-  },
-  'decompress-partial-usingDict.c': {
-    'sources': files(lz4_source_root / 'tests/decompress-partial-usingDict.c'),
-  },
-  'decompress-partial.c': {
-    'sources': files(lz4_source_root / 'tests/decompress-partial.c'),
-  },
-  'frametest': {
-    'sources': files(lz4_source_root / 'tests/frametest.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-    'args': ['-v', '-T@0@s'.format(fuzzer_time)],
-    'test': false,
-  },
-  'freestanding': {
-    'sources': files(lz4_source_root / 'tests/freestanding.c'),
-    'c_args': ['-ffreestanding', '-Wno-unused-parameter', '-Wno-declaration-after-statement'],
-    'link_args': ['-nostdlib'],
-    'build': cc.get_id() in ['gcc', 'clang'] and
-      host_machine.system() == 'linux' and host_machine.cpu_family() == 'x86_64',
-    'override_options': ['optimization=1'],
-  },
-  'fullbench': {
-    'sources': files(lz4_source_root / 'tests/fullbench.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-    'args': ['--no-prompt', '-i1', files(lz4_source_root / 'tests/COPYING')],
-    'test': false,
-  },
-  'fuzzer': {
-    'sources': files(lz4_source_root / 'tests/fuzzer.c'),
-    'include_directories': include_directories(lz4_source_root / 'programs'),
-    'args': ['-T@0@s'.format(fuzzer_time)],
-    'test': false,
-  },
-  'roundTripTest': {
-    'sources': files(lz4_source_root / 'tests/roundTripTest.c'),
-    'test': false,
-  },
-}
-
-targets = {}
-
-foreach e, attrs : test_exes
-  if not attrs.get('build', true)
-    targets += {e: disabler()}
-    continue
-  endif
-
-  t = executable(
-    e,
-    attrs.get('sources'),
-    c_args: attrs.get('c_args', []),
-    link_args: attrs.get('link_args', []),
-    objects: attrs.get('objects', []),
-    dependencies: [liblz4_internal_dep],
-    include_directories: attrs.get('include_directories', []),
-    install: false,
-    override_options: attrs.get('override_options', [])
-  )
-
-  targets += {e: t}
-
-  if not attrs.get('test', true)
-    continue
-  endif
-
-  test(
-    e,
-    t,
-    args: attrs.get('params', []),
-    timeout: 120
-  )
-endforeach
-
-fs = import('fs')
-
-run_env.prepend('PATH', meson.current_build_dir())
-
-test_scripts = {
-  'lz4-basic': {
-    'depends': [lz4, lz4cat, unlz4, targets['datagen']],
-  },
-  'lz4-dict': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4-contentSize': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4-fast-hugefile': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4-frame-concatenation': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4-multiple': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4-multiple-legacy': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4-opt-parser': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4-skippable': {
-    'depends': [lz4],
-  },
-  'lz4-sparse': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4-testmode': {
-    'depends': [lz4, targets['datagen']],
-  },
-  'lz4hc-hugefile': {
-    'depends': [lz4, targets['datagen']],
-  },
-}
-
-foreach s, attrs : test_scripts
-  script = find_program(lz4_source_root / 'tests/test-@0@.sh'.format(s))
-
-  test(
-    '@0@'.format(s),
-    script,
-    depends: attrs.get('depends', []),
-    workdir: fs.parent(script.full_path()),
-    env: run_env,
-    timeout: 360
-  )
-endforeach
@@ -1,40 +0,0 @@
-# #############################################################################
-# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
-# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
-# All rights reserved.
-#
-# This source code is licensed under both the BSD-style license (found in the
-# LICENSE file in the root directory of this source tree) and the GPLv2 (found
-# in the COPYING file in the root directory of this source tree).
-# #############################################################################
-
-option('align-test', type: 'boolean', value: true,
-  description: 'See LZ4_ALIGN_TEST')
-option('contrib', type: 'boolean', value: false,
-  description: 'Enable contrib')
-option('debug-level', type: 'integer', min: 0, max: 7, value: 1,
-  description: 'Enable run-time debug. See lib/lz4hc.c')
-option('disable-memory-allocation', type: 'boolean', value: false,
-  description: 'See LZ4_STATIC_LINKING_ONLY_DISABLE_MEMORY_ALLOCATION. Static builds only')
-option('distance-max', type: 'integer', min: 0, max: 65535, value: 65535,
-  description: 'See LZ4_DISTANCE_MAX')
-option('examples', type: 'boolean', value: false,
-  description: 'Enable examples')
-option('fast-dec-loop', type: 'feature', value: 'auto',
-  description: 'See LZ4_FAST_DEC_LOOP')
-option('force-sw-bitcount', type: 'boolean', value: false,
-  description: 'See LZ4_FORCE_SW_BITCOUNT')
-option('freestanding', type: 'boolean', value: false,
-  description: 'See LZ4_FREESTANDING')
-option('memory-usage', type: 'integer', min: 0, max: 20, value: 0,
-  description: 'See LZ4_MEMORY_USAGE. 0 means use the LZ4 default')
-option('ossfuzz', type: 'boolean', value: true,
-  description: 'Enable ossfuzz')
-option('programs', type: 'boolean', value: false,
-  description: 'Enable programs')
-option('tests', type: 'boolean', value: false,
-  description: 'Enable tests')
-option('unstable', type: 'boolean', value: false,
-  description: 'Expose unstable interfaces')
-option('user-memory-functions', type: 'boolean', value: false,
-  description: 'See LZ4_USER_MEMORY_FUNCTIONS')
@@ -13,19 +13,15 @@

 project(
  'lz4',
-  'c',
+  ['c'],
  license: 'BSD-2-Clause-Patent AND GPL-2.0-or-later',
  default_options: [
    'c_std=c99',
    'buildtype=release',
    'warning_level=3'
  ],
-  version: run_command(
-    find_program('GetLz4LibraryVersion.py'),
-    '../../lib/lz4.h',
-    check: true
-  ).stdout().strip(),
-  meson_version: '>=0.58.0'
+  version: 'DUMMY',
+  meson_version: '>=0.49.0'
 )

 subdir('meson')
@@ -10,7 +10,7 @@

 lz4_source_root = '../../../../..'

-add_languages('cpp', native: true)
+add_languages('cpp')

 sources = files(
  lz4_source_root / 'contrib/gen_manual/gen_manual.cpp'
@@ -33,7 +33,7 @@ foreach mp : manual_pages
    output: '@0@_manual.html'.format(mp),
    command: [
      gen_manual,
-      meson.project_version(),
+      lz4_version,
      '@INPUT@',
      '@OUTPUT@',
    ],
@@ -17,49 +17,41 @@ sources = files(
  lz4_source_root / 'lib/xxhash.c'
 )

-if get_option('unstable')
-  sources += files(lz4_source_root / 'lib/lz4file.c')
-endif
-
 c_args = []

 if host_machine.system() == 'windows' and get_option('default_library') != 'static'
  c_args += '-DLZ4_DLL_EXPORT=1'
 endif

+if get_option('unstable')
+  compile_args += '-DLZ4_STATIC_LINKING_ONLY'
+  if get_option('default_library') != 'static'
+    c_args += '-DLZ4_PUBLISH_STATIC_FUNCTIONS'
+  endif
+endif
+
 liblz4 = library(
  'lz4',
  sources,
-  c_args: c_args,
  install: true,
-  version: meson.project_version(),
+  version: lz4_version,
  gnu_symbol_visibility: 'hidden'
 )

 liblz4_dep = declare_dependency(
  link_with: liblz4,
-  compile_args: compile_args,
  include_directories: include_directories(lz4_source_root / 'lib')
 )

-meson.override_dependency('liblz4', liblz4_dep)
-
-if get_option('tests') or get_option('programs') or get_option('examples') or get_option('ossfuzz')
-  if get_option('default_library') == 'shared'
-    liblz4_internal = static_library(
-      'lz4-internal',
-      objects: liblz4.extract_all_objects(recursive: true),
-      gnu_symbol_visibility: 'hidden'
-    )
-  elif get_option('default_library') == 'static'
-    liblz4_internal = liblz4
-  elif get_option('default_library') == 'both'
-    liblz4_internal = liblz4.get_static_lib()
-  endif
+if get_option('tests') or get_option('programs') or get_option('examples')
+  liblz4_internal = static_library(
+    'lz4-internal',
+    objects: liblz4.extract_all_objects(recursive: true),
+    gnu_symbol_visibility: 'hidden'
+  )

  liblz4_internal_dep = declare_dependency(
    link_with: liblz4_internal,
-    compile_args: compile_args,
    include_directories: include_directories(lz4_source_root / 'lib')
  )
 endif
@@ -69,7 +61,7 @@ pkgconfig.generate(
  name: 'lz4',
  filebase: 'liblz4',
  description: 'extremely fast lossless compression algorithm library',
-  version: meson.project_version(),
+  version: lz4_version,
  url: 'http://www.lz4.org/'
 )

@@ -81,7 +73,4 @@ install_headers(

 if get_option('default_library') != 'shared'
  install_headers(lz4_source_root / 'lib/lz4frame_static.h')
-  if get_option('unstable')
-    install_headers(lz4_source_root / 'lib/lz4file.h')
-  endif
 endif
@@ -0,0 +1,67 @@
+# #############################################################################
+# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
+# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
+# All rights reserved.
+#
+# This source code is licensed under both the BSD-style license (found in the
+# LICENSE file in the root directory of this source tree) and the GPLv2 (found
+# in the COPYING file in the root directory of this source tree).
+# #############################################################################
+
+cc = meson.get_compiler('c')
+
+pkgconfig = import('pkgconfig')
+
+lz4_source_root = '../../..'
+
+lz4_version = meson.project_version()
+
+lz4_h_file = lz4_source_root / 'lib/lz4.h'
+GetLz4LibraryVersion_py = find_program('GetLz4LibraryVersion.py')
+lz4_version = run_command(GetLz4LibraryVersion_py, lz4_h_file, check: true).stdout().strip()
+message('Project version is now: @0@'.format(lz4_version))
+
+add_project_arguments('-DXXH_NAMESPACE=LZ4_', language: 'c')
+
+if get_option('debug')
+  add_project_arguments(cc.get_supported_arguments([
+        '-Wcast-qual',
+        '-Wcast-align',
+        '-Wshadow',
+        '-Wswitch-enum',
+        '-Wdeclaration-after-statement',
+        '-Wstrict-prototypes',
+        '-Wundef',
+        '-Wpointer-arith',
+        '-Wstrict-aliasing=1',
+        '-DLZ4_DEBUG=@0@'.format(get_option('debug-level')),
+      ]
+    ),
+    language: 'c',
+  )
+endif
+
+if get_option('memory-usage') > 0
+  add_project_arguments(
+    '-DLZ4_MEMORY_USAGE=@0@'.format(get_option('memory-usage')),
+    language: 'c'
+  )
+endif
+
+subdir('lib')
+
+if get_option('programs')
+  subdir('programs')
+endif
+
+if get_option('tests')
+  subdir('tests')
+endif
+
+if get_option('contrib')
+  subdir('contrib')
+endif
+
+if get_option('examples')
+  subdir('examples')
+endif
@@ -26,36 +26,6 @@ lz4 = executable(
  install: true
 )

-lz4cat = custom_target(
-  'lz4cat',
-  input: lz4,
-  output: 'lz4cat',
-  command: [
-    'ln',
-    '--symbolic',
-    '--force',
-    fs.name(lz4.full_path()),
-    '@OUTPUT@'
-  ]
-)
-
-unlz4 = custom_target(
-  'unlz4',
-  input: lz4,
-  output: 'unlz4',
-  command: [
-    'ln',
-    '--symbolic',
-    '--force',
-    fs.name(lz4.full_path()),
-    '@OUTPUT@'
-  ]
-)
-
-meson.override_find_program('lz4', lz4)
-
-run_env.prepend('PATH', meson.current_build_dir())
-
 install_man(lz4_source_root / 'programs/lz4.1')

 if meson.version().version_compare('>=0.61.0')
@@ -0,0 +1,52 @@
+# #############################################################################
+# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
+# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
+# All rights reserved.
+#
+# This source code is licensed under both the BSD-style license (found in the
+# LICENSE file in the root directory of this source tree) and the GPLv2 (found
+# in the COPYING file in the root directory of this source tree).
+# #############################################################################
+
+lz4_source_root = '../../../..'
+
+exes = {
+  'fullbench': {
+    'sources': files(lz4_source_root / 'tests/fullbench.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+  },
+  'fuzzer': {
+    'sources': files(lz4_source_root / 'tests/fuzzer.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+  },
+  'frametest': {
+    'sources': files(lz4_source_root / 'tests/frametest.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+  },
+  'roundTripTest': {
+    'sources': files(lz4_source_root / 'tests/roundTripTest.c'),
+  },
+  'datagen': {
+    'sources': files(lz4_source_root / 'tests/datagencli.c'),
+    'objects': lz4.extract_objects(lz4_source_root / 'programs/datagen.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+  },
+  'checkFrame': {
+    'sources': files(lz4_source_root / 'tests/checkFrame.c'),
+    'include_directories': include_directories(lz4_source_root / 'programs'),
+  },
+  'checkTag': {
+    'sources': files(lz4_source_root / 'tests/checkTag.c'),
+  },
+}
+
+foreach e, attrs : exes
+  executable(
+    e,
+    attrs.get('sources'),
+    objects: attrs.get('objects', []),
+    dependencies: [liblz4_internal_dep],
+    include_directories: attrs.get('include_directories', []),
+    install: false
+  )
+endforeach
@@ -0,0 +1,24 @@
+# #############################################################################
+# Copyright (c) 2018-present        lzutao <taolzu(at)gmail.com>
+# Copyright (c) 2022-present        Tristan Partin <tristan(at)partin.io>
+# All rights reserved.
+#
+# This source code is licensed under both the BSD-style license (found in the
+# LICENSE file in the root directory of this source tree) and the GPLv2 (found
+# in the COPYING file in the root directory of this source tree).
+# #############################################################################
+
+option('debug-level', type: 'integer', min: 0, max: 7, value: 1,
+  description: 'Enable run-time debug. See lib/lz4hc.c')
+option('unstable', type: 'boolean', value: false,
+  description: 'Expose unstable interfaces')
+option('programs', type: 'boolean', value: false,
+  description: 'Enable programs build')
+option('tests', type: 'boolean', value: false,
+  description: 'Enable tests build')
+option('contrib', type: 'boolean', value: false,
+  description: 'Enable contrib build')
+option('examples', type: 'boolean', value: false,
+  description: 'Enable examples build')
+option('memory-usage', type: 'integer', min: 0, value: 0,
+  description: 'See LZ4_MEMORY_USAGE. 0 means use the LZ4 default')
@@ -1,5 +1,5 @@
 name: lz4
-version: 1.9.4 
+version: 1.9.3 
 summary: Extremely Fast Compression algorithm 
 description: >
    LZ4 is lossless compression algorithm, providing compression
@@ -20,7 +20,6 @@ description: >
    LZ4 library is provided as open-source software using BSD 2-Clause license.
 confinement: strict
 grade: stable
-base: core20

 apps:
  lz4:
@@ -16,7 +16,7 @@ Distribution of this document is unlimited.

 ### Version

-1.6.3 (12/09/2022)
+1.6.2 (12/08/2020)


 Introduction
@@ -61,9 +61,9 @@ and associated error message explaining which parameter is unsupported.
 General Structure of LZ4 Frame format
 -------------------------------------

-| MagicNb | F. Descriptor | Data Block | (...) | EndMark | C. Checksum |
-|:-------:|:-------------:| ---------- | ----- | ------- | ----------- |
-| 4 bytes |  3-15 bytes   |            |       | 4 bytes | 0-4 bytes   |
+| MagicNb | F. Descriptor | Block | (...) | EndMark | C. Checksum |
+|:-------:|:-------------:| ----- | ----- | ------- | ----------- |
+| 4 bytes |  3-15 bytes   |       |       | 4 bytes | 0-4 bytes   |

 __Magic Number__

@@ -269,7 +269,7 @@ The size does not include the block checksum if present.

 _Block_Size_ shall never be larger than _Block_Maximum_Size_.
 Such an outcome could potentially happen for non-compressible sources.
-In such a case, such data block **must** be passed using uncompressed format.
+In such a case, such data block must be passed using uncompressed format.

 A value of `0x00000000` is invalid, and signifies an _EndMark_ instead.
 Note that this is different from a value of `0x80000000` (highest bit set),
@@ -397,8 +397,6 @@ and trigger an error if it does not fit within acceptable range.
 Version changes
 ---------------

-1.6.3 : minor : clarify Data Block
-
 1.6.2 : clarifies specification of _EndMark_

 1.6.1 : introduced terms "LZ4 Frame Header" and "LZ4 Frame Footer"
@@ -112,21 +112,17 @@ liblz4-dll.o: liblz4-dll.rc
 	$(WINDRES) -i liblz4-dll.rc -o liblz4-dll.o

 $(LIBLZ4): $(SRCFILES) liblz4-dll.o
-ifeq ($(BUILD_SHARED),yes)
 	@echo compiling dynamic library $(LIBVER)
 	$(CC) $(FLAGS) -DLZ4_DLL_EXPORT=1 -shared $^ -o dll/$@.dll -Wl,--out-implib,dll/$(LIBLZ4_EXP)
-endif

 else   # not windows

 $(LIBLZ4): $(SRCFILES)
-ifeq ($(BUILD_SHARED),yes)
 	@echo compiling dynamic library $(LIBVER)
 	$(CC) $(FLAGS) -shared $^ -fPIC -fvisibility=hidden $(SONAME_FLAGS) -o $@
 	@echo creating versioned links
 	$(LN_SF) $@ liblz4.$(SHARED_EXT_MAJOR)
 	$(LN_SF) $@ liblz4.$(SHARED_EXT)
-endif

 endif

@@ -189,7 +185,6 @@ install: lib liblz4.pc
  ifeq ($(BUILD_STATIC),yes)
 	$(INSTALL_DATA) liblz4.a $(DESTDIR)$(libdir)/liblz4.a
 	$(INSTALL_DATA) lz4frame_static.h $(DESTDIR)$(includedir)/lz4frame_static.h
-	$(INSTALL_DATA) lz4file.h $(DESTDIR)$(includedir)/lz4file.h
  endif
  ifeq ($(BUILD_SHARED),yes)
 # Traditionally, one installs the DLLs in the bin directory as programs
@@ -225,7 +220,6 @@ uninstall:
 	$(RM) $(DESTDIR)$(includedir)/lz4hc.h
 	$(RM) $(DESTDIR)$(includedir)/lz4frame.h
 	$(RM) $(DESTDIR)$(includedir)/lz4frame_static.h
-	$(RM) $(DESTDIR)$(includedir)/lz4file.h
 	@echo lz4 libraries successfully uninstalled

 endif
@@ -100,7 +100,7 @@ The following build macro can be selected to adjust source code behavior at comp
  passed as argument to become a compression state is suitably aligned.
  This test can be disabled if it proves flaky, by setting this value to 0.

- `LZ4_USER_MEMORY_FUNCTIONS` : replace calls to `<stdlib.h>`'s `malloc()`, `calloc()` and `free()`
+- `LZ4_USER_MEMORY_FUNCTIONS` : replace calls to `<stdlib,h>`'s `malloc()`, `calloc()` and `free()`
  by user-defined functions, which must be named `LZ4_malloc()`, `LZ4_calloc()` and `LZ4_free()`.
  User functions must be available at link time.

@@ -115,24 +115,6 @@ The following build macro can be selected to adjust source code behavior at comp
  (embedded, bootloader, etc).
  For more details, see description of this macro in `lib/lz4.h`.

- `LZ4_HEAPMODE` : Select how stateless compression functions like `LZ4_compress_default()`
-  allocate memory for their hash table,
-  in memory stack (0:default, fastest), or in memory heap (1:requires malloc()).
-
- `LZ4HC_HEAPMODE` :  Select how stateless HC compression functions like `LZ4_compress_HC()`
-  allocate memory for their workspace:
-  in stack (0), or in heap (1:default).
-  Since workspace is rather large, stack can be inconvenient, hence heap mode is recommended.
-
- `LZ4F_HEAPMODE` : selects how `LZ4F_compressFrame()` allocates the compression state,
-  either on stack (default, value 0) or using heap memory (value 1).
-
-
-#### Makefile variables
-
-The following `Makefile` variables can be selected to alter the profile of produced binaries :
- `BUILD_SHARED` : generate `libzstd` dynamic library (enabled by default)
- `BUILD_STATIC` : generate `libzstd` static library (enabled by default)


 #### Amalgamation
@@ -37,8 +37,7 @@
 **************************************/
 /*
 * LZ4_HEAPMODE :
- * Select how stateless compression functions like `LZ4_compress_default()`
- * allocate memory for their hash table,
+ * Select how default compression functions will allocate memory for their hash table,
 * in memory stack (0:default, fastest), or in memory heap (1:requires malloc()).
 */
 #ifndef LZ4_HEAPMODE
@@ -280,7 +279,7 @@ static const int LZ4_minLength = (MFLIMIT+1);
   static int g_debuglog_enable = 1;
 #  define DEBUGLOG(l, ...) {                          \
        if ((g_debuglog_enable) && (l<=LZ4_DEBUG)) {  \
-            fprintf(stderr, __FILE__  " %i: ", __LINE__); \
+            fprintf(stderr, __FILE__ ": ");           \
            fprintf(stderr, __VA_ARGS__);             \
            fprintf(stderr, " \n");                   \
    }   }
@@ -380,16 +379,14 @@ static void LZ4_write32(void* memPtr, U32 value) { *(U32*)memPtr = value; }

 /* __pack instructions are safer, but compiler specific, hence potentially problematic for some compilers */
 /* currently only defined for gcc and icc */
-typedef struct { U16 u16; } __attribute__((packed)) LZ4_unalign16;
-typedef struct { U32 u32; } __attribute__((packed)) LZ4_unalign32;
-typedef struct { reg_t uArch; } __attribute__((packed)) LZ4_unalignST;
+typedef union { U16 u16; U32 u32; reg_t uArch; } __attribute__((packed)) LZ4_unalign;

-static U16 LZ4_read16(const void* ptr) { return ((const LZ4_unalign16*)ptr)->u16; }
-static U32 LZ4_read32(const void* ptr) { return ((const LZ4_unalign32*)ptr)->u32; }
-static reg_t LZ4_read_ARCH(const void* ptr) { return ((const LZ4_unalignST*)ptr)->uArch; }
+static U16 LZ4_read16(const void* ptr) { return ((const LZ4_unalign*)ptr)->u16; }
+static U32 LZ4_read32(const void* ptr) { return ((const LZ4_unalign*)ptr)->u32; }
+static reg_t LZ4_read_ARCH(const void* ptr) { return ((const LZ4_unalign*)ptr)->uArch; }

-static void LZ4_write16(void* memPtr, U16 value) { ((LZ4_unalign16*)memPtr)->u16 = value; }
-static void LZ4_write32(void* memPtr, U32 value) { ((LZ4_unalign32*)memPtr)->u32 = value; }
+static void LZ4_write16(void* memPtr, U16 value) { ((LZ4_unalign*)memPtr)->u16 = value; }
+static void LZ4_write32(void* memPtr, U32 value) { ((LZ4_unalign*)memPtr)->u32 = value; }

 #else  /* safe and portable access using memcpy() */

@@ -806,19 +803,23 @@ LZ4_FORCE_INLINE void LZ4_putIndexOnHash(U32 idx, U32 h, void* tableBase, tableT
    }
 }

-/* LZ4_putPosition*() : only used in byPtr mode */
 LZ4_FORCE_INLINE void LZ4_putPositionOnHash(const BYTE* p, U32 h,
-                                  void* tableBase, tableType_t const tableType)
+                                  void* tableBase, tableType_t const tableType,
+                            const BYTE* srcBase)
 {
-    const BYTE** const hashTable = (const BYTE**)tableBase;
-    assert(tableType == byPtr); (void)tableType;
-    hashTable[h] = p;
+    switch (tableType)
+    {
+    case clearedTable: { /* illegal! */ assert(0); return; }
+    case byPtr: { const BYTE** hashTable = (const BYTE**)tableBase; hashTable[h] = p; return; }
+    case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] = (U32)(p-srcBase); return; }
+    case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] = (U16)(p-srcBase); return; }
+    }
 }

-LZ4_FORCE_INLINE void LZ4_putPosition(const BYTE* p, void* tableBase, tableType_t tableType)
+LZ4_FORCE_INLINE void LZ4_putPosition(const BYTE* p, void* tableBase, tableType_t tableType, const BYTE* srcBase)
 {
    U32 const h = LZ4_hashPosition(p, tableType);
-    LZ4_putPositionOnHash(p, h, tableBase, tableType);
+    LZ4_putPositionOnHash(p, h, tableBase, tableType, srcBase);
 }

 /* LZ4_getIndexOnHash() :
@@ -843,18 +844,20 @@ LZ4_FORCE_INLINE U32 LZ4_getIndexOnHash(U32 h, const void* tableBase, tableType_
    assert(0); return 0;  /* forbidden case */
 }

-static const BYTE* LZ4_getPositionOnHash(U32 h, const void* tableBase, tableType_t tableType)
+static const BYTE* LZ4_getPositionOnHash(U32 h, const void* tableBase, tableType_t tableType, const BYTE* srcBase)
 {
-    assert(tableType == byPtr); (void)tableType;
-    { const BYTE* const* hashTable = (const BYTE* const*) tableBase; return hashTable[h]; }
+    if (tableType == byPtr) { const BYTE* const* hashTable = (const BYTE* const*) tableBase; return hashTable[h]; }
+    if (tableType == byU32) { const U32* const hashTable = (const U32*) tableBase; return hashTable[h] + srcBase; }
+    { const U16* const hashTable = (const U16*) tableBase; return hashTable[h] + srcBase; }   /* default, to ensure a return */
 }

 LZ4_FORCE_INLINE const BYTE*
 LZ4_getPosition(const BYTE* p,
-                const void* tableBase, tableType_t tableType)
+                const void* tableBase, tableType_t tableType,
+                const BYTE* srcBase)
 {
    U32 const h = LZ4_hashPosition(p, tableType);
-    return LZ4_getPositionOnHash(h, tableBase, tableType);
+    return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase);
 }

 LZ4_FORCE_INLINE void
@@ -900,7 +903,7 @@ LZ4_prepareTable(LZ4_stream_t_internal* const cctx,

 /** LZ4_compress_generic() :
 *  inlined, to ensure branches are decided at compilation time.
- *  The following conditions are presumed already validated:
+ *  Presumed already validated at this stage:
 *  - source != NULL
 *  - inputSize > 0
 */
@@ -918,10 +921,10 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(
                 const int acceleration)
 {
    int result;
-    const BYTE* ip = (const BYTE*)source;
+    const BYTE* ip = (const BYTE*) source;

    U32 const startIndex = cctx->currentOffset;
-    const BYTE* base = (const BYTE*)source - startIndex;
+    const BYTE* base = (const BYTE*) source - startIndex;
    const BYTE* lowLimit;

    const LZ4_stream_t_internal* dictCtx = (const LZ4_stream_t_internal*) cctx->dictCtx;
@@ -929,8 +932,7 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(
        dictDirective == usingDictCtx ? dictCtx->dictionary : cctx->dictionary;
    const U32 dictSize =
        dictDirective == usingDictCtx ? dictCtx->dictSize : cctx->dictSize;
-    const U32 dictDelta =
-        (dictDirective == usingDictCtx) ? startIndex - dictCtx->currentOffset : 0;   /* make indexes in dictCtx comparable with indexes in current context */
+    const U32 dictDelta = (dictDirective == usingDictCtx) ? startIndex - dictCtx->currentOffset : 0;   /* make indexes in dictCtx comparable with index in current context */

    int const maybe_extMem = (dictDirective == usingExtDict) || (dictDirective == usingDictCtx);
    U32 const prefixIdxLimit = startIndex - dictSize;   /* used when dictDirective == dictSmall */
@@ -955,11 +957,11 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(

    DEBUGLOG(5, "LZ4_compress_generic_validated: srcSize=%i, tableType=%u", inputSize, tableType);
    assert(ip != NULL);
-    if (tableType == byU16) assert(inputSize<LZ4_64Klimit);  /* Size too large (not within 64K limit) */
-    if (tableType == byPtr) assert(dictDirective==noDict);   /* only supported use case with byPtr */
    /* If init conditions are not met, we don't have to mark stream
     * as having dirty context, since no action was taken yet */
    if (outputDirective == fillOutput && maxOutputSize < 1) { return 0; } /* Impossible to store anything */
+    if ((tableType == byU16) && (inputSize>=LZ4_64Klimit)) { return 0; }  /* Size too large (not within 64K limit) */
+    if (tableType==byPtr) assert(dictDirective==noDict);      /* only supported use case with byPtr */
    assert(acceleration >= 1);

    lowLimit = (const BYTE*)source - (dictDirective == withPrefix64k ? dictSize : 0);
@@ -979,12 +981,7 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(
    if (inputSize<LZ4_minLength) goto _last_literals;        /* Input too small, no compression (all literals) */

    /* First Byte */
-    {   U32 const h = LZ4_hashPosition(ip, tableType);
-        if (tableType == byPtr) {
-            LZ4_putPositionOnHash(ip, h, cctx->hashTable, byPtr);
-        } else {
-            LZ4_putIndexOnHash(startIndex, h, cctx->hashTable, tableType);
-    }   }
+    LZ4_putPosition(ip, cctx->hashTable, tableType, base);
    ip++; forwardH = LZ4_hashPosition(ip, tableType);

    /* Main Loop */
@@ -1007,9 +1004,9 @@ LZ4_FORCE_INLINE int LZ4_compress_generic_validated(
                if (unlikely(forwardIp > mflimitPlusOne)) goto _last_literals;
                assert(ip < mflimitPlusOne);

-                match = LZ4_getPositionOnHash(h, cctx->hashTable, tableType);
+                match = LZ4_getPositionOnHash(h, cctx->hashTable, tableType, base);
                forwardH = LZ4_hashPosition(forwardIp, tableType);
-                LZ4_putPositionOnHash(ip, h, cctx->hashTable, tableType);
+                LZ4_putPositionOnHash(ip, h, cctx->hashTable, tableType, base);

            } while ( (match+LZ4_DISTANCE_MAX < ip)
                   || (LZ4_read32(match) != LZ4_read32(ip)) );
@@ -1207,19 +1204,13 @@ _next_match:
        if (ip >= mflimitPlusOne) break;

        /* Fill table */
-        {   U32 const h = LZ4_hashPosition(ip-2, tableType);
-            if (tableType == byPtr) {
-                LZ4_putPositionOnHash(ip-2, h, cctx->hashTable, byPtr);
-            } else {
-                U32 const idx = (U32)((ip-2) - base);
-                LZ4_putIndexOnHash(idx, h, cctx->hashTable, tableType);
-        }   }
+        LZ4_putPosition(ip-2, cctx->hashTable, tableType, base);

        /* Test next position */
        if (tableType == byPtr) {

-            match = LZ4_getPosition(ip, cctx->hashTable, tableType);
-            LZ4_putPosition(ip, cctx->hashTable, tableType);
+            match = LZ4_getPosition(ip, cctx->hashTable, tableType, base);
+            LZ4_putPosition(ip, cctx->hashTable, tableType, base);
            if ( (match+LZ4_DISTANCE_MAX >= ip)
              && (LZ4_read32(match) == LZ4_read32(ip)) )
            { token=op++; *token=0; goto _next_match; }
@@ -1233,7 +1224,6 @@ _next_match:
            if (dictDirective == usingDictCtx) {
                if (matchIndex < startIndex) {
                    /* there was no match, try the dictionary */
-                    assert(tableType == byU32);
                    matchIndex = LZ4_getIndexOnHash(h, dictCtx->hashTable, byU32);
                    match = dictBase + matchIndex;
                    lowLimit = dictionary;   /* required for match length counter */
@@ -1387,10 +1377,9 @@ int LZ4_compress_fast_extState(void* state, const char* source, char* dest, int
 */
 int LZ4_compress_fast_extState_fastReset(void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration)
 {
-    LZ4_stream_t_internal* const ctx = &((LZ4_stream_t*)state)->internal_donotuse;
+    LZ4_stream_t_internal* ctx = &((LZ4_stream_t*)state)->internal_donotuse;
    if (acceleration < 1) acceleration = LZ4_ACCELERATION_DEFAULT;
    if (acceleration > LZ4_ACCELERATION_MAX) acceleration = LZ4_ACCELERATION_MAX;
-    assert(ctx != NULL);

    if (dstCapacity >= LZ4_compressBound(srcSize)) {
        if (srcSize < LZ4_64Klimit) {
@@ -1424,17 +1413,17 @@ int LZ4_compress_fast_extState_fastReset(void* state, const char* src, char* dst
 }


-int LZ4_compress_fast(const char* src, char* dest, int srcSize, int dstCapacity, int acceleration)
+int LZ4_compress_fast(const char* source, char* dest, int inputSize, int maxOutputSize, int acceleration)
 {
    int result;
 #if (LZ4_HEAPMODE)
-    LZ4_stream_t* const ctxPtr = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t));   /* malloc-calloc always properly aligned */
+    LZ4_stream_t* ctxPtr = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t));   /* malloc-calloc always properly aligned */
    if (ctxPtr == NULL) return 0;
 #else
    LZ4_stream_t ctx;
    LZ4_stream_t* const ctxPtr = &ctx;
 #endif
-    result = LZ4_compress_fast_extState(ctxPtr, src, dest, srcSize, dstCapacity, acceleration);
+    result = LZ4_compress_fast_extState(ctxPtr, source, dest, inputSize, maxOutputSize, acceleration);

 #if (LZ4_HEAPMODE)
    FREEMEM(ctxPtr);
@@ -1443,9 +1432,9 @@ int LZ4_compress_fast(const char* src, char* dest, int srcSize, int dstCapacity,
 }


-int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity)
+int LZ4_compress_default(const char* src, char* dst, int srcSize, int maxOutputSize)
 {
-    return LZ4_compress_fast(src, dst, srcSize, dstCapacity, 1);
+    return LZ4_compress_fast(src, dst, srcSize, maxOutputSize, 1);
 }


@@ -1472,11 +1461,11 @@ static int LZ4_compress_destSize_extState (LZ4_stream_t* state, const char* src,
 int LZ4_compress_destSize(const char* src, char* dst, int* srcSizePtr, int targetDstSize)
 {
 #if (LZ4_HEAPMODE)
-    LZ4_stream_t* const ctx = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t));   /* malloc-calloc always properly aligned */
+    LZ4_stream_t* ctx = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t));   /* malloc-calloc always properly aligned */
    if (ctx == NULL) return 0;
 #else
    LZ4_stream_t ctxBody;
-    LZ4_stream_t* const ctx = &ctxBody;
+    LZ4_stream_t* ctx = &ctxBody;
 #endif

    int result = LZ4_compress_destSize_extState(ctx, src, dst, srcSizePtr, targetDstSize);
@@ -1551,11 +1540,11 @@ int LZ4_freeStream (LZ4_stream_t* LZ4_stream)
 #define HASH_UNIT sizeof(reg_t)
 int LZ4_loadDict (LZ4_stream_t* LZ4_dict, const char* dictionary, int dictSize)
 {
-    LZ4_stream_t_internal* const dict = &LZ4_dict->internal_donotuse;
+    LZ4_stream_t_internal* dict = &LZ4_dict->internal_donotuse;
    const tableType_t tableType = byU32;
    const BYTE* p = (const BYTE*)dictionary;
    const BYTE* const dictEnd = p + dictSize;
-    U32 idx32;
+    const BYTE* base;

    DEBUGLOG(4, "LZ4_loadDict (%i bytes from %p into %p)", dictSize, dictionary, LZ4_dict);

@@ -1578,15 +1567,14 @@ int LZ4_loadDict (LZ4_stream_t* LZ4_dict, const char* dictionary, int dictSize)
    }

    if ((dictEnd - p) > 64 KB) p = dictEnd - 64 KB;
+    base = dictEnd - dict->currentOffset;
    dict->dictionary = p;
    dict->dictSize = (U32)(dictEnd - p);
    dict->tableType = (U32)tableType;
-    idx32 = dict->currentOffset - dict->dictSize;

    while (p <= dictEnd-HASH_UNIT) {
-        U32 const h = LZ4_hashPosition(p, tableType);
-        LZ4_putIndexOnHash(idx32, h, dict->hashTable, tableType);
-        p+=3; idx32+=3;
+        LZ4_putPosition(p, dict->hashTable, tableType, base);
+        p+=3;
    }

    return (int)dict->dictSize;
@@ -1723,7 +1711,7 @@ int LZ4_compress_fast_continue (LZ4_stream_t* LZ4_stream,
 /* Hidden debug function, to force-test external dictionary mode */
 int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source, char* dest, int srcSize)
 {
-    LZ4_stream_t_internal* const streamPtr = &LZ4_dict->internal_donotuse;
+    LZ4_stream_t_internal* streamPtr = &LZ4_dict->internal_donotuse;
    int result;

    LZ4_renormDictT(streamPtr, srcSize);
@@ -1786,7 +1774,7 @@ typedef enum { decode_full_block = 0, partial_decode = 1 } earlyEnd_directive;
 * does not know end of input
 * presumes input is well formed
 * note : will consume at least one byte */
-static size_t read_long_length_no_check(const BYTE** pp)
+size_t read_long_length_no_check(const BYTE** pp)
 {
    size_t b, l = 0;
    do { b = **pp; (*pp)++; l += b; } while (b==255);
@@ -2005,7 +1993,6 @@ LZ4_decompress_generic(
        }

        /* Fast loop : decode sequences as long as output < oend-FASTLOOP_SAFE_DISTANCE */
-        DEBUGLOG(6, "using fast decode loop");
        while (1) {
            /* Main fastloop assertion: We can always wildcopy FASTLOOP_SAFE_DISTANCE */
            assert(oend - op >= FASTLOOP_SAFE_DISTANCE);
@@ -2016,10 +2003,7 @@ LZ4_decompress_generic(
            /* decode literal length */
            if (length == RUN_MASK) {
                size_t const addl = read_variable_length(&ip, iend-RUN_MASK, 1);
-                if (addl == rvl_error) {
-                    DEBUGLOG(6, "error reading long literal length");
-                    goto _output_error;
-                }
+                if (addl == rvl_error) { goto _output_error; }
                length += addl;
                if (unlikely((uptrval)(op)+length<(uptrval)(op))) { goto _output_error; } /* overflow detection */
                if (unlikely((uptrval)(ip)+length<(uptrval)(ip))) { goto _output_error; } /* overflow detection */
@@ -2042,7 +2026,6 @@ LZ4_decompress_generic(

            /* get offset */
            offset = LZ4_readLE16(ip); ip+=2;
-            DEBUGLOG(6, " offset = %zu", offset);
            match = op - offset;
            assert(match <= op);  /* overflow check */

@@ -2051,17 +2034,11 @@ LZ4_decompress_generic(

            if (length == ML_MASK) {
                size_t const addl = read_variable_length(&ip, iend - LASTLITERALS + 1, 0);
-                if (addl == rvl_error) {
-                    DEBUGLOG(6, "error reading long match length");
-                    goto _output_error;
-                }
+                if (addl == rvl_error) { goto _output_error; }
                length += addl;
                length += MINMATCH;
                if (unlikely((uptrval)(op)+length<(uptrval)op)) { goto _output_error; } /* overflow detection */
-                if ((checkOffset) && (unlikely(match + dictSize < lowPrefix))) {
-                    DEBUGLOG(6, "Error : offset outside buffers");
-                    goto _output_error;
-                }
+                if ((checkOffset) && (unlikely(match + dictSize < lowPrefix))) { goto _output_error; } /* Error : offset outside buffers */
                if (op + length >= oend - FASTLOOP_SAFE_DISTANCE) {
                    goto safe_match_copy;
                }
@@ -2085,10 +2062,7 @@ LZ4_decompress_generic(
                        continue;
            }   }   }

-            if ( checkOffset && (unlikely(match + dictSize < lowPrefix)) ) {
-                DEBUGLOG(6, "Error : pos=%zi, offset=%zi => outside buffers", op-lowPrefix, op-match);
-                goto _output_error;
-            }
+            if (checkOffset && (unlikely(match + dictSize < lowPrefix))) { goto _output_error; } /* Error : offset outside buffers */
            /* match starting within external dictionary */
            if ((dict==usingExtDict) && (match < lowPrefix)) {
                assert(dictEnd != NULL);
@@ -2097,8 +2071,7 @@ LZ4_decompress_generic(
                        DEBUGLOG(7, "partialDecoding: dictionary match, close to dstEnd");
                        length = MIN(length, (size_t)(oend-op));
                    } else {
-                        DEBUGLOG(6, "end-of-block condition violated")
-                        goto _output_error;
+                        goto _output_error;  /* end-of-block condition violated */
                }   }

                if (length <= (size_t)(lowPrefix-match)) {
@@ -2138,7 +2111,6 @@ LZ4_decompress_generic(
 #endif

        /* Main Loop : decode remaining sequences where output < FASTLOOP_SAFE_DISTANCE */
-        DEBUGLOG(6, "using safe decode loop");
        while (1) {
            assert(ip < iend);
            token = *ip++;
@@ -2446,7 +2418,6 @@ int LZ4_decompress_safe_forceExtDict(const char* source, char* dest,
                                     int compressedSize, int maxOutputSize,
                                     const void* dictStart, size_t dictSize)
 {
-    DEBUGLOG(5, "LZ4_decompress_safe_forceExtDict");
    return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
                                  decode_full_block, usingExtDict,
                                  (BYTE*)dest, (const BYTE*)dictStart, dictSize);
@@ -189,9 +189,8 @@ LZ4LIB_API const char* LZ4_versionString (void);   /**< library version string;
 LZ4LIB_API int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity);

 /*! LZ4_decompress_safe() :
- * @compressedSize : is the exact complete size of the compressed block.
- * @dstCapacity : is the size of destination buffer (which must be already allocated),
- *                is an upper bound of decompressed size.
+ *  compressedSize : is the exact complete size of the compressed block.
+ *  dstCapacity : is the size of destination buffer (which must be already allocated), presumed an upper bound of decompressed size.
 * @return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
 *           If destination buffer is not large enough, decoding will stop and output an error code (negative value).
 *           If the source stream is detected malformed, the function will stop decoding and return a negative result.
@@ -444,24 +443,11 @@ LZ4LIB_API int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const
 LZ4LIB_API int LZ4_decoderRingBufferSize(int maxBlockSize);
 #define LZ4_DECODER_RING_BUFFER_SIZE(maxBlockSize) (65536 + 14 + (maxBlockSize))  /* for static allocation; maxBlockSize presumed valid */

-/*! LZ4_decompress_safe_continue() :
- *  This decoding function allows decompression of consecutive blocks in "streaming" mode.
- *  The difference with the usual independent blocks is that
- *  new blocks are allowed to find references into former blocks.
- *  A block is an unsplittable entity, and must be presented entirely to the decompression function.
- *  LZ4_decompress_safe_continue() only accepts one block at a time.
- *  It's modeled after `LZ4_decompress_safe()` and behaves similarly.
- *
- * @LZ4_streamDecode : decompression state, tracking the position in memory of past data
- * @compressedSize : exact complete size of one compressed block.
- * @dstCapacity : size of destination buffer (which must be already allocated),
- *                must be an upper bound of decompressed size.
- * @return : number of bytes decompressed into destination buffer (necessarily <= dstCapacity)
- *           If destination buffer is not large enough, decoding will stop and output an error code (negative value).
- *           If the source stream is detected malformed, the function will stop decoding and return a negative result.
- *
- *  The last 64KB of previously decoded data *must* remain available and unmodified
- *  at the memory position where they were previously decoded.
+/*! LZ4_decompress_*_continue() :
+ *  These decoding functions allow decompression of consecutive blocks in "streaming" mode.
+ *  A block is an unsplittable entity, it must be presented entirely to a decompression function.
+ *  Decompression functions only accepts one block at a time.
+ *  The last 64KB of previously decoded data *must* remain available and unmodified at the memory position where they were decoded.
 *  If less than 64KB of data has been decoded, all the data must be present.
 *
 *  Special : if decompression side sets a ring buffer, it must respect one of the following conditions :
@@ -488,10 +474,10 @@ LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode,
                        int srcSize, int dstCapacity);


-/*! LZ4_decompress_safe_usingDict() :
- *  Works the same as
- *  a combination of LZ4_setStreamDecode() followed by LZ4_decompress_safe_continue()
- *  However, it's stateless: it doesn't need any LZ4_streamDecode_t state.
+/*! LZ4_decompress_*_usingDict() :
+ *  These decoding functions work the same as
+ *  a combination of LZ4_setStreamDecode() followed by LZ4_decompress_*_continue()
+ *  They are stand-alone, and don't need an LZ4_streamDecode_t structure.
 *  Dictionary is presumed stable : it must remain accessible and unmodified during decompression.
 *  Performance tip : Decompression speed can be substantially increased
 *                    when dst == dictStart + dictSize.
@@ -501,12 +487,6 @@ LZ4_decompress_safe_usingDict(const char* src, char* dst,
                              int srcSize, int dstCapacity,
                              const char* dictStart, int dictSize);

-/*! LZ4_decompress_safe_partial_usingDict() :
- *  Behaves the same as LZ4_decompress_safe_partial()
- *  with the added ability to specify a memory segment for past data.
- *  Performance tip : Decompression speed can be substantially increased
- *                    when dst == dictStart + dictSize.
- */
 LZ4LIB_API int
 LZ4_decompress_safe_partial_usingDict(const char* src, char* dst,
                                      int compressedSize,
@@ -247,9 +247,9 @@ LZ4F_errorCode_t LZ4F_writeOpen(LZ4_writeFile_t** lz4fWrite, FILE* fp, const LZ4
  return LZ4F_OK_NoError;
 }

-size_t LZ4F_write(LZ4_writeFile_t* lz4fWrite, const void* buf, size_t size)
+size_t LZ4F_write(LZ4_writeFile_t* lz4fWrite, void* buf, size_t size)
 {
-  const LZ4_byte* p = (const LZ4_byte*)buf;
+  LZ4_byte* p = (LZ4_byte*)buf;
  size_t remain = size;
  size_t chunk;
  size_t ret;
@@ -78,7 +78,7 @@ LZ4FLIB_STATIC_API LZ4F_errorCode_t LZ4F_writeOpen(LZ4_writeFile_t** lz4fWrite,
 * `buf` write data buffer.
 * `size` write data buffer size.
 */
-LZ4FLIB_STATIC_API size_t LZ4F_write(LZ4_writeFile_t* lz4fWrite, const void* buf, size_t size);
+LZ4FLIB_STATIC_API size_t LZ4F_write(LZ4_writeFile_t* lz4fWrite, void* buf, size_t size);

 /*! LZ4F_writeClose() :
 * Close lz4file handle.
@@ -54,8 +54,8 @@
 **************************************/
 /*
 * LZ4F_HEAPMODE :
- * Control how LZ4F_compressFrame allocates the Compression State,
- * either on stack (0:default, fastest), or in memory heap (1:requires malloc()).
+ * Select how default compression functions will allocate memory for their hash table,
+ * in memory stack (0:default, fastest), or in memory heap (1:requires malloc()).
 */
 #ifndef LZ4F_HEAPMODE
 #  define LZ4F_HEAPMODE 0
@@ -264,7 +264,7 @@ typedef struct LZ4F_cctx_s
    LZ4F_CustomMem cmem;
    LZ4F_preferences_t prefs;
    U32    version;
-    U32    cStage;     /* 0 : compression uninitialized ; 1 : initialized, can compress */
+    U32    cStage;
    const LZ4F_CDict* cdict;
    size_t maxBlockSize;
    size_t maxBufferSize;
@@ -732,7 +732,7 @@ size_t LZ4F_compressBegin_usingCDict(LZ4F_cctx* cctxPtr,
        if (cctxPtr->maxBufferSize < requiredBuffSize) {
            cctxPtr->maxBufferSize = 0;
            LZ4F_free(cctxPtr->tmpBuff, cctxPtr->cmem);
-            cctxPtr->tmpBuff = (BYTE*)LZ4F_malloc(requiredBuffSize, cctxPtr->cmem);
+            cctxPtr->tmpBuff = (BYTE*)LZ4F_calloc(requiredBuffSize, cctxPtr->cmem);
            RETURN_ERROR_IF(cctxPtr->tmpBuff == NULL, allocation_failed);
            cctxPtr->maxBufferSize = requiredBuffSize;
    }   }
@@ -1085,6 +1085,7 @@ size_t LZ4F_uncompressedUpdate(LZ4F_cctx* cctxPtr,
                               void* dstBuffer, size_t dstCapacity,
                         const void* srcBuffer, size_t srcSize,
                         const LZ4F_compressOptions_t* compressOptionsPtr) {
+    RETURN_ERROR_IF(cctxPtr->prefs.frameInfo.blockMode != LZ4F_blockIndependent, blockMode_invalid);
    return LZ4F_compressUpdateImpl(cctxPtr,
                                   dstBuffer, dstCapacity,
                                   srcBuffer, srcSize,
@@ -1175,6 +1176,7 @@ size_t LZ4F_compressEnd(LZ4F_cctx* cctxPtr,
    }

    cctxPtr->cStage = 0;   /* state is now re-usable (with identical preferences) */
+    cctxPtr->maxBufferSize = 0;  /* reuse HC context */

    if (cctxPtr->prefs.frameInfo.contentSize) {
        if (cctxPtr->prefs.frameInfo.contentSize != cctxPtr->totalInSize)
@@ -1720,10 +1722,10 @@ size_t LZ4F_decompress(LZ4F_dctx* dctx,
                    /* history management (linked blocks only)*/
                    if (dctx->frameInfo.blockMode == LZ4F_blockLinked) {
                        LZ4F_updateDict(dctx, dstPtr, sizeToCopy, dstStart, 0);
-                    }
-                    srcPtr += sizeToCopy;
-                    dstPtr += sizeToCopy;
-                }
+                }   }
+
+                srcPtr += sizeToCopy;
+                dstPtr += sizeToCopy;
                if (sizeToCopy == dctx->tmpInTarget) {   /* all done */
                    if (dctx->frameInfo.blockChecksumFlag) {
                        dctx->tmpInSize = 0;
@@ -215,19 +215,9 @@ LZ4FLIB_API int LZ4F_compressionLevel_max(void);   /* v1.8.0+ */
 LZ4FLIB_API size_t LZ4F_compressFrameBound(size_t srcSize, const LZ4F_preferences_t* preferencesPtr);

 /*! LZ4F_compressFrame() :
- *  Compress srcBuffer content into an LZ4-compressed frame.
- *  It's a one shot operation, all input content is consumed, and all output is generated.
- *
- *  Note : it's a stateless operation (no LZ4F_cctx state needed).
- *  In order to reduce load on the allocator, LZ4F_compressFrame(), by default,
- *  uses the stack to allocate space for the compression state and some table.
- *  If this usage of the stack is too much for your application,
- *  consider compiling `lz4frame.c` with compile-time macro LZ4F_HEAPMODE set to 1 instead.
- *  All state allocations will use the Heap.
- *  It also means each invocation of LZ4F_compressFrame() will trigger several internal alloc/free invocations.
- *
- * @dstCapacity MUST be >= LZ4F_compressFrameBound(srcSize, preferencesPtr).
- * @preferencesPtr is optional : one can provide NULL, in which case all preferences are set to default.
+ *  Compress an entire srcBuffer into a valid LZ4 frame.
+ *  dstCapacity MUST be >= LZ4F_compressFrameBound(srcSize, preferencesPtr).
+ *  The LZ4F_preferences_t structure is optional : you can provide NULL as argument. All preferences will be set to default.
 * @return : number of bytes written into dstBuffer.
 *           or an error code if it fails (can be tested using LZ4F_isError())
 */
@@ -288,7 +278,7 @@ LZ4FLIB_API LZ4F_errorCode_t LZ4F_freeCompressionContext(LZ4F_cctx* cctx);
 /*! LZ4F_compressBegin() :
 *  will write the frame header into dstBuffer.
 *  dstCapacity must be >= LZ4F_HEADER_SIZE_MAX bytes.
- * `prefsPtr` is optional : NULL can be provided to set all preferences to default.
+ * `prefsPtr` is optional : you can provide NULL as argument, all preferences will then be set to default.
 * @return : number of bytes written into dstBuffer for the header
 *           or an error code (which can be tested using LZ4F_isError())
 */
@@ -473,11 +463,6 @@ LZ4F_getFrameInfo(LZ4F_dctx* dctx,
 * `dstBuffer` can freely change between each consecutive function invocation.
 * `dstBuffer` content will be overwritten.
 *
- *  Note: if `LZ4F_getFrameInfo()` is called before `LZ4F_decompress()`, srcBuffer must be updated to reflect
- *  the number of bytes consumed after reading the frame header. Failure to update srcBuffer before calling
- *  `LZ4F_decompress()` will cause decompression failure or, even worse, successful but incorrect decompression.
- *  See the `LZ4F_getFrameInfo()` docs for details.
- *
 * @return : an hint of how many `srcSize` bytes LZ4F_decompress() expects for next call.
 *  Schematically, it's the size of the current (or remaining) compressed block + header of next block.
 *  Respecting the hint provides some small speed benefit, because it skips intermediate buffers.
@@ -675,7 +660,7 @@ LZ4F_decompress_usingDict(LZ4F_dctx* dctxPtr,
                    const LZ4F_decompressOptions_t* decompressOptionsPtr);


-/*! Custom memory allocation : v1.9.4+
+/*! Custom memory allocation :
 *  These prototypes make it possible to pass custom allocation/free functions.
 *  LZ4F_customMem is provided at state creation time, using LZ4F_create*_advanced() listed below.
 *  All allocation/free operations will be completed using these custom variants instead of regular <stdlib.h> ones.
@@ -39,10 +39,9 @@
 ***************************************/

 /*! HEAPMODE :
- *  Select how stateless HC compression functions like `LZ4_compress_HC()`
- *  allocate memory for their workspace:
- *  in stack (0:fastest), or in heap (1:default, requires malloc()).
- *  Since workspace is rather large, heap mode is recommended.
+ *  Select how default compression function will allocate workplace memory,
+ *  in stack (0:fastest), or in heap (1:requires malloc()).
+ *  Since workplace is rather large, heap mode is recommended.
 **/
 #ifndef LZ4HC_HEAPMODE
 #  define LZ4HC_HEAPMODE 1
@@ -86,7 +85,6 @@ typedef enum { noDictCtx, usingDictCtxHc } dictCtx_directive;
 /* Make fields passed to, and updated by LZ4HC_encodeSequence explicit */
 #define UPDATABLE(ip, op, anchor) &ip, &op, &anchor

-#define LZ4HC_HASHSIZE 4
 static U32 LZ4HC_hashPtr(const void* ptr) { return HASH_FUNCTION(LZ4_read32(ptr)); }


@@ -103,7 +101,6 @@ static void LZ4HC_init_internal (LZ4HC_CCtx_internal* hc4, const BYTE* start)
 {
    size_t const bufferSize = (size_t)(hc4->end - hc4->prefixStart);
    size_t newStartingOffset = bufferSize + hc4->dictLimit;
-    DEBUGLOG(5, "LZ4HC_init_internal");
    assert(newStartingOffset >= bufferSize);  /* check overflow */
    if (newStartingOffset > 1 GB) {
        LZ4HC_clearTables(hc4);
@@ -239,17 +236,13 @@ static int LZ4HC_protectDictEnd(U32 const dictLimit, U32 const matchIndex)
 typedef enum { rep_untested, rep_not, rep_confirmed } repeat_state_e;
 typedef enum { favorCompressionRatio=0, favorDecompressionSpeed } HCfavor_e;

-typedef struct {
-    int off;
-    int len;
-} LZ4HC_match_t;
-
-LZ4_FORCE_INLINE LZ4HC_match_t
+LZ4_FORCE_INLINE int
 LZ4HC_InsertAndGetWiderMatch (
        LZ4HC_CCtx_internal* const hc4,
        const BYTE* const ip,
        const BYTE* const iLowLimit, const BYTE* const iHighLimit,
        int longest,
+        const BYTE** matchpos,
        const BYTE** startpos,
        const int maxNbAttempts,
        const int patternAnalysis, const int chainSwap,
@@ -257,7 +250,7 @@ LZ4HC_InsertAndGetWiderMatch (
        const HCfavor_e favorDecSpeed)
 {
    U16* const chainTable = hc4->chainTable;
-    U32* const hashTable = hc4->hashTable;
+    U32* const HashTable = hc4->hashTable;
    const LZ4HC_CCtx_internal * const dictCtx = hc4->dictCtx;
    const BYTE* const prefixPtr = hc4->prefixStart;
    const U32 prefixIdx = hc4->dictLimit;
@@ -274,26 +267,22 @@ LZ4HC_InsertAndGetWiderMatch (
    U32 matchIndex;
    repeat_state_e repeat = rep_untested;
    size_t srcPatternLength = 0;
-    int offset = 0;

    DEBUGLOG(7, "LZ4HC_InsertAndGetWiderMatch");
-    assert(startpos != NULL);
-    *startpos = ip;  /* in case there is no solution */
    /* First Match */
-    LZ4HC_Insert(hc4, ip);  /* insert all prior positions up to ip (excluded) */
-    matchIndex = hashTable[LZ4HC_hashPtr(ip)];
-    DEBUGLOG(7, "First candidate match for pos %u found at index %u / %u (lowestMatchIndex)",
-                ipIndex, matchIndex, lowestMatchIndex);
+    LZ4HC_Insert(hc4, ip);
+    matchIndex = HashTable[LZ4HC_hashPtr(ip)];
+    DEBUGLOG(7, "First match at index %u / %u (lowestMatchIndex)",
+                matchIndex, lowestMatchIndex);

    while ((matchIndex>=lowestMatchIndex) && (nbAttempts>0)) {
        int matchLength=0;
        nbAttempts--;
        assert(matchIndex < ipIndex);
        if (favorDecSpeed && (ipIndex - matchIndex < 8)) {
-            /* do nothing:
-             * favorDecSpeed intentionally skips matches with offset < 8 */
+            /* do nothing */
        } else if (matchIndex >= prefixIdx) {   /* within current Prefix */
-            const BYTE* const matchPtr = prefixPtr + (matchIndex - prefixIdx);
+            const BYTE* const matchPtr = prefixPtr + matchIndex - prefixIdx;
            assert(matchPtr < ip);
            assert(longest >= 1);
            if (LZ4_read16(iLowLimit + longest - 1) == LZ4_read16(matchPtr - lookBackLength + longest - 1)) {
@@ -303,11 +292,10 @@ LZ4HC_InsertAndGetWiderMatch (
                    matchLength -= back;
                    if (matchLength > longest) {
                        longest = matchLength;
-                        offset = (int)(ipIndex - matchIndex);
+                        *matchpos = matchPtr + back;
                        *startpos = ip + back;
-                        DEBUGLOG(7, "Found match of len=%i within prefix, offset=%i, back=%i", longest, offset, -back);
            }   }   }
-        } else {   /* lowestMatchIndex <= matchIndex < dictLimit : within Ext Dict */
+        } else {   /* lowestMatchIndex <= matchIndex < dictLimit */
            const BYTE* const matchPtr = dictStart + (matchIndex - dictIdx);
            assert(matchIndex >= dictIdx);
            if ( likely(matchIndex <= prefixIdx - 4)
@@ -322,9 +310,8 @@ LZ4HC_InsertAndGetWiderMatch (
                matchLength -= back;
                if (matchLength > longest) {
                    longest = matchLength;
-                    offset = (int)(ipIndex - matchIndex);
+                    *matchpos = prefixPtr - prefixIdx + matchIndex + back;   /* virtual pos, relative to ip, to retrieve offset */
                    *startpos = ip + back;
-                    DEBUGLOG(7, "Found match of len=%i within dict, offset=%i, back=%i", longest, offset, -back);
        }   }   }

        if (chainSwap && matchLength==longest) {   /* better match => select a better chain */
@@ -357,7 +344,6 @@ LZ4HC_InsertAndGetWiderMatch (
                if (repeat == rep_untested) {
                    if ( ((pattern & 0xFFFF) == (pattern >> 16))
                      &  ((pattern & 0xFF)   == (pattern >> 24)) ) {
-                        DEBUGLOG(7, "Repeat pattern detected, char %02X", pattern >> 24);
                        repeat = rep_confirmed;
                        srcPatternLength = LZ4HC_countPattern(ip+sizeof(pattern), iHighLimit, pattern) + sizeof(pattern);
                    } else {
@@ -366,7 +352,7 @@ LZ4HC_InsertAndGetWiderMatch (
                if ( (repeat == rep_confirmed) && (matchCandidateIdx >= lowestMatchIndex)
                  && LZ4HC_protectDictEnd(prefixIdx, matchCandidateIdx) ) {
                    const int extDict = matchCandidateIdx < prefixIdx;
-                    const BYTE* const matchPtr = extDict ? dictStart + (matchCandidateIdx - dictIdx) : prefixPtr + (matchCandidateIdx - prefixIdx);
+                    const BYTE* const matchPtr = (extDict ? dictStart - dictIdx : prefixPtr - prefixIdx) + matchCandidateIdx;
                    if (LZ4_read32(matchPtr) == pattern) {  /* good candidate */
                        const BYTE* const iLimit = extDict ? dictEnd : iHighLimit;
                        size_t forwardPatternLength = LZ4HC_countPattern(matchPtr+sizeof(pattern), iLimit, pattern) + sizeof(pattern);
@@ -412,9 +398,8 @@ LZ4HC_InsertAndGetWiderMatch (
                                            if ((size_t)(ip - prefixPtr) + prefixIdx - matchIndex > LZ4_DISTANCE_MAX) break;
                                            assert(maxML < 2 GB);
                                            longest = (int)maxML;
-                                            offset = (int)(ipIndex - matchIndex);
+                                            *matchpos = prefixPtr - prefixIdx + matchIndex;   /* virtual pos, relative to ip, to retrieve offset */
                                            *startpos = ip;
-                                            DEBUGLOG(7, "Found repeat pattern match of len=%i, offset=%i", longest, offset);
                                        }
                                        {   U32 const distToNextPattern = DELTANEXTU16(chainTable, matchIndex);
                                            if (distToNextPattern > matchIndex) break;  /* avoid overflow */
@@ -436,7 +421,6 @@ LZ4HC_InsertAndGetWiderMatch (
        U32 dictMatchIndex = dictCtx->hashTable[LZ4HC_hashPtr(ip)];
        assert(dictEndOffset <= 1 GB);
        matchIndex = dictMatchIndex + lowestMatchIndex - (U32)dictEndOffset;
-        if (dictMatchIndex>0) DEBUGLOG(7, "dictEndOffset = %zu, dictMatchIndex = %u => relative matchIndex = %i", dictEndOffset, dictMatchIndex, (int)dictMatchIndex - (int)dictEndOffset);
        while (ipIndex - matchIndex <= LZ4_DISTANCE_MAX && nbAttempts--) {
            const BYTE* const matchPtr = dictCtx->prefixStart - dictCtx->dictLimit + dictMatchIndex;

@@ -450,9 +434,8 @@ LZ4HC_InsertAndGetWiderMatch (
                mlt -= back;
                if (mlt > longest) {
                    longest = mlt;
-                    offset = (int)(ipIndex - matchIndex);
+                    *matchpos = prefixPtr - prefixIdx + matchIndex + back;
                    *startpos = ip + back;
-                    DEBUGLOG(7, "found match of length %i within extDictCtx", longest);
            }   }

            {   U32 const nextOffset = DELTANEXTU16(dictCtx->chainTable, dictMatchIndex);
@@ -460,27 +443,22 @@ LZ4HC_InsertAndGetWiderMatch (
                matchIndex -= nextOffset;
    }   }   }

-    {   LZ4HC_match_t md;
-        assert(longest >= 0);
-        md.len = longest;
-        md.off = offset;
-        return md;
-    }
+    return longest;
 }

-LZ4_FORCE_INLINE LZ4HC_match_t
+LZ4_FORCE_INLINE int
 LZ4HC_InsertAndFindBestMatch(LZ4HC_CCtx_internal* const hc4,   /* Index table will be updated */
                       const BYTE* const ip, const BYTE* const iLimit,
+                       const BYTE** matchpos,
                       const int maxNbAttempts,
                       const int patternAnalysis,
                       const dictCtx_directive dict)
 {
    const BYTE* uselessPtr = ip;
-    DEBUGLOG(7, "LZ4HC_InsertAndFindBestMatch");
    /* note : LZ4HC_InsertAndGetWiderMatch() is able to modify the starting position of a match (*startpos),
     * but this won't be the case here, as we define iLowLimit==ip,
     * so LZ4HC_InsertAndGetWiderMatch() won't be allowed to search past ip */
-    return LZ4HC_InsertAndGetWiderMatch(hc4, ip, ip, iLimit, MINMATCH-1, &uselessPtr, maxNbAttempts, patternAnalysis, 0 /*chainSwap*/, dict, favorCompressionRatio);
+    return LZ4HC_InsertAndGetWiderMatch(hc4, ip, ip, iLimit, MINMATCH-1, matchpos, &uselessPtr, maxNbAttempts, patternAnalysis, 0 /*chainSwap*/, dict, favorCompressionRatio);
 }

 /* LZ4HC_encodeSequence() :
@@ -491,7 +469,7 @@ LZ4_FORCE_INLINE int LZ4HC_encodeSequence (
    BYTE** _op,
    const BYTE** _anchor,
    int matchLength,
-    int offset,
+    const BYTE* const match,
    limitedOutput_directive limit,
    BYTE* oend)
 {
@@ -512,9 +490,9 @@ LZ4_FORCE_INLINE int LZ4HC_encodeSequence (
    U32 const cost = 1 + llAdd + ll + 2 + mlAdd;
    if (start==NULL) start = anchor;  /* only works for single segment */
    /* g_debuglog_enable = (pos >= 2228) & (pos <= 2262); */
-    DEBUGLOG(6, "pos:%7u -- literals:%4u, match:%4i, offset:%5i, cost:%4u + %5u",
+    DEBUGLOG(6, "pos:%7u -- literals:%4u, match:%4i, offset:%5u, cost:%4u + %5u",
                pos,
-                (U32)(ip - anchor), matchLength, offset,
+                (U32)(ip - anchor), matchLength, (U32)(ip-match),
                cost, totalCost);
    totalCost += cost;
 #endif
@@ -542,9 +520,8 @@ LZ4_FORCE_INLINE int LZ4HC_encodeSequence (
    op += length;

    /* Encode Offset */
-    assert(offset <= LZ4_DISTANCE_MAX );
-    assert(offset > 0);
-    LZ4_writeLE16(op, (U16)(offset)); op += 2;
+    assert( (ip - match) <= LZ4_DISTANCE_MAX );   /* note : consider providing offset as a value, rather than as a pointer difference */
+    LZ4_writeLE16(op, (U16)(ip - match)); op += 2;

    /* Encode MatchLength */
    assert(matchLength >= MINMATCH);
@@ -597,116 +574,127 @@ LZ4_FORCE_INLINE int LZ4HC_compress_hashChain (
    BYTE* op = (BYTE*) dest;
    BYTE* oend = op + maxOutputSize;

+    int   ml0, ml, ml2, ml3;
    const BYTE* start0;
+    const BYTE* ref0;
+    const BYTE* ref = NULL;
    const BYTE* start2 = NULL;
+    const BYTE* ref2 = NULL;
    const BYTE* start3 = NULL;
-    LZ4HC_match_t m0, m1, m2, m3;
-    const LZ4HC_match_t nomatch = {0, 0};
+    const BYTE* ref3 = NULL;

    /* init */
-    DEBUGLOG(5, "LZ4HC_compress_hashChain (dict?=>%i)", dict);
    *srcSizePtr = 0;
    if (limit == fillOutput) oend -= LASTLITERALS;                  /* Hack for support LZ4 format restriction */
    if (inputSize < LZ4_minLength) goto _last_literals;             /* Input too small, no compression (all literals) */

    /* Main Loop */
    while (ip <= mflimit) {
-        m1 = LZ4HC_InsertAndFindBestMatch(ctx, ip, matchlimit, maxNbAttempts, patternAnalysis, dict);
-        if (m1.len<MINMATCH) { ip++; continue; }
+        ml = LZ4HC_InsertAndFindBestMatch(ctx, ip, matchlimit, &ref, maxNbAttempts, patternAnalysis, dict);
+        if (ml<MINMATCH) { ip++; continue; }

        /* saved, in case we would skip too much */
-        start0 = ip; m0 = m1;
+        start0 = ip; ref0 = ref; ml0 = ml;

 _Search2:
-        DEBUGLOG(7, "_Search2 (currently found match of size %i)", m1.len);
-        if (ip+m1.len <= mflimit) {
-            m2 = LZ4HC_InsertAndGetWiderMatch(ctx,
-                            ip + m1.len - 2, ip + 0, matchlimit, m1.len, &start2,
+        if (ip+ml <= mflimit) {
+            ml2 = LZ4HC_InsertAndGetWiderMatch(ctx,
+                            ip + ml - 2, ip + 0, matchlimit, ml, &ref2, &start2,
                            maxNbAttempts, patternAnalysis, 0, dict, favorCompressionRatio);
        } else {
-            m2 = nomatch;  /* do not search further */
+            ml2 = ml;
        }

-        if (m2.len <= m1.len) { /* No better match => encode ML1 immediately */
+        if (ml2 == ml) { /* No better match => encode ML1 */
            optr = op;
-            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, limit, oend)) goto _dest_overflow;
+            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, limit, oend)) goto _dest_overflow;
            continue;
        }

        if (start0 < ip) {   /* first match was skipped at least once */
-            if (start2 < ip + m0.len) {  /* squeezing ML1 between ML0(original ML1) and ML2 */
-                ip = start0; m1 = m0;  /* restore initial Match1 */
+            if (start2 < ip + ml0) {  /* squeezing ML1 between ML0(original ML1) and ML2 */
+                ip = start0; ref = ref0; ml = ml0;  /* restore initial ML1 */
        }   }

        /* Here, start0==ip */
        if ((start2 - ip) < 3) {  /* First Match too small : removed */
+            ml = ml2;
            ip = start2;
-            m1 = m2;
+            ref =ref2;
            goto _Search2;
        }

 _Search3:
+        /* At this stage, we have :
+        *  ml2 > ml1, and
+        *  ip1+3 <= ip2 (usually < ip1+ml1) */
        if ((start2 - ip) < OPTIMAL_ML) {
            int correction;
-            int new_ml = m1.len;
+            int new_ml = ml;
            if (new_ml > OPTIMAL_ML) new_ml = OPTIMAL_ML;
-            if (ip+new_ml > start2 + m2.len - MINMATCH)
-                new_ml = (int)(start2 - ip) + m2.len - MINMATCH;
+            if (ip+new_ml > start2 + ml2 - MINMATCH) new_ml = (int)(start2 - ip) + ml2 - MINMATCH;
            correction = new_ml - (int)(start2 - ip);
            if (correction > 0) {
                start2 += correction;
-                m2.len -= correction;
+                ref2 += correction;
+                ml2 -= correction;
            }
        }
+        /* Now, we have start2 = ip+new_ml, with new_ml = min(ml, OPTIMAL_ML=18) */

-        if (start2 + m2.len <= mflimit) {
-            m3 = LZ4HC_InsertAndGetWiderMatch(ctx,
-                            start2 + m2.len - 3, start2, matchlimit, m2.len, &start3,
+        if (start2 + ml2 <= mflimit) {
+            ml3 = LZ4HC_InsertAndGetWiderMatch(ctx,
+                            start2 + ml2 - 3, start2, matchlimit, ml2, &ref3, &start3,
                            maxNbAttempts, patternAnalysis, 0, dict, favorCompressionRatio);
        } else {
-            m3 = nomatch;  /* do not search further */
+            ml3 = ml2;
        }

-        if (m3.len <= m2.len) {  /* No better match => encode ML1 and ML2 */
+        if (ml3 == ml2) {  /* No better match => encode ML1 and ML2 */
            /* ip & ref are known; Now for ml */
-            if (start2 < ip+m1.len) m1.len = (int)(start2 - ip);
+            if (start2 < ip+ml)  ml = (int)(start2 - ip);
            /* Now, encode 2 sequences */
            optr = op;
-            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, limit, oend))
-                goto _dest_overflow;
+            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, limit, oend)) goto _dest_overflow;
            ip = start2;
            optr = op;
-            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m2.len, m2.off, limit, oend)) {
-                m1 = m2;
+            if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml2, ref2, limit, oend)) {
+                ml  = ml2;
+                ref = ref2;
                goto _dest_overflow;
            }
            continue;
        }

-        if (start3 < ip+m1.len+3) {  /* Not enough space for match 2 : remove it */
-            if (start3 >= (ip+m1.len)) {  /* can write Seq1 immediately ==> Seq2 is removed, so Seq3 becomes Seq1 */
-                if (start2 < ip+m1.len) {
-                    int correction = (int)(ip+m1.len - start2);
+        if (start3 < ip+ml+3) {  /* Not enough space for match 2 : remove it */
+            if (start3 >= (ip+ml)) {  /* can write Seq1 immediately ==> Seq2 is removed, so Seq3 becomes Seq1 */
+                if (start2 < ip+ml) {
+                    int correction = (int)(ip+ml - start2);
                    start2 += correction;
-                    m2.len -= correction;
-                    if (m2.len < MINMATCH) {
+                    ref2 += correction;
+                    ml2 -= correction;
+                    if (ml2 < MINMATCH) {
                        start2 = start3;
-                        m2 = m3;
+                        ref2 = ref3;
+                        ml2 = ml3;
                    }
                }

                optr = op;
-                if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, limit, oend)) goto _dest_overflow;
+                if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, limit, oend)) goto _dest_overflow;
                ip  = start3;
-                m1 = m3;
+                ref = ref3;
+                ml  = ml3;

                start0 = start2;
-                m0 = m2;
+                ref0 = ref2;
+                ml0 = ml2;
                goto _Search2;
            }

            start2 = start3;
-            m2 = m3;
+            ref2 = ref3;
+            ml2 = ml3;
            goto _Search3;
        }

@@ -715,29 +703,29 @@ _Search3:
        * let's write the first one ML1.
        * ip & ref are known; Now decide ml.
        */
-        if (start2 < ip+m1.len) {
+        if (start2 < ip+ml) {
            if ((start2 - ip) < OPTIMAL_ML) {
                int correction;
-                if (m1.len > OPTIMAL_ML) m1.len = OPTIMAL_ML;
-                if (ip + m1.len > start2 + m2.len - MINMATCH)
-                    m1.len = (int)(start2 - ip) + m2.len - MINMATCH;
-                correction = m1.len - (int)(start2 - ip);
+                if (ml > OPTIMAL_ML) ml = OPTIMAL_ML;
+                if (ip + ml > start2 + ml2 - MINMATCH) ml = (int)(start2 - ip) + ml2 - MINMATCH;
+                correction = ml - (int)(start2 - ip);
                if (correction > 0) {
                    start2 += correction;
-                    m2.len -= correction;
+                    ref2 += correction;
+                    ml2 -= correction;
                }
            } else {
-                m1.len = (int)(start2 - ip);
+                ml = (int)(start2 - ip);
            }
        }
        optr = op;
-        if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, limit, oend)) goto _dest_overflow;
+        if (LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, limit, oend)) goto _dest_overflow;

        /* ML2 becomes ML1 */
-        ip = start2; m1 = m2;
+        ip = start2; ref = ref2; ml = ml2;

        /* ML3 becomes ML2 */
-        start2 = start3; m2 = m3;
+        start2 = start3; ref2 = ref3; ml2 = ml3;

        /* let's find a new ML3 */
        goto _Search3;
@@ -788,10 +776,10 @@ _dest_overflow:
            /* ll validated; now adjust match length */
            size_t const bytesLeftForMl = (size_t)(maxLitPos - (op+ll_totalCost));
            size_t const maxMlSize = MINMATCH + (ML_MASK-1) + (bytesLeftForMl * 255);
-            assert(maxMlSize < INT_MAX); assert(m1.len >= 0);
-            if ((size_t)m1.len > maxMlSize) m1.len = (int)maxMlSize;
-            if ((oend + LASTLITERALS) - (op + ll_totalCost + 2) - 1 + m1.len >= MFLIMIT) {
-                LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), m1.len, m1.off, notLimited, oend);
+            assert(maxMlSize < INT_MAX); assert(ml >= 0);
+            if ((size_t)ml > maxMlSize) ml = (int)maxMlSize;
+            if ((oend + LASTLITERALS) - (op + ll_totalCost + 2) - 1 + ml >= MFLIMIT) {
+                LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ref, notLimited, oend);
        }   }
        goto _last_literals;
    }
@@ -809,17 +797,16 @@ static int LZ4HC_compress_optimal( LZ4HC_CCtx_internal* ctx,
    const HCfavor_e favorDecSpeed);


-LZ4_FORCE_INLINE int
-LZ4HC_compress_generic_internal (
-            LZ4HC_CCtx_internal* const ctx,
-            const char* const src,
-            char* const dst,
-            int* const srcSizePtr,
-            int const dstCapacity,
-            int cLevel,
-            const limitedOutput_directive limit,
-            const dictCtx_directive dict
-            )
+LZ4_FORCE_INLINE int LZ4HC_compress_generic_internal (
+    LZ4HC_CCtx_internal* const ctx,
+    const char* const src,
+    char* const dst,
+    int* const srcSizePtr,
+    int const dstCapacity,
+    int cLevel,
+    const limitedOutput_directive limit,
+    const dictCtx_directive dict
+    )
 {
    typedef enum { lz4hc, lz4opt } lz4hc_strat_e;
    typedef struct {
@@ -843,8 +830,8 @@ LZ4HC_compress_generic_internal (
        { lz4opt,16384,LZ4_OPT_NUM },  /* 12==LZ4HC_CLEVEL_MAX */
    };

-    DEBUGLOG(5, "LZ4HC_compress_generic_internal(src=%p, srcSize=%d)",
-                src, *srcSizePtr);
+    DEBUGLOG(4, "LZ4HC_compress_generic(ctx=%p, src=%p, srcSize=%d, limit=%d)",
+                ctx, src, *srcSizePtr, limit);

    if (limit == fillOutput && dstCapacity < 1) return 0;   /* Impossible to store anything */
    if ((U32)*srcSizePtr > (U32)LZ4_MAX_INPUT_SIZE) return 0;    /* Unsupported input size (too large or negative) */
@@ -978,7 +965,6 @@ int LZ4_compress_HC(const char* src, char* dst, int srcSize, int dstCapacity, in
    LZ4_streamHC_t state;
    LZ4_streamHC_t* const statePtr = &state;
 #endif
-    DEBUGLOG(5, "LZ4_compress_HC")
    cSize = LZ4_compress_HC_extStateHC(statePtr, src, dst, srcSize, dstCapacity, compressionLevel);
 #if defined(LZ4HC_HEAPMODE) && LZ4HC_HEAPMODE==1
    FREEMEM(statePtr);
@@ -1046,16 +1032,18 @@ void LZ4_resetStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr, int compressionLevel)

 void LZ4_resetStreamHC_fast (LZ4_streamHC_t* LZ4_streamHCPtr, int compressionLevel)
 {
-    LZ4HC_CCtx_internal* const s = &LZ4_streamHCPtr->internal_donotuse;
-    DEBUGLOG(5, "LZ4_resetStreamHC_fast(%p, %d)", LZ4_streamHCPtr, compressionLevel);
-    if (s->dirty) {
+    DEBUGLOG(4, "LZ4_resetStreamHC_fast(%p, %d)", LZ4_streamHCPtr, compressionLevel);
+    if (LZ4_streamHCPtr->internal_donotuse.dirty) {
        LZ4_initStreamHC(LZ4_streamHCPtr, sizeof(*LZ4_streamHCPtr));
    } else {
-        assert(s->end >= s->prefixStart);
-        s->dictLimit += (U32)(s->end - s->prefixStart);
-        s->prefixStart = NULL;
-        s->end = NULL;
-        s->dictCtx = NULL;
+        /* preserve end - prefixStart : can trigger clearTable's threshold */
+        if (LZ4_streamHCPtr->internal_donotuse.end != NULL) {
+            LZ4_streamHCPtr->internal_donotuse.end -= (uptrval)LZ4_streamHCPtr->internal_donotuse.prefixStart;
+        } else {
+            assert(LZ4_streamHCPtr->internal_donotuse.prefixStart == NULL);
+        }
+        LZ4_streamHCPtr->internal_donotuse.prefixStart = NULL;
+        LZ4_streamHCPtr->internal_donotuse.dictCtx = NULL;
    }
    LZ4_setCompressionLevel(LZ4_streamHCPtr, compressionLevel);
 }
@@ -1092,7 +1080,7 @@ int LZ4_loadDictHC (LZ4_streamHC_t* LZ4_streamHCPtr,
    }
    LZ4HC_init_internal (ctxPtr, (const BYTE*)dictionary);
    ctxPtr->end = (const BYTE*)dictionary + dictSize;
-    if (dictSize >= LZ4HC_HASHSIZE) LZ4HC_Insert (ctxPtr, ctxPtr->end-3);
+    if (dictSize >= 4) LZ4HC_Insert (ctxPtr, ctxPtr->end-3);
    return dictSize;
 }

@@ -1152,8 +1140,7 @@ LZ4_compressHC_continue_generic (LZ4_streamHC_t* LZ4_streamHCPtr,
            if (sourceEnd > dictEnd) sourceEnd = dictEnd;
            ctxPtr->lowLimit += (U32)(sourceEnd - ctxPtr->dictStart);
            ctxPtr->dictStart += (U32)(sourceEnd - ctxPtr->dictStart);
-            /* invalidate dictionary is it's too small */
-            if (ctxPtr->dictLimit - ctxPtr->lowLimit < LZ4HC_HASHSIZE) {
+            if (ctxPtr->dictLimit - ctxPtr->lowLimit < 4) {
                ctxPtr->lowLimit = ctxPtr->dictLimit;
                ctxPtr->dictStart = ctxPtr->prefixStart;
    }   }   }
@@ -1163,7 +1150,6 @@ LZ4_compressHC_continue_generic (LZ4_streamHC_t* LZ4_streamHCPtr,

 int LZ4_compress_HC_continue (LZ4_streamHC_t* LZ4_streamHCPtr, const char* src, char* dst, int srcSize, int dstCapacity)
 {
-    DEBUGLOG(5, "LZ4_compress_HC_continue");
    if (dstCapacity < LZ4_compressBound(srcSize))
        return LZ4_compressHC_continue_generic (LZ4_streamHCPtr, src, dst, &srcSize, dstCapacity, limitedOutput);
    else
@@ -1193,10 +1179,10 @@ int LZ4_saveDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, char* safeBuffer, int dictS
    if (dictSize > prefixSize) dictSize = prefixSize;
    if (safeBuffer == NULL) assert(dictSize == 0);
    if (dictSize > 0)
-        LZ4_memmove(safeBuffer, streamPtr->end - dictSize, (size_t)dictSize);
+        LZ4_memmove(safeBuffer, streamPtr->end - dictSize, dictSize);
    {   U32 const endIndex = (U32)(streamPtr->end - streamPtr->prefixStart) + streamPtr->dictLimit;
-        streamPtr->end = (safeBuffer == NULL) ? NULL : (const BYTE*)safeBuffer + dictSize;
-        streamPtr->prefixStart = (const BYTE*)safeBuffer;
+        streamPtr->end = (const BYTE*)safeBuffer + dictSize;
+        streamPtr->prefixStart = streamPtr->end - dictSize;
        streamPtr->dictLimit = endIndex - (U32)dictSize;
        streamPtr->lowLimit = endIndex - (U32)dictSize;
        streamPtr->dictStart = streamPtr->prefixStart;
@@ -1268,10 +1254,10 @@ int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char* src, c

 char* LZ4_slideInputBufferHC(void* LZ4HC_Data)
 {
-    LZ4HC_CCtx_internal* const s = &((LZ4_streamHC_t*)LZ4HC_Data)->internal_donotuse;
-    const BYTE* const bufferStart = s->prefixStart - s->dictLimit + s->lowLimit;
-    LZ4_resetStreamHC_fast((LZ4_streamHC_t*)LZ4HC_Data, s->compressionLevel);
-    /* ugly conversion trick, required to evade (const char*) -> (char*) cast-qual warning :( */
+    LZ4_streamHC_t* const ctx = (LZ4_streamHC_t*)LZ4HC_Data;
+    const BYTE* bufferStart = ctx->internal_donotuse.prefixStart - ctx->internal_donotuse.dictLimit + ctx->internal_donotuse.lowLimit;
+    LZ4_resetStreamHC_fast(ctx, ctx->internal_donotuse.compressionLevel);
+    /* avoid const char * -> char * conversion warning :( */
    return (char*)(uptrval)bufferStart;
 }

@@ -1313,6 +1299,10 @@ LZ4_FORCE_INLINE int LZ4HC_sequencePrice(int litlen, int mlen)
 }


+typedef struct {
+    int off;
+    int len;
+} LZ4HC_match_t;

 LZ4_FORCE_INLINE LZ4HC_match_t
 LZ4HC_FindLongerMatch(LZ4HC_CCtx_internal* const ctx,
@@ -1321,16 +1311,19 @@ LZ4HC_FindLongerMatch(LZ4HC_CCtx_internal* const ctx,
                      const dictCtx_directive dict,
                      const HCfavor_e favorDecSpeed)
 {
-    LZ4HC_match_t const match0 = { 0 , 0 };
+    LZ4HC_match_t match = { 0 , 0 };
+    const BYTE* matchPtr = NULL;
    /* note : LZ4HC_InsertAndGetWiderMatch() is able to modify the starting position of a match (*startpos),
     * but this won't be the case here, as we define iLowLimit==ip,
-    ** so LZ4HC_InsertAndGetWiderMatch() won't be allowed to search past ip */
-    LZ4HC_match_t md = LZ4HC_InsertAndGetWiderMatch(ctx, ip, ip, iHighLimit, minLen, &ip, nbSearches, 1 /*patternAnalysis*/, 1 /*chainSwap*/, dict, favorDecSpeed);
-    if (md.len <= minLen) return match0;
+     * so LZ4HC_InsertAndGetWiderMatch() won't be allowed to search past ip */
+    int matchLength = LZ4HC_InsertAndGetWiderMatch(ctx, ip, ip, iHighLimit, minLen, &matchPtr, &ip, nbSearches, 1 /*patternAnalysis*/, 1 /*chainSwap*/, dict, favorDecSpeed);
+    if (matchLength <= minLen) return match;
    if (favorDecSpeed) {
-        if ((md.len>18) & (md.len<=36)) md.len=18;   /* favor shortcut */
+        if ((matchLength>18) & (matchLength<=36)) matchLength=18;   /* favor shortcut */
    }
-    return md;
+    match.len = matchLength;
+    match.off = (int)(ip-matchPtr);
+    return match;
 }


@@ -1363,7 +1356,7 @@ static int LZ4HC_compress_optimal ( LZ4HC_CCtx_internal* ctx,
    BYTE* opSaved = (BYTE*) dst;
    BYTE* oend = op + dstCapacity;
    int ovml = MINMATCH;  /* overflow - last sequence */
-    int ovoff = 0;
+    const BYTE* ovref = NULL;

    /* init */
 #if defined(LZ4HC_HEAPMODE) && LZ4HC_HEAPMODE==1
@@ -1386,10 +1379,11 @@ static int LZ4HC_compress_optimal ( LZ4HC_CCtx_internal* ctx,
         if ((size_t)firstMatch.len > sufficient_len) {
             /* good enough solution : immediate encoding */
             int const firstML = firstMatch.len;
+             const BYTE* const matchPos = ip - firstMatch.off;
             opSaved = op;
-             if ( LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), firstML, firstMatch.off, limit, oend) ) {  /* updates ip, op and anchor */
+             if ( LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), firstML, matchPos, limit, oend) ) {  /* updates ip, op and anchor */
                 ovml = firstML;
-                 ovoff = firstMatch.off;
+                 ovref = matchPos;
                 goto _dest_overflow;
             }
             continue;
@@ -1563,9 +1557,9 @@ encode: /* cur, last_match_pos, best_mlen, best_off must be set */
                 assert(ml >= MINMATCH);
                 assert((offset >= 1) && (offset <= LZ4_DISTANCE_MAX));
                 opSaved = op;
-                 if ( LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, offset, limit, oend) ) {  /* updates ip, op and anchor */
+                 if ( LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ml, ip - offset, limit, oend) ) {  /* updates ip, op and anchor */
                     ovml = ml;
-                     ovoff = offset;
+                     ovref = ip - offset;
                     goto _dest_overflow;
         }   }   }
     }  /* while (ip <= mflimit) */
@@ -1624,7 +1618,7 @@ if (limit == fillOutput) {
         if ((oend + LASTLITERALS) - (op + ll_totalCost + 2) - 1 + ovml >= MFLIMIT) {
             DEBUGLOG(6, "Space to end : %i + ml (%i)", (int)((oend + LASTLITERALS) - (op + ll_totalCost + 2) - 1), ovml);
             DEBUGLOG(6, "Before : ip = %p, anchor = %p", ip, anchor);
-             LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ovml, ovoff, notLimited, oend);
+             LZ4HC_encodeSequence(UPDATABLE(ip, op, anchor), ovml, ovref, notLimited, oend);
             DEBUGLOG(6, "After : ip = %p, anchor = %p", ip, anchor);
     }   }
     goto _last_literals;
@@ -67,7 +67,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
    /* Partial decompression with small prefix. */
    LZ4_decompress_safe_partial_usingDict((char const*)dataAfterDict, dst, size,
                                  dstCapacity, dstCapacity, smallDict, smallDictSize);
-    /* Partial decompression with large prefix. */
+    /* Partial decompression wtih large prefix. */
    LZ4_decompress_safe_partial_usingDict((char const*)dataAfterDict, dst, size,
                                  dstCapacity, dstCapacity, largeDict, largeDictSize);
    free(dst);
@@ -242,7 +242,6 @@ LZ4_compressBlockStream(const struct compressionParameters* pThis,
                        int srcSize, int dstSize)
 {
    int const acceleration = (pThis->cLevel < 0) ? -pThis->cLevel + 1 : 1;
-    LZ4_compressResetStream(pThis);
    return LZ4_compress_fast_continue(pThis->LZ4_stream, src, dst, srcSize, dstSize, acceleration);
 }

@@ -251,7 +250,6 @@ LZ4_compressBlockStreamHC(const struct compressionParameters* pThis,
                          const char* src, char* dst,
                          int srcSize, int dstSize)
 {
-    LZ4_compressResetStreamHC(pThis);
    return LZ4_compress_HC_continue(pThis->LZ4_streamHC, src, dst, srcSize, dstSize);
 }

@@ -369,7 +367,6 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
    size_t const maxInSize = (size_t)LZ4_MAX_INPUT_SIZE / decMultiplier;
    size_t const maxDecSize = srcSize < maxInSize ? srcSize * decMultiplier : LZ4_MAX_INPUT_SIZE;
    void* const resultBuffer = malloc(maxDecSize);
-    int benchError = 0;
    U32 nbBlocks;
    struct compressionParameters compP;

@@ -466,10 +463,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                            &compP,
                            blockTable[blockNb].srcPtr, blockTable[blockNb].cPtr,
                            (int)blockTable[blockNb].srcSize, (int)blockTable[blockNb].cRoom);
-                        if (LZ4_isError(rSize)) {
-                            DISPLAY("LZ4 compression failed on block %u", blockNb);
-                            benchError =1 ;
-                        }
+                        if (LZ4_isError(rSize)) END_PROCESS(1, "LZ4 compression failed");
                        blockTable[blockNb].cSize = rSize;
                }   }
                {   U64 const clockSpan = UTIL_clockSpanNano(clockStart);
@@ -528,7 +522,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                                decString, blockNb, (unsigned)blockTable[blockNb].srcSize);
                            if (g_decodeOnly)
                                DISPLAY("Is input using LZ4 Frame format ? \n");
-                            benchError = 1;
+                            END_PROCESS(2, "error during decoding");
                            break;
                        }
                        blockTable[blockNb].resSize = (size_t)regenSize;
@@ -566,7 +560,6 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
                if (crcOrig!=crcCheck) {
                    size_t u;
                    DISPLAY("\n!!! WARNING !!! %17s : Invalid Checksum : %x != %x   \n", displayName, (unsigned)crcOrig, (unsigned)crcCheck);
-                    benchError = 1;
                    for (u=0; u<srcSize; u++) {
                        if (((const BYTE*)srcBuffer)[u] != ((const BYTE*)resultBuffer)[u]) {
                            U32 segNb, bNb, pos;
@@ -605,7 +598,7 @@ static int BMK_benchMem(const void* srcBuffer, size_t srcSize,
    free(blockTable);
    free(compressedBuffer);
    free(resultBuffer);
-    return benchError;
+    return 0;
 }


@@ -633,13 +626,13 @@ static size_t BMK_findMaxMem(U64 requiredMem)
 }


-static int BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
+static void BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
                            const char* displayName, int cLevel, int cLevelLast,
                            const size_t* fileSizes, unsigned nbFiles,
                            const char* dictBuf, int dictSize)
 {
    int l;
-    int benchError = 0;
+
    const char* pch = strrchr(displayName, '\\'); /* Windows */
    if (!pch) pch = strrchr(displayName, '/'); /* Linux */
    if (pch) displayName = pch+1;
@@ -652,13 +645,11 @@ static int BMK_benchCLevel(void* srcBuffer, size_t benchedSize,
    if (cLevelLast < cLevel) cLevelLast = cLevel;

    for (l=cLevel; l <= cLevelLast; l++) {
-        benchError |= BMK_benchMem(
-                            srcBuffer, benchedSize,
-                            displayName, l,
-                            fileSizes, nbFiles,
-                            dictBuf, dictSize);
+        BMK_benchMem(srcBuffer, benchedSize,
+                     displayName, l,
+                     fileSizes, nbFiles,
+                     dictBuf, dictSize);
    }
-    return benchError;
 }


@@ -697,13 +688,12 @@ static void BMK_loadFiles(void* buffer, size_t bufferSize,
    if (totalSize == 0) END_PROCESS(12, "no data to bench");
 }

-static int BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
-                              int cLevel, int cLevelLast,
-                              const char* dictBuf, int dictSize)
+static void BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
+                               int cLevel, int cLevelLast,
+                               const char* dictBuf, int dictSize)
 {
    void* srcBuffer;
    size_t benchedSize;
-    int benchError = 0;
    size_t* fileSizes = (size_t*)malloc(nbFiles * sizeof(size_t));
    U64 const totalSizeToLoad = UTIL_getTotalFileSize(fileNamesTable, nbFiles);
    char mfName[20] = {0};
@@ -730,7 +720,7 @@ static int BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
    /* Bench */
    snprintf (mfName, sizeof(mfName), " %u files", nbFiles);
    {   const char* displayName = (nbFiles > 1) ? mfName : fileNamesTable[0];
-        benchError = BMK_benchCLevel(srcBuffer, benchedSize,
+        BMK_benchCLevel(srcBuffer, benchedSize,
                        displayName, cLevel, cLevelLast,
                        fileSizes, nbFiles,
                        dictBuf, dictSize);
@@ -739,14 +729,12 @@ static int BMK_benchFileTable(const char** fileNamesTable, unsigned nbFiles,
    /* clean up */
    free(srcBuffer);
    free(fileSizes);
-    return benchError;
 }


-static int BMK_syntheticTest(int cLevel, int cLevelLast, double compressibility,
-                             const char* dictBuf, int dictSize)
+static void BMK_syntheticTest(int cLevel, int cLevelLast, double compressibility,
+                              const char* dictBuf, int dictSize)
 {
-    int benchError = 0;
    char name[20] = {0};
    size_t benchedSize = 10000000;
    void* const srcBuffer = malloc(benchedSize);
@@ -759,12 +747,10 @@ static int BMK_syntheticTest(int cLevel, int cLevelLast, double compressibility,

    /* Bench */
    snprintf (name, sizeof(name), "Synthetic %2u%%", (unsigned)(compressibility*100));
-    benchError = BMK_benchCLevel(srcBuffer, benchedSize, name, cLevel, cLevelLast, &benchedSize, 1, dictBuf, dictSize);
+    BMK_benchCLevel(srcBuffer, benchedSize, name, cLevel, cLevelLast, &benchedSize, 1, dictBuf, dictSize);

    /* clean up */
    free(srcBuffer);
-
-    return benchError;
 }


@@ -773,16 +759,15 @@ BMK_benchFilesSeparately(const char** fileNamesTable, unsigned nbFiles,
                   int cLevel, int cLevelLast,
                   const char* dictBuf, int dictSize)
 {
-    int benchError = 0;
    unsigned fileNb;
    if (cLevel > LZ4HC_CLEVEL_MAX) cLevel = LZ4HC_CLEVEL_MAX;
    if (cLevelLast > LZ4HC_CLEVEL_MAX) cLevelLast = LZ4HC_CLEVEL_MAX;
    if (cLevelLast < cLevel) cLevelLast = cLevel;

    for (fileNb=0; fileNb<nbFiles; fileNb++)
-        benchError |= BMK_benchFileTable(fileNamesTable+fileNb, 1, cLevel, cLevelLast, dictBuf, dictSize);
+        BMK_benchFileTable(fileNamesTable+fileNb, 1, cLevel, cLevelLast, dictBuf, dictSize);

-    return benchError;
+    return 0;
 }


@@ -790,7 +775,6 @@ int BMK_benchFiles(const char** fileNamesTable, unsigned nbFiles,
                   int cLevel, int cLevelLast,
                   const char* dictFileName)
 {
-    int benchError = 0;
    double const compressibility = (double)g_compressibilityDefault / 100;
    char* dictBuf = NULL;
    size_t dictSize = 0;
@@ -840,14 +824,14 @@ int BMK_benchFiles(const char** fileNamesTable, unsigned nbFiles,
    }

    if (nbFiles == 0)
-        benchError = BMK_syntheticTest(cLevel, cLevelLast, compressibility, dictBuf, (int)dictSize);
+        BMK_syntheticTest(cLevel, cLevelLast, compressibility, dictBuf, (int)dictSize);
    else {
        if (g_benchSeparately)
-            benchError = BMK_benchFilesSeparately(fileNamesTable, nbFiles, cLevel, cLevelLast, dictBuf, (int)dictSize);
+            BMK_benchFilesSeparately(fileNamesTable, nbFiles, cLevel, cLevelLast, dictBuf, (int)dictSize);
        else
-            benchError = BMK_benchFileTable(fileNamesTable, nbFiles, cLevel, cLevelLast, dictBuf, (int)dictSize);
+            BMK_benchFileTable(fileNamesTable, nbFiles, cLevel, cLevelLast, dictBuf, (int)dictSize);
    }

    free(dictBuf);
-    return benchError;
+    return 0;
 }
@@ -325,16 +325,11 @@ static FILE* LZ4IO_openSrcFile(const char* srcFileName)
        DISPLAYLEVEL(4,"Using stdin for input \n");
        f = stdin;
        SET_BINARY_MODE(stdin);
-        return f;
+    } else {
+        f = fopen(srcFileName, "rb");
+        if (f==NULL) DISPLAYLEVEL(1, "%s: %s \n", srcFileName, strerror(errno));
    }

-    if (UTIL_isDirectory(srcFileName)) {
-        DISPLAYLEVEL(1, "lz4: %s is a directory -- ignored \n", srcFileName);
-        return NULL;
-    }
-
-    f = fopen(srcFileName, "rb");
-    if (f==NULL) DISPLAYLEVEL(1, "%s: %s \n", srcFileName, strerror(errno));
    return f;
 }

@@ -342,8 +337,7 @@ static FILE* LZ4IO_openSrcFile(const char* srcFileName)
 *  prefs is writable, because sparseFileSupport might be updated.
 *  condition : `dstFileName` must be non-NULL.
 * @result : FILE* to `dstFileName`, or NULL if it fails */
-static FILE*
-LZ4IO_openDstFile(const char* dstFileName, const LZ4IO_prefs_t* const prefs)
+static FILE* LZ4IO_openDstFile(const char* dstFileName, const LZ4IO_prefs_t* const prefs)
 {
    FILE* f;
    assert(dstFileName != NULL);
@@ -387,6 +381,7 @@ LZ4IO_openDstFile(const char* dstFileName, const LZ4IO_prefs_t* const prefs)
 }


+
 /***************************************
 *   Legacy Compression
 ***************************************/
@@ -414,10 +409,8 @@ static int LZ4IO_LZ4_compress(const char* src, char* dst, int srcSize, int dstSi
 /* LZ4IO_compressFilename_Legacy :
 * This function is intentionally "hidden" (not published in .h)
 * It generates compressed streams using the old 'legacy' format */
-int LZ4IO_compressFilename_Legacy(const char* input_filename,
-                                  const char* output_filename,
-                                  int compressionlevel,
-                                  const LZ4IO_prefs_t* prefs)
+int LZ4IO_compressFilename_Legacy(const char* input_filename, const char* output_filename,
+                                  int compressionlevel, const LZ4IO_prefs_t* prefs)
 {
    typedef int (*compress_f)(const char* src, char* dst, int srcSize, int dstSize, int cLevel);
    compress_f const compressionFunction = (compressionlevel < 3) ? LZ4IO_LZ4_compress : LZ4_compress_HC;
@@ -570,14 +563,11 @@ static void* LZ4IO_createDict(size_t* dictSize, const char* const dictFilename)
    char*  dictBuf;
    FILE* dictFile;

-    if (!dictFilename)
-        END_PROCESS(26, "Dictionary error : no filename provided");
-    if (!circularBuf)
-        END_PROCESS(25, "Allocation error : not enough memory for circular buffer");
+    if (!circularBuf) END_PROCESS(25, "Allocation error : not enough memory for circular buffer");
+    if (!dictFilename) END_PROCESS(26, "Dictionary error : no filename provided");

    dictFile = LZ4IO_openSrcFile(dictFilename);
-    if (!dictFile)
-        END_PROCESS(27, "Dictionary error : could not open dictionary file");
+    if (!dictFile) END_PROCESS(27, "Dictionary error : could not open dictionary file");

    /* opportunistically seek to the part of the file we care about.
     * If this fails it's not a problem since we'll just read everything anyways. */
@@ -1321,8 +1311,7 @@ LZ4IO_decompressSrcFile(dRess_t ress,

 static int
 LZ4IO_decompressDstFile(dRess_t ress,
-                        const char* input_filename,
-                        const char* output_filename,
+                        const char* input_filename, const char* output_filename,
                        const LZ4IO_prefs_t* const prefs)
 {
    int result;
@@ -195,10 +195,13 @@ UTIL_STATIC void* UTIL_realloc(void* ptr, size_t size)
 /*-****************************************
 *  String functions
 ******************************************/
-/* supports a==NULL or b==NULL */
+/*
+ * A modified version of realloc().
+ * If UTIL_realloc() fails the original block is freed.
+*/
 UTIL_STATIC int UTIL_sameString(const char* a, const char* b)
 {
-    assert(a != NULL || b != NULL);  /* unsupported scenario */
+    assert(a!=NULL && b!=NULL);  /* unsupported scenario */
    if (a==NULL) return 0;
    if (b==NULL) return 0;
    return !strcmp(a,b);
@@ -409,6 +412,7 @@ UTIL_STATIC int UTIL_getFileStat(const char* infilename, stat_t *statbuf)
    return 1;
 }

+
 UTIL_STATIC int UTIL_isRegFD(int fd)
 {
    stat_t statbuf;
@@ -419,25 +423,26 @@ UTIL_STATIC int UTIL_isRegFD(int fd)
    return UTIL_getFDStat(fd, &statbuf); /* Only need to know whether it is a regular file */
 }

+
 UTIL_STATIC int UTIL_isRegFile(const char* infilename)
 {
    stat_t statbuf;
    return UTIL_getFileStat(infilename, &statbuf); /* Only need to know whether it is a regular file */
 }

-UTIL_STATIC int UTIL_isDirectory(const char* infilename)
+
+UTIL_STATIC U32 UTIL_isDirectory(const char* infilename)
 {
-    stat_t statbuf;
    int r;
+    stat_t statbuf;
 #if defined(_MSC_VER)
    r = _stat64(infilename, &statbuf);
-    if (r) return 0;
-    return (statbuf.st_mode & S_IFDIR);
+    if (!r && (statbuf.st_mode & _S_IFDIR)) return 1;
 #else
    r = stat(infilename, &statbuf);
-    if (r) return 0;
-    return (S_ISDIR(statbuf.st_mode));
+    if (!r && S_ISDIR(statbuf.st_mode)) return 1;
 #endif
+    return 0;
 }


@@ -115,6 +115,9 @@ decompress-partial: lz4.o decompress-partial.c
 decompress-partial-usingDict: lz4.o decompress-partial-usingDict.c
 	$(CC) $(FLAGS) $^ -o $@$(EXT)

+freestanding: freestanding.c
+	$(CC) -ffreestanding -nostdlib $^ -o $@$(EXT)
+
 .PHONY: clean
 clean:
 	@$(MAKE) -C $(LZ4DIR) $@ > $(VOID)
@@ -170,7 +173,6 @@ endif
 CAT:=cat
 DD:=dd
 DATAGEN:=./datagen
-PATH:=../programs:$(shell pwd):$(PATH)

 .PHONY: list
 list:
@@ -209,55 +211,312 @@ test-compile-with-lz4-memory-usage:
 # All FPREFIX must start with `tmp`, for `make clean`
 # All tests must clean their temporary test files on successful completion,
 # and only their test files : do not employ sweeping statements such `rm tmp*` or `rm *.lz4`
+test-lz4-sparse: FPREFIX = tmp-tls
 test-lz4-sparse: lz4 datagen
 	@echo "\n ---- test sparse file support ----"
-	./test-lz4-sparse.sh
+	$(DATAGEN) -g5M  -P100 > $(FPREFIX)dg5M
+	$(LZ4) -B4D $(FPREFIX)dg5M -c | $(LZ4) -dv --sparse > $(FPREFIX)cB4
+	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)cB4
+	$(LZ4) -B5D $(FPREFIX)dg5M -c | $(LZ4) -dv --sparse > $(FPREFIX)cB5
+	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)cB5
+	$(LZ4) -B6D $(FPREFIX)dg5M -c | $(LZ4) -dv --sparse > $(FPREFIX)cB6
+	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)cB6
+	$(LZ4) -B7D $(FPREFIX)dg5M -c | $(LZ4) -dv --sparse > $(FPREFIX)cB7
+	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)cB7
+	$(LZ4) $(FPREFIX)dg5M -c | $(LZ4) -dv --no-sparse > $(FPREFIX)nosparse
+	$(DIFF) -s $(FPREFIX)dg5M $(FPREFIX)nosparse
+	ls -ls $(FPREFIX)*
+	$(DATAGEN) -s1 -g1200007 -P100 | $(LZ4) | $(LZ4) -dv --sparse > $(FPREFIX)odd   # Odd size file (to generate non-full last block)
+	$(DATAGEN) -s1 -g1200007 -P100 | $(DIFF) -s - $(FPREFIX)odd
+	ls -ls $(FPREFIX)odd
+	@$(RM) $(FPREFIX)*
+	@echo "\n Compatibility with Console :"
+	echo "Hello World 1 !" | $(LZ4) | $(LZ4) -d -c
+	echo "Hello World 2 !" | $(LZ4) | $(LZ4) -d | $(CAT)
+	echo "Hello World 3 !" | $(LZ4) --no-frame-crc | $(LZ4) -d -c
+	@echo "\n Compatibility with Append :"
+	$(DATAGEN) -P100 -g1M > $(FPREFIX)dg1M
+	$(CAT) $(FPREFIX)dg1M $(FPREFIX)dg1M > $(FPREFIX)2M
+	$(LZ4) -B5 -v $(FPREFIX)dg1M $(FPREFIX)c
+	$(LZ4) -d -v $(FPREFIX)c $(FPREFIX)r
+	$(LZ4) -d -v $(FPREFIX)c -c >> $(FPREFIX)r
+	ls -ls $(FPREFIX)*
+	$(DIFF) $(FPREFIX)2M $(FPREFIX)r
+	@$(RM) $(FPREFIX)*

+test-lz4-contentSize: FPREFIX = tmp-lzc
 test-lz4-contentSize: lz4 datagen
 	@echo "\n ---- test original size support ----"
-	./test-lz4-contentSize.sh
+	$(DATAGEN) -g15M > $(FPREFIX)
+	$(LZ4) -v $(FPREFIX) -c | $(LZ4) -t
+	$(LZ4) -v --content-size $(FPREFIX) -c | $(LZ4) -d > $(FPREFIX)-dup
+	$(DIFF) $(FPREFIX) $(FPREFIX)-dup
+	$(LZ4) -f $(FPREFIX) -c > $(FPREFIX).lz4 # compressed with content size
+	$(LZ4) --content-size $(FPREFIX) -c > $(FPREFIX)-wcz.lz4
+	! $(DIFF) $(FPREFIX).lz4 $(FPREFIX)-wcz.lz4  # must differ, due to content size
+	$(LZ4) --content-size < $(FPREFIX) > $(FPREFIX)-wcz2.lz4 # can determine content size because stdin is just a file
+	$(DIFF) $(FPREFIX)-wcz.lz4 $(FPREFIX)-wcz2.lz4  # both must contain content size
+	$(CAT) $(FPREFIX) | $(LZ4) > $(FPREFIX)-ncz.lz4
+	$(DIFF) $(FPREFIX).lz4 $(FPREFIX)-ncz.lz4  # both don't have content size
+	$(CAT) $(FPREFIX) | $(LZ4) --content-size > $(FPREFIX)-ncz2.lz4 # can't determine content size
+	$(DIFF) $(FPREFIX).lz4 $(FPREFIX)-ncz2.lz4  # both don't have content size
+	@$(RM) $(FPREFIX)*

+test-lz4-frame-concatenation: FPREFIX = tmp-lfc
 test-lz4-frame-concatenation: lz4 datagen
 	@echo "\n ---- test frame concatenation ----"
-	./test-lz4-frame-concatenation.sh
+	@echo -n > $(FPREFIX)-empty
+	@echo hi > $(FPREFIX)-nonempty
+	$(CAT) $(FPREFIX)-nonempty $(FPREFIX)-empty $(FPREFIX)-nonempty > $(FPREFIX)-src
+	$(LZ4) -zq $(FPREFIX)-empty -c > $(FPREFIX)-empty.lz4
+	$(LZ4) -zq $(FPREFIX)-nonempty -c > $(FPREFIX)-nonempty.lz4
+	$(CAT) $(FPREFIX)-nonempty.lz4 $(FPREFIX)-empty.lz4 $(FPREFIX)-nonempty.lz4 > $(FPREFIX)-concat.lz4
+	$(LZ4) -d $(FPREFIX)-concat.lz4 -c > $(FPREFIX)-result
+	$(CMP) $(FPREFIX)-src $(FPREFIX)-result
+	@$(RM) $(FPREFIX)*
+	@echo frame concatenation test completed

+test-lz4-multiple: FPREFIX = tmp-tml
 test-lz4-multiple: lz4 datagen
 	@echo "\n ---- test multiple files ----"
-	./test-lz4-multiple.sh
+	@$(DATAGEN) -s1        > $(FPREFIX)1 2> $(VOID)
+	@$(DATAGEN) -s2 -g100K > $(FPREFIX)2 2> $(VOID)
+	@$(DATAGEN) -s3 -g200K > $(FPREFIX)3 2> $(VOID)
+	# compress multiple files : one .lz4 per source file
+	$(LZ4) -f -m $(FPREFIX)*
+	test -f $(FPREFIX)1.lz4
+	test -f $(FPREFIX)2.lz4
+	test -f $(FPREFIX)3.lz4
+	# decompress multiple files : one output file per .lz4
+	mv $(FPREFIX)1 $(FPREFIX)1-orig
+	mv $(FPREFIX)2 $(FPREFIX)2-orig
+	mv $(FPREFIX)3 $(FPREFIX)3-orig
+	$(LZ4) -d -f -m $(FPREFIX)*.lz4
+	$(CMP) $(FPREFIX)1 $(FPREFIX)1-orig   # must be identical
+	$(CMP) $(FPREFIX)2 $(FPREFIX)2-orig
+	$(CMP) $(FPREFIX)3 $(FPREFIX)3-orig
+	# compress multiple files into stdout
+	$(CAT) $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 > $(FPREFIX)-concat1
+	$(RM) $(FPREFIX)*.lz4
+	$(LZ4) -m $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3 -c > $(FPREFIX)-concat2
+	test ! -f $(FPREFIX)1.lz4  # must not create .lz4 artefact
+	$(CMP) $(FPREFIX)-concat1 $(FPREFIX)-concat2  # must be equivalent
+	# decompress multiple files into stdout
+	$(RM) $(FPREFIX)-concat1 $(FPREFIX)-concat2
+	$(LZ4) -f -m $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3   # generate .lz4 to decompress
+	$(CAT) $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3 > $(FPREFIX)-concat1   # create concatenated reference
+	$(RM) $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3
+	$(LZ4) -d -m $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 -c > $(FPREFIX)-concat2
+	test ! -f $(FPREFIX)1  # must not create file artefact
+	$(CMP) $(FPREFIX)-concat1 $(FPREFIX)-concat2  # must be equivalent
+	# compress multiple files, one of which is absent (must fail)
+	! $(LZ4) -f -m $(FPREFIX)-concat1 notHere $(FPREFIX)-concat2  # must fail : notHere not present
+	# test lz4-compressed file
+	$(LZ4) -tm $(FPREFIX)-concat1.lz4
+	$(LZ4) -tm $(FPREFIX)-concat1.lz4 $(FPREFIX)-concat2.lz4
+	# test multiple lz4 files, one of which is absent (must fail)
+	! $(LZ4) -tm $(FPREFIX)-concat1.lz4 notHere.lz4 $(FPREFIX)-concat2.lz4
+	@$(RM) $(FPREFIX)*

+test-lz4-multiple-legacy: FPREFIX = tmp-lml
 test-lz4-multiple-legacy: lz4 datagen
 	@echo "\n ---- test multiple files (Legacy format) ----"
-	./test-lz4-multiple-legacy.sh
+	@$(DATAGEN) -s1        > $(FPREFIX)1 2> $(VOID)
+	@$(DATAGEN) -s2 -g100K > $(FPREFIX)2 2> $(VOID)
+	@$(DATAGEN) -s3 -g200K > $(FPREFIX)3 2> $(VOID)
+	# compress multiple files using legacy format: one .lz4 per source file
+	$(LZ4) -f -l -m $(FPREFIX)*
+	test -f $(FPREFIX)1.lz4
+	test -f $(FPREFIX)2.lz4
+	test -f $(FPREFIX)3.lz4
+	# decompress multiple files compressed using legacy format: one output file per .lz4
+	mv $(FPREFIX)1 $(FPREFIX)1-orig
+	mv $(FPREFIX)2 $(FPREFIX)2-orig
+	mv $(FPREFIX)3 $(FPREFIX)3-orig
+	$(LZ4) -d -f -m $(FPREFIX)*.lz4
+	$(LZ4) -l -d -f -m $(FPREFIX)*.lz4 # -l mustn't impact -d option
+	$(CMP) $(FPREFIX)1 $(FPREFIX)1-orig   # must be identical
+	$(CMP) $(FPREFIX)2 $(FPREFIX)2-orig
+	$(CMP) $(FPREFIX)3 $(FPREFIX)3-orig
+	# compress multiple files into stdout using legacy format
+	$(CAT) $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 > $(FPREFIX)-concat1
+	$(RM) $(FPREFIX)*.lz4
+	$(LZ4) -l -m $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3 -c > $(FPREFIX)-concat2
+	test ! -f $(FPREFIX)1.lz4  # must not create .lz4 artefact
+	$(CMP) $(FPREFIX)-concat1 $(FPREFIX)-concat2  # must be equivalent
+	# # # decompress multiple files into stdout using legacy format
+	$(RM) $(FPREFIX)-concat1 $(FPREFIX)-concat2
+	$(LZ4) -l -f -m $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3   # generate .lz4 to decompress
+	$(CAT) $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3 > $(FPREFIX)-concat1   # create concatenated reference
+	$(RM) $(FPREFIX)1 $(FPREFIX)2 $(FPREFIX)3
+	$(LZ4) -d -m $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 -c > $(FPREFIX)-concat2
+	$(LZ4) -d -l -m $(FPREFIX)1.lz4 $(FPREFIX)2.lz4 $(FPREFIX)3.lz4 -c > $(FPREFIX)-concat2 # -l mustn't impact option -d
+	test ! -f $(FPREFIX)1  # must not create file artefact
+	$(CMP) $(FPREFIX)-concat1 $(FPREFIX)-concat2  # must be equivalent
+	# # # compress multiple files, one of which is absent (must fail)
+	! $(LZ4) -f -l -m $(FPREFIX)-concat1 notHere-legacy $(FPREFIX)-concat2  # must fail : notHere-legacy not present
+	@$(RM) $(FPREFIX)*

-test-lz4-skippable: lz4
+SKIPFILE = goldenSamples/skip.bin
+test-lz4-skippable: FPREFIX = tmp-lsk
+test-lz4-skippable: lz4 datagen
 	@echo "\n ---- test lz4 with skippable frames ----"
-	./test-lz4-skippable.sh
+	$(LZ4) -dc $(SKIPFILE)
+	$(LZ4) -dc < $(SKIPFILE)
+	cat $(SKIPFILE) | $(LZ4) -dc
+	echo "Hello from Valid Frame!\n" | $(LZ4) -c > $(FPREFIX).lz4
+	cat $(SKIPFILE) $(FPREFIX).lz4 $(SKIPFILE) | $(LZ4) -dc
+	$(RM) $(FPREFIX)*

+test-lz4-basic: FPREFIX = tmp-tlb
 test-lz4-basic: lz4 datagen unlz4 lz4cat
 	@echo "\n ---- test lz4 basic compression/decompression ----"
-	./test-lz4-basic.sh
+	$(DATAGEN) -g0       | $(LZ4) -v     | $(LZ4) -t
+	$(DATAGEN) -g16KB    | $(LZ4) -9     | $(LZ4) -t
+	$(DATAGEN) -g20KB > $(FPREFIX)-dg20k
+	$(LZ4) < $(FPREFIX)-dg20k | $(LZ4) -d > $(FPREFIX)-dec
+	$(DIFF) -q $(FPREFIX)-dg20k $(FPREFIX)-dec
+	$(LZ4) --no-frame-crc < $(FPREFIX)-dg20k | $(LZ4) -d > $(FPREFIX)-dec
+	$(DIFF) -q $(FPREFIX)-dg20k $(FPREFIX)-dec
+	$(DATAGEN)           | $(LZ4) -BI    | $(LZ4) -t
+	$(DATAGEN)           | $(LZ4) --no-crc | $(LZ4) -t
+	$(DATAGEN) -g6M -P99 | $(LZ4) -9BD   | $(LZ4) -t
+	$(DATAGEN) -g17M     | $(LZ4) -9v    | $(LZ4) -qt
+	$(DATAGEN) -g33M     | $(LZ4) --no-frame-crc | $(LZ4) -t
+	$(DATAGEN) -g256MB   | $(LZ4) -vqB4D | $(LZ4) -t --no-crc
+	@echo "hello world" > $(FPREFIX)-hw
+	$(LZ4) --rm -f $(FPREFIX)-hw $(FPREFIX)-hw.lz4
+	test ! -f $(FPREFIX)-hw                      # must fail (--rm)
+	test   -f $(FPREFIX)-hw.lz4
+	$(PRGDIR)/lz4cat $(FPREFIX)-hw.lz4 | $(GREP) "hello world"
+	$(PRGDIR)/unlz4 --rm $(FPREFIX)-hw.lz4 $(FPREFIX)-hw
+	test   -f $(FPREFIX)-hw
+	test ! -f $(FPREFIX)-hw.lz4                  # must fail (--rm)
+	test ! -f $(FPREFIX)-hw.lz4.lz4              # must fail (unlz4)
+	$(PRGDIR)/lz4cat $(FPREFIX)-hw               # pass-through mode
+	test   -f $(FPREFIX)-hw
+	test ! -f $(FPREFIX)-hw.lz4                  # must fail (lz4cat)
+	$(LZ4) $(FPREFIX)-hw $(FPREFIX)-hw.lz4          # creates $(FPREFIX)-hw.lz4
+	$(PRGDIR)/lz4cat < $(FPREFIX)-hw.lz4 > $(FPREFIX)3  # checks lz4cat works with stdin (#285)
+	$(DIFF) -q $(FPREFIX)-hw $(FPREFIX)3
+	$(PRGDIR)/lz4cat < $(FPREFIX)-hw > $(FPREFIX)2      # checks lz4cat works in pass-through mode
+	$(DIFF) -q $(FPREFIX)-hw $(FPREFIX)2
+	cp $(FPREFIX)-hw ./-d
+	$(LZ4) --rm -- -d -d.lz4               # compresses ./d into ./-d.lz4
+	test   -f ./-d.lz4
+	test ! -f ./-d
+	mv ./-d.lz4 ./-z
+	$(LZ4) -d --rm -- -z $(FPREFIX)4          # uncompresses ./-z into $(FPREFIX)4
+	test ! -f ./-z
+	$(DIFF) -q $(FPREFIX)-hw $(FPREFIX)4
+	! $(LZ4) $(FPREFIX)2 $(FPREFIX)3 $(FPREFIX)4    # must fail: refuse to handle 3+ file names
+	$(LZ4) -f $(FPREFIX)-hw                   # create $(FPREFIX)-hw.lz4, for next tests
+	$(LZ4) --list $(FPREFIX)-hw.lz4           # test --list on valid single-frame file
+	$(LZ4) --list < $(FPREFIX)-hw.lz4         # test --list from stdin (file only)
+	$(CAT) $(FPREFIX)-hw >> $(FPREFIX)-hw.lz4
+	! $(LZ4) -f $(FPREFIX)-hw.lz4             # uncompress valid frame followed by invalid data (must fail now)
+	$(LZ4) -BX $(FPREFIX)-hw -c -q | $(LZ4) -tv  # test block checksum
+	# $(DATAGEN) -g20KB generates the same file every single time
+	# cannot save output of $(DATAGEN) -g20KB as input file to lz4 because the following shell commands are run before $(DATAGEN) -g20KB
+	test "$(shell $(DATAGEN) -g20KB | $(LZ4) -c --fast | wc -c)" -lt "$(shell $(DATAGEN) -g20KB | $(LZ4) -c --fast=9 | wc -c)" # -1 vs -9
+	test "$(shell $(DATAGEN) -g20KB | $(LZ4) -c -1 | wc -c)" -lt "$(shell $(DATAGEN) -g20KB| $(LZ4) -c --fast=1 | wc -c)" # 1 vs -1
+	test "$(shell $(DATAGEN) -g20KB | $(LZ4) -c --fast=1 | wc -c)" -eq "$(shell $(DATAGEN) -g20KB| $(LZ4) -c --fast| wc -c)" # checks default fast compression is -1
+	! $(LZ4) -c --fast=0 $(FPREFIX)-dg20K # lz4 should fail when fast=0
+	! $(LZ4) -c --fast=-1 $(FPREFIX)-dg20K # lz4 should fail when fast=-1
+	# High --fast values can result in out-of-bound dereferences #876
+	$(DATAGEN) -g1M | $(LZ4) -c --fast=999999999 > /dev/null
+	# Test for #596
+	@echo "TEST" > $(FPREFIX)-test
+	$(LZ4) -m $(FPREFIX)-test
+	$(LZ4) $(FPREFIX)-test.lz4 $(FPREFIX)-test2
+	$(DIFF) -q $(FPREFIX)-test $(FPREFIX)-test2
+	@$(RM) $(FPREFIX)*

+
+test-lz4-dict: FPREFIX = tmp-dict
 test-lz4-dict: lz4 datagen
 	@echo "\n ---- test lz4 compression/decompression with dictionary ----"
-	./test-lz4-dict.sh
+	$(DATAGEN) -g16KB > $(FPREFIX)
+	$(DATAGEN) -g32KB > $(FPREFIX)-sample-32k
+	< $(FPREFIX)-sample-32k $(LZ4) -D $(FPREFIX) | $(LZ4) -dD $(FPREFIX) | diff - $(FPREFIX)-sample-32k
+	$(DATAGEN) -g128MB > $(FPREFIX)-sample-128m
+	< $(FPREFIX)-sample-128m $(LZ4) -D $(FPREFIX) | $(LZ4) -dD $(FPREFIX) | diff - $(FPREFIX)-sample-128m
+	touch $(FPREFIX)-sample-0
+	< $(FPREFIX)-sample-0 $(LZ4) -D $(FPREFIX) | $(LZ4) -dD $(FPREFIX) | diff - $(FPREFIX)-sample-0
+
+	< $(FPREFIX)-sample-32k $(LZ4) -D $(FPREFIX)-sample-0 | $(LZ4) -dD $(FPREFIX)-sample-0 | diff - $(FPREFIX)-sample-32k
+	< $(FPREFIX)-sample-0 $(LZ4) -D $(FPREFIX)-sample-0 | $(LZ4) -dD $(FPREFIX)-sample-0 | diff - $(FPREFIX)-sample-0
+
+	@echo "\n ---- test lz4 dictionary loading ----"
+	$(DATAGEN) -g128KB > $(FPREFIX)-data-128KB
+	set -e; \
+	for l in 0 1 4 128 32767 32768 32769 65535 65536 65537 98303 98304 98305 131071 131072 131073; do \
+		$(DATAGEN) -g$$l > $(FPREFIX)-$$l; \
+		$(DD) if=$(FPREFIX)-$$l of=$(FPREFIX)-$$l-tail bs=1 count=65536 skip=$$((l > 65536 ? l - 65536 : 0)); \
+		< $(FPREFIX)-$$l      $(LZ4) -D stdin $(FPREFIX)-data-128KB -c | $(LZ4) -dD $(FPREFIX)-$$l-tail | $(DIFF) - $(FPREFIX)-data-128KB; \
+		< $(FPREFIX)-$$l-tail $(LZ4) -D stdin $(FPREFIX)-data-128KB -c | $(LZ4) -dD $(FPREFIX)-$$l      | $(DIFF) - $(FPREFIX)-data-128KB; \
+	done
+	@$(RM) $(FPREFIX)*

 test-lz4hc-hugefile: lz4 datagen
 	@echo "\n ---- test HC compression/decompression of huge files ----"
-	./test-lz4hc-hugefile.sh
+	$(DATAGEN) -g4200MB | $(LZ4) -v3BD | $(LZ4) -qt

+test-lz4-fast-hugefile: FPREFIX = tmp-lfh
 test-lz4-fast-hugefile: lz4 datagen
 	@echo "\n ---- test huge files compression/decompression ----"
-	./test-lz4-fast-hugefile.sh
+	$(DATAGEN) -g6GB    | $(LZ4) -vB5D | $(LZ4) -qt
+	# test large file size [2-4] GB
+	@$(DATAGEN) -g3G -P100 | $(LZ4) -vv | $(LZ4) --decompress --force --sparse - $(FPREFIX)1
+	@ls -ls $(FPREFIX)1
+	@$(DATAGEN) -g3G -P100 | $(LZ4) --quiet --content-size | $(LZ4) --verbose --decompress --force --sparse - $(FPREFIX)2
+	@ls -ls $(FPREFIX)2
+	$(DIFF) -s $(FPREFIX)1 $(FPREFIX)2
+	@$(RM) $(FPREFIX)*

 test-lz4-hugefile: test-lz4-fast-hugefile test-lz4hc-hugefile

+test-lz4-testmode: FPREFIX = tmp-ltm
 test-lz4-testmode: lz4 datagen
 	@echo "\n ---- bench mode ----"
-	./test-lz4-testmode.sh
+	$(LZ4) -bi0
+	$(DATAGEN) > $(FPREFIX)
+	$(LZ4) -f $(FPREFIX) -c > $(FPREFIX).lz4
+	$(LZ4) -bdi0 $(FPREFIX).lz4 # test benchmark decode-only mode
+	$(LZ4) -bdi0 --no-crc $(FPREFIX).lz4 # test benchmark decode-only mode
+	@echo "\n ---- test mode ----"
+	! $(DATAGEN) | $(LZ4) -t
+	! $(DATAGEN) | $(LZ4) -tf
+	@echo "\n ---- pass-through mode ----"
+	@echo "Why hello there " > $(FPREFIX)2.lz4
+	! $(LZ4) -f $(FPREFIX)2.lz4 > $(VOID)
+	! $(DATAGEN) | $(LZ4) -dc  > $(VOID)
+	! $(DATAGEN) | $(LZ4) -df > $(VOID)
+	$(DATAGEN) | $(LZ4) -dcf > $(VOID)
+	@echo "Hello World !" > $(FPREFIX)1
+	$(LZ4) -dcf $(FPREFIX)1
+	@echo "from underground..." > $(FPREFIX)2
+	$(LZ4) -dcfm $(FPREFIX)1 $(FPREFIX)2
+	@echo "\n ---- non-existing source (must fail cleanly) ----"
+	! $(LZ4)     file-does-not-exist
+	! $(LZ4) -f  file-does-not-exist
+	! $(LZ4) -t  file-does-not-exist
+	! $(LZ4) -fm file1-dne file2-dne
+	@$(RM) $(FPREFIX)*

 test-lz4-opt-parser: lz4 datagen
 	@echo "\n ---- test opt-parser ----"
-	./test-lz4-opt-parser.sh
+	$(DATAGEN) -g16KB      | $(LZ4) -12      | $(LZ4) -t
+	$(DATAGEN) -P10        | $(LZ4) -12B4    | $(LZ4) -t
+	$(DATAGEN) -g256K      | $(LZ4) -12B4D   | $(LZ4) -t
+	$(DATAGEN) -g512K -P25 | $(LZ4) -12BD    | $(LZ4) -t
+	$(DATAGEN) -g1M        | $(LZ4) -12B5    | $(LZ4) -t
+	$(DATAGEN) -g1M -s2    | $(LZ4) -12B4D   | $(LZ4) -t
+	$(DATAGEN) -g2M -P99   | $(LZ4) -11B4D   | $(LZ4) -t
+	$(DATAGEN) -g4M        | $(LZ4) -11vq    | $(LZ4) -qt
+	$(DATAGEN) -g8M        | $(LZ4) -11B4    | $(LZ4) -t
+	$(DATAGEN) -g16M -P90  | $(LZ4) -11B5    | $(LZ4) -t
+	$(DATAGEN) -g32M -P10  | $(LZ4) -11B5D   | $(LZ4) -t

 test-lz4-essentials : lz4 datagen test-lz4-basic test-lz4-multiple test-lz4-multiple-legacy \
                      test-lz4-frame-concatenation test-lz4-testmode \
@@ -350,44 +609,10 @@ test-decompress-partial : decompress-partial decompress-partial-usingDict
 	@echo "\n ---- test decompress-partial-usingDict ----"
 	./decompress-partial-usingDict$(EXT)

-
-#-----------------------------------------------------------------------------
-# freestanding test only for Linux x86_64
-#-----------------------------------------------------------------------------
-ifeq ($(OS),Windows_NT)
-  UNAME_S := Windows
-else
-  UNAME_S := $(shell uname -s)
-endif
-
-ifeq ($(OS),Windows_NT)
-  UNAME_P := Unknown
-else
-  UNAME_P := $(shell uname -p)
-endif
-
-FREESTANDING_CFLAGS := -ffreestanding -nostdlib
-
-ifneq ($(UNAME_S), Linux)
-  FREESTANDING_CFLAGS :=
-endif
-
-ifneq ($(UNAME_P), x86_64)
-  FREESTANDING_CFLAGS :=
-endif
-
-freestanding: freestanding.c
-	$(CC) $(FREESTANDING_CFLAGS) $^ -o $@$(EXT)
-
 test-freestanding: freestanding
 	@echo "\n ---- test freestanding ----"
-ifeq ($(FREESTANDING_CFLAGS),)
-	@echo "\n (skip)"
-else
 	./freestanding$(EXT)
 	-strace ./freestanding$(EXT)
 	-ltrace ./freestanding$(EXT)
-endif
-

 endif
@@ -30,6 +30,7 @@


 #if !defined(__x86_64__) || !defined(__linux__)
+EXTERN_C void _start(void) { }
 int main(int argc, char** argv) { return 0; }
 #else

@@ -371,33 +371,6 @@ static int local_LZ4F_compressFrame(const char* in, char* out, int inSize)
    return (int)LZ4F_compressFrame(out, LZ4F_compressFrameBound((size_t)inSize, NULL), in, (size_t)inSize, NULL);
 }

-LZ4F_cctx* g_cctx = NULL;
-static int local_LZ4F_compress(const char* in, char* out, int inSize)
-{
-    /* output buffer size is assumed */
-    size_t const outSize = LZ4F_compressFrameBound((size_t)inSize, NULL);
-    size_t cSize = 0;
-    assert(inSize >= 0);
-    if (g_cctx == NULL) {
-        /* create and initialize LZ4F compression context the first time */
-        LZ4F_createCompressionContext(&g_cctx, LZ4F_VERSION);
-        assert(g_cctx != NULL);
-    } /* re-use existing compression context otherwise */
-    {   size_t const cbSize = LZ4F_compressBegin(g_cctx, out, outSize, NULL);
-        assert(!LZ4F_isError(cbSize));
-        cSize += cbSize;
-    }
-    {   size_t const cuSize = LZ4F_compressUpdate(g_cctx, out+cSize, outSize-cSize, in, (size_t)inSize, NULL);
-        assert(!LZ4F_isError(cuSize));
-        cSize += cuSize;
-    }
-    {   size_t const ceSize = LZ4F_compressEnd(g_cctx, out+cSize, outSize-cSize, NULL);
-        assert(!LZ4F_isError(ceSize));
-        cSize += ceSize;
-    }
-    return (int)cSize;
-}
-
 static LZ4F_decompressionContext_t g_dCtx;

 static int local_LZ4F_decompress(const char* in, char* out, int inSize, int outSize)
@@ -613,9 +586,6 @@ int fullSpeedBench(const char** fileNamesTable, int nbFiles)
            case 30: compressionFunction = local_LZ4F_compressFrame; compressorName = "LZ4F_compressFrame";
                        chunkP[0].origSize = (int)benchedSize; nbChunks=1;
                        break;
-            case 31: compressionFunction = local_LZ4F_compress; compressorName = "LZ4F_compressUpdate";
-                        chunkP[0].origSize = (int)benchedSize; nbChunks=1;
-                        break;
            case 40: compressionFunction = local_LZ4_saveDict; compressorName = "LZ4_saveDict";
                        if (chunkP[0].origSize < 8) { DISPLAY(" cannot bench %s with less then 8 bytes \n", compressorName); continue; }
                        LZ4_loadDict(&LZ4_stream, chunkP[0].origBuffer, chunkP[0].origSize);
@@ -1,76 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-tlb"
-
-set -e
-
-remove () {
-    rm -rf $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-datagen -g0       | lz4 -v     | lz4 -t
-datagen -g16KB    | lz4 -9     | lz4 -t
-datagen -g20KB > $FPREFIX-dg20k
-lz4 < $FPREFIX-dg20k | lz4 -d > $FPREFIX-dec
-diff -q $FPREFIX-dg20k $FPREFIX-dec
-lz4 --no-frame-crc < $FPREFIX-dg20k | lz4 -d > $FPREFIX-dec
-diff -q $FPREFIX-dg20k $FPREFIX-dec
-datagen           | lz4 -BI    | lz4 -t
-datagen           | lz4 --no-crc | lz4 -t
-datagen -g6M -P99 | lz4 -9BD   | lz4 -t
-datagen -g17M     | lz4 -9v    | lz4 -qt
-datagen -g33M     | lz4 --no-frame-crc | lz4 -t
-datagen -g256MB   | lz4 -vqB4D | lz4 -t --no-crc
-echo "hello world" > $FPREFIX-hw
-lz4 --rm -f $FPREFIX-hw $FPREFIX-hw.lz4
-test ! -f $FPREFIX-hw                   # must fail (--rm)
-test   -f $FPREFIX-hw.lz4
-lz4cat $FPREFIX-hw.lz4 | grep "hello world"
-unlz4 --rm $FPREFIX-hw.lz4 $FPREFIX-hw
-test   -f $FPREFIX-hw
-test ! -f $FPREFIX-hw.lz4               # must fail (--rm)
-test ! -f $FPREFIX-hw.lz4.lz4           # must fail (unlz4)
-lz4cat $FPREFIX-hw                      # pass-through mode
-test   -f $FPREFIX-hw
-test ! -f $FPREFIX-hw.lz4               # must fail (lz4cat)
-lz4 $FPREFIX-hw $FPREFIX-hw.lz4         # creates $FPREFIX-hw.lz4
-lz4cat < $FPREFIX-hw.lz4 > ${FPREFIX}3  # checks lz4cat works with stdin (#285)
-diff -q $FPREFIX-hw ${FPREFIX}3
-lz4cat < $FPREFIX-hw > ${FPREFIX}2      # checks lz4cat works in pass-through mode
-diff -q $FPREFIX-hw ${FPREFIX}2
-cp $FPREFIX-hw ./-d
-lz4 --rm -- -d -d.lz4                   # compresses ./d into ./-d.lz4
-test   -f ./-d.lz4
-test ! -f ./-d
-mv ./-d.lz4 ./-z
-lz4 -d --rm -- -z ${FPREFIX}4           # uncompresses ./-z into $FPREFIX4
-test ! -f ./-z
-diff -q $FPREFIX-hw ${FPREFIX}4
-lz4 ${FPREFIX}2 ${FPREFIX}3 ${FPREFIX}4 && exit 1  # must fail: refuse to handle 3+ file names
-mkdir -p ${FPREFIX}-dir
-lz4 ${FPREFIX}-dir && exit 1          # must fail: refuse to compress directory
-test ! -f ${FPREFIX}-dir.lz4          # must not create artifact (#1211)
-lz4 -f $FPREFIX-hw                    # create $FPREFIX-hw.lz4, for next tests
-lz4 --list $FPREFIX-hw.lz4            # test --list on valid single-frame file
-lz4 --list < $FPREFIX-hw.lz4          # test --list from stdin (file only)
-cat $FPREFIX-hw >> $FPREFIX-hw.lz4
-lz4 -f $FPREFIX-hw.lz4 && exit 1      # uncompress valid frame followed by invalid data (must fail now)
-lz4 -BX $FPREFIX-hw -c -q | lz4 -tv   # test block checksum
-# datagen -g20KB generates the same file every single time
-# cannot save output of datagen -g20KB as input file to lz4 because the following shell commands are run before datagen -g20KB
-test "$(datagen -g20KB | lz4 -c --fast | wc -c)" -lt "$(datagen -g20KB | lz4 -c --fast=9 | wc -c)" # -1 vs -9
-test "$(datagen -g20KB | lz4 -c -1 | wc -c)" -lt "$(datagen -g20KB| lz4 -c --fast=1 | wc -c)" # 1 vs -1
-test "$(datagen -g20KB | lz4 -c --fast=1 | wc -c)" -eq "$(datagen -g20KB| lz4 -c --fast| wc -c)" # checks default fast compression is -1
-lz4 -c --fast=0 $FPREFIX-dg20K && exit 1  # lz4 should fail when fast=0
-lz4 -c --fast=-1 $FPREFIX-dg20K && exit 1 # lz4 should fail when fast=-1
-# High --fast values can result in out-of-bound dereferences #876
-datagen -g1M | lz4 -c --fast=999999999 > $FPREFIX-trash
-# Test for #596
-echo "TEST" > $FPREFIX-test
-lz4 -m $FPREFIX-test
-lz4 $FPREFIX-test.lz4 $FPREFIX-test2
-diff -q $FPREFIX-test $FPREFIX-test2
@@ -1,27 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-lzc"
-
-set -e
-
-remove () {
-    rm $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-datagen -g15M > $FPREFIX
-lz4 -v $FPREFIX -c | lz4 -t
-lz4 -v --content-size $FPREFIX -c | lz4 -d > $FPREFIX-dup
-diff $FPREFIX $FPREFIX-dup
-lz4 -f $FPREFIX -c > $FPREFIX.lz4 # compressed with content size
-lz4 --content-size $FPREFIX -c > $FPREFIX-wcz.lz4
-diff $FPREFIX.lz4 $FPREFIX-wcz.lz4 && exit 1 # must differ, due to content size
-lz4 --content-size < $FPREFIX > $FPREFIX-wcz2.lz4 # can determine content size because stdin is just a file
-diff $FPREFIX-wcz.lz4 $FPREFIX-wcz2.lz4  # both must contain content size
-cat $FPREFIX | lz4 > $FPREFIX-ncz.lz4
-diff $FPREFIX.lz4 $FPREFIX-ncz.lz4  # both don't have content size
-cat $FPREFIX | lz4 --content-size > $FPREFIX-ncz2.lz4 # can't determine content size
-diff $FPREFIX.lz4 $FPREFIX-ncz2.lz4  # both don't have content size
@@ -1,35 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-dict"
-
-set -e
-
-remove () {
-    rm $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-datagen -g16KB > $FPREFIX
-datagen -g32KB > $FPREFIX-sample-32k
-< $FPREFIX-sample-32k lz4 -D $FPREFIX | lz4 -dD $FPREFIX | diff - $FPREFIX-sample-32k
-datagen -g128MB > $FPREFIX-sample-128m
-< $FPREFIX-sample-128m lz4 -D $FPREFIX | lz4 -dD $FPREFIX | diff - $FPREFIX-sample-128m
-touch $FPREFIX-sample-0
-< $FPREFIX-sample-0 lz4 -D $FPREFIX | lz4 -dD $FPREFIX | diff - $FPREFIX-sample-0
-
-< $FPREFIX-sample-32k lz4 -D $FPREFIX-sample-0 | lz4 -dD $FPREFIX-sample-0 | diff - $FPREFIX-sample-32k
-< $FPREFIX-sample-0 lz4 -D $FPREFIX-sample-0 | lz4 -dD $FPREFIX-sample-0 | diff - $FPREFIX-sample-0
-lz4 -bi0 -D $FPREFIX $FPREFIX-sample-32k $FPREFIX-sample-32k
-
-echo "---- test lz4 dictionary loading ----"
-datagen -g128KB > $FPREFIX-data-128KB
-set -e; \
-for l in 0 1 4 128 32767 32768 32769 65535 65536 65537 98303 98304 98305 131071 131072 131073; do \
-    datagen -g$$l > $FPREFIX-$$l; \
-    dd if=$FPREFIX-$$l of=$FPREFIX-$$l-tail bs=1 count=65536 skip=$((l > 65536 ? l - 65536 : 0)); \
-    < $FPREFIX-$$l      lz4 -D stdin $FPREFIX-data-128KB -c | lz4 -dD $FPREFIX-$$l-tail | diff - $FPREFIX-data-128KB; \
-    < $FPREFIX-$$l-tail lz4 -D stdin $FPREFIX-data-128KB -c | lz4 -dD $FPREFIX-$$l      | diff - $FPREFIX-data-128KB; \
-done
@@ -1,21 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-lfh"
-
-set -e
-
-remove () {
-    rm $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-datagen -g6GB    | lz4 -vB5D | lz4 -qt
-# test large file size [2-4] GB
-datagen -g3G -P100 | lz4 -vv | lz4 --decompress --force --sparse - ${FPREFIX}1
-ls -ls ${FPREFIX}1
-datagen -g3G -P100 | lz4 --quiet --content-size | lz4 --verbose --decompress --force --sparse - ${FPREFIX}2
-ls -ls ${FPREFIX}2
-diff -s ${FPREFIX}1 ${FPREFIX}2
@@ -1,21 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-lfc"
-
-set -e
-remove () {
-    rm $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-echo > $FPREFIX-empty
-echo hi > $FPREFIX-nonempty
-cat $FPREFIX-nonempty $FPREFIX-empty $FPREFIX-nonempty > $FPREFIX-src
-lz4 -zq $FPREFIX-empty -c > $FPREFIX-empty.lz4
-lz4 -zq $FPREFIX-nonempty -c > $FPREFIX-nonempty.lz4
-cat $FPREFIX-nonempty.lz4 $FPREFIX-empty.lz4 $FPREFIX-nonempty.lz4 > $FPREFIX-concat.lz4
-lz4 -d $FPREFIX-concat.lz4 -c > $FPREFIX-result
-cmp $FPREFIX-src $FPREFIX-result
@@ -1,49 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-lml"
-
-set -e
-
-remove () {
-    rm $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-datagen -s1        > "${FPREFIX}1" 2> $FPREFIX-trash
-datagen -s2 -g100K > "${FPREFIX}2" 2> $FPREFIX-trash
-datagen -s3 -g200K > "${FPREFIX}3" 2> $FPREFIX-trash
-# compress multiple files using legacy format: one .lz4 per source file
-lz4 -f -l -m $FPREFIX*
-test -f ${FPREFIX}1.lz4
-test -f ${FPREFIX}2.lz4
-test -f ${FPREFIX}3.lz4
-# decompress multiple files compressed using legacy format: one output file per .lz4
-mv ${FPREFIX}1 ${FPREFIX}1-orig
-mv ${FPREFIX}2 ${FPREFIX}2-orig
-mv ${FPREFIX}3 ${FPREFIX}3-orig
-lz4 -d -f -m $FPREFIX*.lz4
-lz4 -l -d -f -m $FPREFIX*.lz4 # -l mustn't impact -d option
-cmp ${FPREFIX}1 ${FPREFIX}1-orig   # must be identical
-cmp ${FPREFIX}2 ${FPREFIX}2-orig
-cmp ${FPREFIX}3 ${FPREFIX}3-orig
-# compress multiple files into stdout using legacy format
-cat ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 > $FPREFIX-concat1
-rm $FPREFIX*.lz4
-lz4 -l -m ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3 -c > $FPREFIX-concat2
-test ! -f ${FPREFIX}1.lz4  # must not create .lz4 artefact
-cmp $FPREFIX-concat1 $FPREFIX-concat2  # must be equivalent
-# # # decompress multiple files into stdout using legacy format
-rm $FPREFIX-concat1 $FPREFIX-concat2
-lz4 -l -f -m ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3   # generate .lz4 to decompress
-cat ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3 > ${FPREFIX}-concat1   # create concatenated reference
-rm ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3
-lz4 -d -m ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 -c > $FPREFIX-concat2
-lz4 -d -l -m ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 -c > $FPREFIX-concat2 # -l mustn't impact option -d
-test ! -f ${FPREFIX}1  # must not create file artefact
-cmp $FPREFIX-concat1 $FPREFIX-concat2  # must be equivalent
-# # # compress multiple files, one of which is absent (must fail)
-lz4 -f -l -m $FPREFIX-concat1 notHere-legacy $FPREFIX-concat2 && exit 1 # must fail : notHere-legacy not present
-true
@@ -1,58 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-tml"
-
-set -e
-
-remove () {
-    rm $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-datagen -s1        > ${FPREFIX}1 2> $FPREFIX-trash
-datagen -s2 -g100K > ${FPREFIX}2 2> $FPREFIX-trash
-datagen -s3 -g200K > ${FPREFIX}3 2> $FPREFIX-trash
-# compress multiple files : one .lz4 per source file
-lz4 -f -m $FPREFIX*
-test -f ${FPREFIX}1.lz4
-test -f ${FPREFIX}2.lz4
-test -f ${FPREFIX}3.lz4
-# decompress multiple files : one output file per .lz4
-mv ${FPREFIX}1 ${FPREFIX}1-orig
-mv ${FPREFIX}2 ${FPREFIX}2-orig
-mv ${FPREFIX}3 ${FPREFIX}3-orig
-lz4 -d -f -m $FPREFIX*.lz4
-cmp ${FPREFIX}1 ${FPREFIX}1-orig   # must be identical
-cmp ${FPREFIX}2 ${FPREFIX}2-orig
-cmp ${FPREFIX}3 ${FPREFIX}3-orig
-# compress multiple files into stdout
-cat ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 > $FPREFIX-concat1
-rm $FPREFIX*.lz4
-lz4 -m ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3 -c > $FPREFIX-concat2
-test ! -f ${FPREFIX}1.lz4  # must not create .lz4 artefact
-cmp $FPREFIX-concat1 $FPREFIX-concat2  # must be equivalent
-# decompress multiple files into stdout
-rm $FPREFIX-concat1 $FPREFIX-concat2
-lz4 -f -m ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3   # generate .lz4 to decompress
-cat ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3 > $FPREFIX-concat1   # create concatenated reference
-rm ${FPREFIX}1 ${FPREFIX}2 ${FPREFIX}3
-lz4 -d -m ${FPREFIX}1.lz4 ${FPREFIX}2.lz4 ${FPREFIX}3.lz4 -c > $FPREFIX-concat2
-test ! -f ${FPREFIX}1  # must not create file artefact
-cmp $FPREFIX-concat1 $FPREFIX-concat2  # must be equivalent
-# compress multiple files, one of which is absent (must fail)
-lz4 -f -m $FPREFIX-concat1 notHere $FPREFIX-concat2 && exit 1 # must fail : notHere not present
-# test lz4-compressed file
-rm $FPREFIX-concat1 $FPREFIX-concat2
-lz4 -tm $FPREFIX-concat1.lz4
-# ensure the test doesn't create artifact
-test ! -f $FPREFIX-concat1  # must not create file artefact
-# test multiple lz4-compressed file
-lz4 -tm $FPREFIX-concat1.lz4 $FPREFIX-concat2.lz4
-test ! -f $FPREFIX-concat1  # must not create file artefact
-test ! -f $FPREFIX-concat2  # must not create file artefact
-# test multiple lz4 files, one of which is absent (must fail)
-lz4 -tm $FPREFIX-concat1.lz4 notHere.lz4 $FPREFIX-concat2.lz4 && exit 1
-true
@@ -1,16 +0,0 @@
-#!/bin/sh
-
-set -e
-set -x
-
-datagen -g16KB      | lz4 -12      | lz4 -t
-datagen -P10        | lz4 -12B4    | lz4 -t
-datagen -g256K      | lz4 -12B4D   | lz4 -t
-datagen -g512K -P25 | lz4 -12BD    | lz4 -t
-datagen -g1M        | lz4 -12B5    | lz4 -t
-datagen -g1M -s2    | lz4 -12B4D   | lz4 -t
-datagen -g2M -P99   | lz4 -11B4D   | lz4 -t
-datagen -g4M        | lz4 -11vq    | lz4 -qt
-datagen -g8M        | lz4 -11B4    | lz4 -t
-datagen -g16M -P90  | lz4 -11B5    | lz4 -t
-datagen -g32M -P10  | lz4 -11B5D   | lz4 -t
@@ -1,19 +0,0 @@
-#!/bin/sh
-
-SKIPFILE="goldenSamples/skip.bin"
-FPREFIX="tmp-lsk"
-
-set -e
-
-remove () {
-    rm "$FPREFIX"*
-}
-
-trap remove EXIT
-
-set -x
-
-lz4 -dc $SKIPFILE
-lz4 -dc < $SKIPFILE
-printf "Hello from Valid Frame!\n" | lz4 -c > $FPREFIX.lz4
-cat $SKIPFILE $FPREFIX.lz4 $SKIPFILE | lz4 -dc
@@ -1,42 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-tls"
-
-set -e
-
-remove () {
-    rm $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-datagen -g5M  -P100 > ${FPREFIX}dg5M
-lz4 -B4D ${FPREFIX}dg5M -c | lz4 -dv --sparse > ${FPREFIX}cB4
-diff -s ${FPREFIX}dg5M ${FPREFIX}cB4
-lz4 -B5D ${FPREFIX}dg5M -c | lz4 -dv --sparse > ${FPREFIX}cB5
-diff -s ${FPREFIX}dg5M ${FPREFIX}cB5
-lz4 -B6D ${FPREFIX}dg5M -c | lz4 -dv --sparse > ${FPREFIX}cB6
-diff -s ${FPREFIX}dg5M ${FPREFIX}cB6
-lz4 -B7D ${FPREFIX}dg5M -c | lz4 -dv --sparse > ${FPREFIX}cB7
-diff -s ${FPREFIX}dg5M ${FPREFIX}cB7
-lz4 ${FPREFIX}dg5M -c | lz4 -dv --no-sparse > ${FPREFIX}nosparse
-diff -s ${FPREFIX}dg5M ${FPREFIX}nosparse
-ls -ls $FPREFIX*
-datagen -s1 -g1200007 -P100 | lz4 | lz4 -dv --sparse > ${FPREFIX}odd   # Odd size file (to generate non-full last block)
-datagen -s1 -g1200007 -P100 | diff -s - ${FPREFIX}odd
-ls -ls ${FPREFIX}odd
-rm $FPREFIX*
-printf "\n Compatibility with Console :"
-echo "Hello World 1 !" | lz4 | lz4 -d -c
-echo "Hello World 2 !" | lz4 | lz4 -d | cat
-echo "Hello World 3 !" | lz4 --no-frame-crc | lz4 -d -c
-printf "\n Compatibility with Append :"
-datagen -P100 -g1M > ${FPREFIX}dg1M
-cat ${FPREFIX}dg1M ${FPREFIX}dg1M > ${FPREFIX}2M
-lz4 -B5 -v ${FPREFIX}dg1M ${FPREFIX}c
-lz4 -d -v ${FPREFIX}c ${FPREFIX}r
-lz4 -d -v ${FPREFIX}c -c >> ${FPREFIX}r
-ls -ls $FPREFIX*
-diff ${FPREFIX}2M ${FPREFIX}r
@@ -1,38 +0,0 @@
-#!/bin/sh
-
-FPREFIX="tmp-ltm"
-
-set -e
-
-remove () {
-    rm $FPREFIX*
-}
-
-trap remove EXIT
-
-set -x
-
-lz4 -bi0
-datagen > $FPREFIX
-lz4 -f $FPREFIX -c > $FPREFIX.lz4
-lz4 -bdi0 $FPREFIX.lz4 # test benchmark decode-only mode
-lz4 -bdi0 --no-crc $FPREFIX.lz4 # test benchmark decode-only mode
-echo "---- test mode ----"
-datagen | lz4 -t && exit 1
-datagen | lz4 -tf && exit 1
-echo "---- pass-through mode ----"
-echo "Why hello there " > ${FPREFIX}2.lz4
-lz4 -f ${FPREFIX}2.lz4 > $FPREFIX-trash && exit 1
-datagen | lz4 -dc  > $FPREFIX-trash && exit 1
-datagen | lz4 -df > $FPREFIX-trash && exit 1
-datagen | lz4 -dcf > $FPREFIX-trash
-echo "Hello World !" > ${FPREFIX}1
-lz4 -dcf ${FPREFIX}1
-echo "from underground..." > ${FPREFIX}2
-lz4 -dcfm ${FPREFIX}1 ${FPREFIX}2
-echo "---- non-existing source (must fail cleanly) ----"
-lz4     file-does-not-exist && exit 1
-lz4 -f  file-does-not-exist && exit 1
-lz4 -t  file-does-not-exist && exit 1
-lz4 -fm file1-dne file2-dne && exit 1
-true
@@ -1,6 +0,0 @@
-#!/bin/sh
-
-set -e
-set -x
-
-datagen -g4200MB | lz4 -v3BD | lz4 -qt
Author	SHA1	Message	Date
Yann Collet	5ff8396801	Merge pull request #1138 from lz4/dev stage v1.9.4	2022-08-15 15:45:31 -07:00
Yann Collet	416bc96fac	Merge branch 'dev' into release	2020-11-17 19:42:24 -08:00
Yann Collet	d44371841a	Merge branch 'dev' into release	2020-11-15 20:59:35 -08:00
Yann Collet	d3972cfa7e	Merge pull request #954 from lz4/dev v1.9.3	2020-11-15 15:53:42 -08:00