fixed incorrect assert()

Incredibly evaded detection so far, probably because -size_t is still perfectly defined size_t which seems good enough for gcc and clang, but Visual finds that rather fishy.
Merge pull request #2244 from hctim/dev
2020-07-15 12:43:11 -07:00 · 2020-07-15 12:40:17 -07:00 · 2020-07-14 09:03:02 -07:00 · 2020-07-13 14:26:13 -04:00 · 2020-07-13 01:20:11 +00:00 · 2020-07-11 02:16:52 -07:00
248 changed files with 11508 additions and 7191 deletions
@@ -12,7 +12,8 @@ jobs:
      - run:
          name: Test
          command: |
-            cc -v; CFLAGS="-O0 -Werror" make all && make clean
+            ./tests/test-license.py
+            cc -v; CFLAGS="-O0 -Werror -pedantic" make all && make clean
            make c99build         ; make clean
            make c11build         ; make clean
            make aarch64build     ; make clean
@@ -33,7 +34,7 @@ jobs:
            make ppc64build; make clean
            make ppcbuild  ; make clean
            make armbuild  ; make clean
-            make -C tests test-legacy test-longmatch test-symbols; make clean
+            make -C tests test-legacy test-longmatch; make clean
            make -C lib libzstd-nomt; make clean
  # This step is only run on release tags.
  # It publishes the source tarball as artifacts and if the GITHUB_TOKEN
@@ -1,16 +1,11 @@
-env:
-  CIRRUS_CLONE_DEPTH: 1
-  ARCH: amd64
-
 task:
+  name: FreeBSD (shortest)
  freebsd_instance:
    matrix:
-      image: freebsd-12-0-release-amd64
-      image: freebsd-11-2-release-amd64
-  install_script:
-    - sed -i.bak -e 's,pkg+http://pkg.FreeBSD.org/\${ABI}/quarterly,pkg+http://pkg.FreeBSD.org/\${ABI}/latest,' /etc/pkg/FreeBSD.conf
-    - pkg upgrade -y
-    - pkg install -y gmake coreutils
+      image_family: freebsd-12-1
+      # The stable 11.3 image causes "Agent is not responding" so use a snapshot
+      image_family: freebsd-11-3-snap
+  install_script: pkg install -y gmake coreutils
  script: |
    MOREFLAGS="-Werror" gmake -j all
    gmake shortest
@@ -0,0 +1,35 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Downloads data '...'
+2. Run '...' with flags '...'
+3. Scroll up on the log to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots and charts**
+If applicable, add screenshots and charts to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. Mac]
+ - Version [e.g. 22]
+ - Compiler [e.g gcc]
+ - Flags [e.g O2]
+ - Other relevant hardware specs [e.g. Dual-core]
+ - Build system [e.g. Makefile]
+
+**Additional context**
+Add any other context about the problem here.
@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
@@ -0,0 +1,184 @@
+name: generic-dev
+
+on:
+  pull_request:
+    branches: [ dev, master, actionsTest ]
+
+jobs:
+
+# Dev PR jobs that still have to be migrated from travis
+#
+# icc (need self-hosted)
+# versionTag
+# valgrindTest (keeps failing for some reason. need investigation)
+# staticAnalyze (need trusty so need self-hosted)
+# pcc-fuzz: (need trusty so need self-hosted)
+# min-decomp-macros (flakey)
+#
+# setting up self-hosted is pretty straightforward, but
+# I need admins permissions to the repo for that it looks like
+# So I'm tabling that for now
+#
+# The master branch exclusive jobs will be in a separate
+# workflow file (the osx tests and meson build that is)
+
+  benchmarking:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: make benchmarking
+      run: make benchmarking
+
+  test:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: make test
+      run: make test
+
+  gcc-6-7-libzstd:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: gcc-6 + gcc-7 + libzstdmt compilation
+      run: |
+        make gcc6install gcc7install
+        CC=gcc-6 CFLAGS=-Werror make -j all
+        make clean
+        CC=gcc-7 CFLAGS=-Werror make -j all
+        make clean
+        LDFLAGS=-Wl,--no-undefined make -C lib libzstd-mt
+        make -C tests zbufftest-dll
+
+  gcc-8-asan-ubsan-testzstd:
+    runs-on: ubuntu-16.04 # fails on 18.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: gcc-8 + ASan + UBSan + Test Zstd
+      run: |
+        make gcc8install
+        CC=gcc-8 CFLAGS="-Werror" make -j all
+        make clean
+        CC=gcc-8 make -j uasan-test-zstd </dev/null
+
+  gcc-6-asan-ubsan-testzstd-32bit:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: gcc-6 + ASan + UBSan + Test Zstd, 32bit mode
+      run: |
+        make gcc6install libc6install
+        CC=gcc-6 CFLAGS="-Werror -m32" make -j all32
+        make clean
+        CC=gcc-6 make -j uasan-test-zstd32
+
+  clang-38-msan-testzstd:
+    runs-on: ubuntu-16.04 # fails on 18.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: clang-3.8 + MSan + Test Zstd
+      run: |
+        # make clang38install (doesn't work)
+        sudo apt-add-repository "deb http://llvm.org/apt/trusty/ llvm-toolchain-trusty-3.8 main"
+        sudo apt-get update
+        sudo apt-get install clang-3.8
+        CC=clang-3.8 make clean msan-test-zstd HAVE_ZLIB=0 HAVE_LZ4=0 HAVE_LZMA=0
+
+  cmake-build-and-test-check:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: cmake build and test check
+      run: make cmakebuild
+
+  gcc-8-asan-ubsan-fuzz:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: gcc-8 + ASan + UBSan + Fuzz Test
+      run: |
+        make gcc8install
+        CC=gcc-8 make clean uasan-fuzztest
+
+  gcc-6-asan-ubsan-fuzz32:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: gcc-6 + ASan + UBSan + Fuzz Test 32bit
+      run: |
+        make gcc6install libc6install
+        CC=gcc-6 CFLAGS="-O2 -m32" make uasan-fuzztest
+
+  clang-38-msan-fuzz:
+    runs-on: ubuntu-16.04 # fails on 18.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: clang-3.8 + MSan + Fuzz Test
+      run: |
+        # make clang38install (doesn't work)
+        sudo apt-add-repository "deb http://llvm.org/apt/trusty/ llvm-toolchain-trusty-3.8 main"
+        sudo apt-get update
+        sudo apt-get install clang-3.8
+        CC=clang-3.8 make clean msan-fuzztest
+
+  asan-ubsan-msan-regression:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: ASan + UBSan + MSan + Regression Test
+      run: |
+        make -j uasanregressiontest
+        make clean
+        make -j msanregressiontest
+
+  cpp-gnu90-c99-compatibility:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: C++, gnu90 and c99 compatibility
+      run: |
+        make cxxtest
+        make clean
+        make gnu90build
+        make clean
+        make c99build
+        make clean
+        make travis-install
+
+  mingw-cross-compilation:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: mingw cross-compilation
+      run: |
+        # sudo update-alternatives --set x86_64-w64-mingw32-g++ /usr/bin/x86_64-w64-mingw32-g++-posix; (doesn't work)
+        sudo apt-get install gcc-mingw-w64
+        CC=x86_64-w64-mingw32-gcc CXX=x86_64-w64-mingw32-g++ CFLAGS="-Werror -O1" make zstd
+
+  armbuild:
+    runs-on: ubuntu-16.04 # doesn't work on latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: ARM Build Test
+      run: |
+        make arminstall
+        make armbuild
+
+  armfuzz:
+    runs-on: ubuntu-16.04 # doesn't work on latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Qemu ARM emulation + Fuzz Test
+      run: |
+        make arminstall
+        make armfuzz
+
+  bourne-shell:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2
+    - name: Bourne shell compatibility (shellcheck)
+      run: |
+        wget https://github.com/koalaman/shellcheck/releases/download/v0.7.1/shellcheck-v0.7.1.linux.x86_64.tar.xz
+        tar -xf shellcheck-v0.7.1.linux.x86_64.tar.xz
+        shellcheck-v0.7.1/shellcheck --shell=sh --severity=warning --exclude=SC2010 tests/playTests.sh
@@ -0,0 +1,68 @@
+name: generic-release
+
+on:
+  pull_request:
+    # This will eventually only be for pushes to master
+    # but for dogfooding purposes, I'm running it even 
+    # on dev pushes
+    branches: [ dev, master, actionsTest ]
+
+jobs:
+  # missing jobs
+  # 
+  # ppc64le + fuzz test
+  # Qemu PPC64 + Fuzz test
+  # Qemu aarch64 + Fuzz Test (on Xenial)
+  # versions comp   
+  # meson test
+
+  osx:
+    runs-on: macos-10.15
+    steps:
+    - uses: actions/checkout@v2
+    - name: OS-X
+      run: |
+        make test
+        # make -c lib all (need to fix. not working right now)
+  
+  zbuff:
+    runs-on: ubuntu-16.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: zbuff test
+      run: |
+        make -C tests test-zbuff
+        
+  tsan:
+    runs-on: ubuntu-16.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: thread sanitizer
+      run: |
+        sudo apt-add-repository "deb http://llvm.org/apt/trusty/ llvm-toolchain-trusty-3.8 main"
+        sudo apt-get update
+        sudo apt-get install clang-3.8
+        CC=clang-3.8 make tsan-test-zstream
+        CC=clang-3.8 make tsan-fuzztest
+  zlib-wrapper:
+    runs-on: ubuntu-16.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: zlib wrapper test
+      run: |
+        make gpp6install valgrindinstall
+        make -C zlibWrapper test
+        make -C zlibWrapper valgrindTest
+  
+  lz4-threadpool-partial-libs:
+    runs-on: ubuntu-16.04
+    steps:
+    - uses: actions/checkout@v2
+    - name: LZ4, thread pool, and partial libs testslib wrapper test
+      run: |
+        make lz4install
+        make -C tests test-lz4
+        make clean
+        make -C tests test-pool
+        make clean
+        bash tests/libzstd_partial_builds.sh
@@ -0,0 +1,30 @@
+name: CIFuzz
+on: [pull_request]
+jobs:
+  Fuzzing:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        sanitizer: [address, undefined, memory]
+    steps:
+    - name: Build Fuzzers (${{ matrix.sanitizer }})
+      id: build
+      uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
+      with:
+        oss-fuzz-project-name: 'zstd'
+        dry-run: false
+        sanitizer: ${{ matrix.sanitizer }}
+    - name: Run Fuzzers (${{ matrix.sanitizer }})
+      uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
+      with:
+        oss-fuzz-project-name: 'zstd'
+        fuzz-seconds: 600
+        dry-run: false
+        sanitizer: ${{ matrix.sanitizer }}
+    - name: Upload Crash
+      uses: actions/upload-artifact@v1
+      if: failure() && steps.build.outcome == 'success'
+      with:
+        name: ${{ matrix.sanitizer }}-artifacts
+        path: ./out/artifacts
@@ -22,6 +22,7 @@ zstdmt

 # Test artefacts
 tmp*
+*.zst
 dictionary.
 dictionary
 NUL
@@ -32,6 +33,7 @@ bin/
 .buckd/
 buck-out/
 build-*
+*.gcda

 # Other files
 .directory
@@ -25,6 +25,22 @@ env:
 matrix:
  fast_finish: true
  include:
+    - name: icc
+      compiler: icc
+      env:
+      - C_COMPILER=icc
+      - CXX_COMPILER=icpc
+      install:
+      - source /opt/intel/inteloneapi/compiler/latest/env/vars.sh
+      addons:
+        apt:
+          sources:
+          - sourceline: 'deb https://apt.repos.intel.com/oneapi all main'
+            key_url: 'https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2023.PUB'
+          packages:
+          - intel-oneapi-icc
+      script: make check
+
    - name: arm64    # ~2.5 mn
      os: linux
      arch: arm64
@@ -40,12 +56,17 @@ matrix:
        # DEVNULLRIGHTS : will request sudo rights to test permissions on /dev/null
        - DEVNULLRIGHTS=test make test

-    - name: gcc-6 + gcc-7 compilation   # ~ 3mn
+    - name: gcc-6 + gcc-7 + libzstdmt compilation   # ~ 6mn
      script:
        - make gcc6install gcc7install
        - CC=gcc-6 CFLAGS=-Werror make -j all
        - make clean
        - CC=gcc-7 CFLAGS=-Werror make -j all
+        - make clean
+        - LDFLAGS=-Wl,--no-undefined make -C lib libzstd-mt
+        - make -C tests zbufftest-dll
+        # LDFLAGS=-Wl,--no-undefined : will make the linker fail if dll is underlinked
+        # zbufftest-dll : test that a user program can link to multi-threaded libzstd without specifying -pthread

    - name: gcc-8 + ASan + UBSan + Test Zstd   # ~6.5mn
      script:
@@ -72,19 +93,15 @@ matrix:
    - name: Minimal Decompressor Macros    # ~5mn
      script:
        - make clean
-        - CFLAGS=-Werror make -j all MOREFLAGS="-Werror -DHUF_FORCE_DECOMPRESS_X1 -DZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT"
+        - make -j all check ZSTD_LIB_MINIFY=1 MOREFLAGS="-Werror"
        - make clean
-        - make -j check MOREFLAGS="-Werror -DHUF_FORCE_DECOMPRESS_X1 -DZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT"
+        - make -j all check MOREFLAGS="-Werror -DHUF_FORCE_DECOMPRESS_X1 -DZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT"
        - make clean
-        - CFLAGS=-Werror make -j all MOREFLAGS="-Werror -DHUF_FORCE_DECOMPRESS_X2 -DZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG"
+        - make -j all check MOREFLAGS="-Werror -DHUF_FORCE_DECOMPRESS_X2 -DZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG"
        - make clean
-        - make -j check MOREFLAGS="-Werror -DHUF_FORCE_DECOMPRESS_X2 -DZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG"
-        - make clean
-        - CFLAGS=-Werror make -j all MOREFLAGS="-Werror -DZSTD_NO_INLINE -DZSTD_STRIP_ERROR_STRINGS"
-        - make clean
-        - make -j check MOREFLAGS="-Werror -DZSTD_NO_INLINE -DZSTD_STRIP_ERROR_STRINGS"
+        - make -j all check MOREFLAGS="-Werror -DZSTD_NO_INLINE -DZSTD_STRIP_ERROR_STRINGS"

-    - name: cmake build and test check    # ~2mn
+    - name: cmake build and test check    # ~6mn
      script:
        - make cmakebuild

@@ -167,6 +184,12 @@ matrix:
      os: osx
      script:
        - make test
+        - make -C lib all
+
+    - name: zbuff test
+      if: branch = master
+      script:
+        - make -C tests test-zbuff

    - name: Versions Compatibility Test   # 11.5mn
      if: branch = master
@@ -180,12 +203,6 @@ matrix:
        - CC=clang-3.8 make tsan-test-zstream
        - CC=clang-3.8 make tsan-fuzztest

-    - name: Qemu PPC64 + Fuzz test  # ~13mn
-      if: branch = master
-      script:
-        - make ppcinstall
-        - make ppc64fuzz
-
    - name: PPC64LE + Fuzz test  # ~13mn
      if: branch = master
      arch: ppc64le
@@ -193,6 +210,13 @@ matrix:
        - cat /proc/cpuinfo
        - make test

+    - name: Qemu PPC64 + Fuzz test  # ~13mn, presumed Big-Endian (?)
+      dist: trusty  # note : PPC64 cross-compilation for Qemu tests seems broken on Xenial
+      if: branch = master
+      script:
+        - make ppcinstall
+        - make ppc64fuzz
+
    # note : we already have aarch64 tests on hardware
    - name: Qemu aarch64 + Fuzz Test (on Xenial)    # ~14mn
      if: branch = master
@@ -219,7 +243,7 @@ matrix:
        - bash tests/libzstd_partial_builds.sh

    # meson dedicated test
-    - name: Xenial (Meson + clang)    # ~8mn
+    - name: Xenial (Meson + clang)    # ~15mn
      if: branch = master
      dist: xenial
      language: cpp
@@ -1,9 +1,28 @@
 v1.4.5
-perf: Improved decompression speed, +10% on clang, +5% on gcc, by @terrelln
-cli: new command : provide list of files to de/compress within a file with --filelist=FILE, by @AhmedAbdellah19
-api: low latency blocks, new experimental parameter by @ephiepark
-build: mingw cross-compilation from unix, by @Ericson2314
+fix : Compression ratio regression on huge files (> 3 GB) using high levels (--ultra) and multithreading, by @terrelln
+perf: Improved decompression speed: x64 : +10% (clang) / +5% (gcc); ARM : from +15% to +50%, depending on SoC, by @terrelln
+perf: Automatically downsizes ZSTD_DCtx when too large for too long (#2069, by @bimbashreshta)
+perf: Improved fast compression speed on aarch64 (#2040, ~+3%, by @caoyzh)
+perf: Small level 1 compression speed gains (depending on compiler)
+cli : New --patch-from command, create and apply patches from files, by @bimbashreshta
+cli : New --filelist= : Provide a list of files to operate upon from a file
+cli : -b -d command can now benchmark decompression on multiple files
+cli : New --no-content-size command
+cli : New --show-default-cparams information command
+api : ZDICT_finalizeDictionary() is promoted to stable (#2111)
+api : new experimental parameter ZSTD_d_stableOutBuffer (#2094)
+build: Generate a single-file libzstd library (#2065, by @cwoffenden)
+build: Relative includes no longer require -I compiler flags for zstd lib subdirs (#2103, by @felixhandte)
+build: zstd now compiles cleanly under -pedantic (#2099)
+build: zstd now compiles with make-4.3
+build: Support mingw cross-compilation from Linux, by @Ericson2314
+build: Meson multi-thread build fix on windows
+build: Some misc icc fixes backed by new ci test on travis
 misc: bitflip analyzer tool, by @felixhandte
+misc: Extend largeNbDicts benchmark to compression
+misc: Edit-distance match finder in contrib/
+doc : Improved beginner CONTRIBUTING.md docs
+doc : New issue templates for zstd

 v1.4.4
 perf: Improved decompression speed, by > 10%, by @terrelln
@@ -26,6 +26,370 @@ to do this once to work on any of Facebook's open source projects.

 Complete your CLA here: <https://code.facebook.com/cla>

+## Workflow
+Zstd uses a branch-based workflow for making changes to the codebase. Typically, zstd
+will use a new branch per sizable topic. For smaller changes, it is okay to lump multiple
+related changes into a branch.
+
+Our contribution process works in three main stages:
+1. Local development
+    * Update:
+        * Checkout your fork of zstd if you have not already
+        ```
+        git checkout https://github.com/<username>/zstd
+        cd zstd
+        ```
+        * Update your local dev branch
+        ```
+        git pull https://github.com/facebook/zstd dev
+        git push origin dev
+        ```
+    * Topic and development:
+        * Make a new branch on your fork about the topic you're developing for
+        ```
+        # branch names should be consise but sufficiently informative
+        git checkout -b <branch-name>
+        git push origin <branch-name>
+        ```
+        * Make commits and push
+        ```
+        # make some changes =
+        git add -u && git commit -m <message>
+        git push origin <branch-name>
+        ```
+        * Note: run local tests to ensure that your changes didn't break existing functionality
+            * Quick check
+            ```
+            make shortest
+            ```
+            * Longer check
+            ```
+            make test
+            ```
+2. Code Review and CI tests
+    * Ensure CI tests pass:
+        * Before sharing anything to the community, make sure that all CI tests pass on your local fork.
+        See our section on setting up your CI environment for more information on how to do this.
+        * Ensure that static analysis passes on your development machine. See the Static Analysis section
+        below to see how to do this.
+    * Create a pull request:
+        * When you are ready to share you changes to the community, create a pull request from your branch
+        to facebook:dev. You can do this very easily by clicking 'Create Pull Request' on your fork's home
+        page.
+        * From there, select the branch where you made changes as your source branch and facebook:dev
+        as the destination.
+        * Examine the diff presented between the two branches to make sure there is nothing unexpected.
+    * Write a good pull request description:
+        * While there is no strict template that our contributors follow, we would like them to
+        sufficiently summarize and motivate the changes they are proposing. We recommend all pull requests,
+        at least indirectly, address the following points.
+            * Is this pull request important and why?
+            * Is it addressing an issue? If so, what issue? (provide links for convenience please)
+            * Is this a new feature? If so, why is it useful and/or necessary?
+            * Are there background references and documents that reviewers should be aware of to properly assess this change?
+        * Note: make sure to point out any design and architectural decisions that you made and the rationale behind them.
+        * Note: if you have been working with a specific user and would like them to review your work, make sure you mention them using (@<username>)
+    * Submit the pull request and iterate with feedback.
+3. Merge and Release
+    * Getting approval:
+        * You will have to iterate on your changes with feedback from other collaborators to reach a point
+        where your pull request can be safely merged.
+        * To avoid too many comments on style and convention, make sure that you have a
+        look at our style section below before creating a pull request.
+        * Eventually, someone from the zstd team will approve your pull request and not long after merge it into
+        the dev branch.
+    * Housekeeping:
+        * Most PRs are linked with one or more Github issues. If this is the case for your PR, make sure
+        the corresponding issue is mentioned. If your change 'fixes' or completely addresses the
+        issue at hand, then please indicate this by requesting that an issue be closed by commenting.
+        * Just because your changes have been merged does not mean the topic or larger issue is complete. Remember
+        that the change must make it to an official zstd release for it to be meaningful. We recommend
+        that contributers track the activity on their pull request and corresponding issue(s) page(s) until
+        their change makes it to the next release of zstd. Users will often discover bugs in your code or
+        suggest ways to refine and improve your initial changes even after the pull request is merged.
+
+## Static Analysis
+Static analysis is a process for examining the correctness or validity of a program without actually
+executing it. It usually helps us find many simple bugs. Zstd uses clang's `scan-build` tool for
+static analysis. You can install it by following the instructions for your OS on https://clang-analyzer.llvm.org/scan-build.
+
+Once installed, you can ensure that our static analysis tests pass on your local development machine
+by running:
+```
+make staticAnalyze
+```
+
+In general, you can use `scan-build` to static analyze any build script. For example, to static analyze
+just `contrib/largeNbDicts` and nothing else, you can run:
+
+```
+scan-build make -C contrib/largeNbDicts largeNbDicts
+```
+
+### Pitfalls of static analysis
+`scan-build` is part of our regular CI suite. Other static analyzers are not.
+
+It can be useful to look at additional static analyzers once in a while (and we do), but it's not a good idea to multiply the nb of analyzers run continuously at each commit and PR. The reasons are :
+
+- Static analyzers are full of false positive. The signal to noise ratio is actually pretty low.
+- A good CI policy is "zero-warning tolerance". That means that all issues must be solved, including false positives. This quickly becomes a tedious workload.
+- Multiple static analyzers will feature multiple kind of false positives, sometimes applying to the same code but in different ways leading to :
+   + torteous code, trying to please multiple constraints, hurting readability and therefore maintenance. Sometimes, such complexity introduce other more subtle bugs, that are just out of scope of the analyzers.
+   + sometimes, these constraints are mutually exclusive : if one try to solve one, the other static analyzer will complain, they can't be both happy at the same time.
+- As if that was not enough, the list of false positives change with each version. It's hard enough to follow one static analyzer, but multiple ones with their own update agenda, this quickly becomes a massive velocity reducer.
+
+This is different from running a static analyzer once in a while, looking at the output, and __cherry picking__ a few warnings that seem helpful, either because they detected a genuine risk of bug, or because it helps expressing the code in a way which is more readable or more difficult to misuse. These kind of reports can be useful, and are accepted.
+
+## Performance
+Performance is extremely important for zstd and we only merge pull requests whose performance
+landscape and corresponding trade-offs have been adequately analyzed, reproduced, and presented.
+This high bar for performance means that every PR which has the potential to
+impact performance takes a very long time for us to properly review. That being said, we
+always welcome contributions to improve performance (or worsen performance for the trade-off of
+something else). Please keep the following in mind before submitting a performance related PR:
+
+1. Zstd isn't as old as gzip but it has been around for time now and its evolution is
+very well documented via past Github issues and pull requests. It may be the case that your
+particular performance optimization has already been considered in the past. Please take some
+time to search through old issues and pull requests using keywords specific to your
+would-be PR. Of course, just because a topic has already been discussed (and perhaps rejected
+on some grounds) in the past, doesn't mean it isn't worth bringing up again. But even in that case,
+it will be helpful for you to have context from that topic's history before contributing.
+2. The distinction between noise and actual performance gains can unfortunately be very subtle
+especially when microbenchmarking extremely small wins or losses. The only remedy to getting
+something subtle merged is extensive benchmarking. You will be doing us a great favor if you
+take the time to run extensive, long-duration, and potentially cross-(os, platform, process, etc)
+benchmarks on your end before submitting a PR. Of course, you will not be able to benchmark
+your changes on every single processor and os out there (and neither will we) but do that best
+you can:) We've adding some things to think about when benchmarking below in the Benchmarking
+Performance section which might be helpful for you.
+3. Optimizing performance for a certain OS, processor vendor, compiler, or network system is a perfectly
+legitimate thing to do as long as it does not harm the overall performance health of Zstd.
+This is a hard balance to strike but please keep in mind other aspects of Zstd when
+submitting changes that are clang-specific, windows-specific, etc.
+
+## Benchmarking Performance
+Performance microbenchmarking is a tricky subject but also essential for Zstd. We value empirical
+testing over theoretical speculation. This guide it not perfect but for most scenarios, it
+is a good place to start.
+
+### Stability
+Unfortunately, the most important aspect in being able to benchmark reliably is to have a stable
+benchmarking machine. A virtual machine, a machine with shared resources, or your laptop
+will typically not be stable enough to obtain reliable benchmark results. If you can get your
+hands on a desktop, this is usually a better scenario.
+
+Of course, benchmarking can be done on non-hyper-stable machines as well. You will just have to
+do a little more work to ensure that you are in fact measuring the changes you've made not and
+noise. Here are some things you can do to make your benchmarks more stable:
+
+1. The most simple thing you can do to drastically improve the stability of your benchmark is
+to run it multiple times and then aggregate the results of those runs. As a general rule of
+thumb, the smaller the change you are trying to measure, the more samples of benchmark runs
+you will have to aggregate over to get reliable results. Here are some additional things to keep in
+mind when running multiple trials:
+    * How you aggregate your samples are important. You might be tempted to use the mean of your
+    results. While this is certainly going to be a more stable number than a raw single sample
+    benchmark number, you might have more luck by taking the median. The mean is not robust to
+    outliers whereas the median is. Better still, you could simply take the fastest speed your
+    benchmark achieved on each run since that is likely the fastest your process will be
+    capable of running your code. In our experience, this (aggregating by just taking the sample
+    with the fastest running time) has been the most stable approach.
+    * The more samples you have, the more stable your benchmarks should be. You can verify
+    your improved stability by looking at the size of your confidence intervals as you
+    increase your sample count. These should get smaller and smaller. Eventually hopefully
+    smaller than the performance win you are expecting.
+    * Most processors will take some time to get `hot` when running anything. The observations
+    you collect during that time period will very different from the true performance number. Having
+    a very large number of sample will help alleviate this problem slightly but you can also
+    address is directly by simply not including the first `n` iterations of your benchmark in
+    your aggregations. You can determine `n` by simply looking at the results from each iteration
+    and then hand picking a good threshold after which the variance in results seems to stabilize.
+2. You cannot really get reliable benchmarks if your host machine is simultaneously running
+another cpu/memory-intensive application in the background. If you are running benchmarks on your
+personal laptop for instance, you should close all applications (including your code editor and
+browser) before running your benchmarks. You might also have invisible background applications
+running. You can see what these are by looking at either Activity Monitor on Mac or Task Manager
+on Windows. You will get more stable benchmark results of you end those processes as well.
+    * If you have multiple cores, you can even run your benchmark on a reserved core to prevent
+    pollution from other OS and user processes. There are a number of ways to do this depending
+    on your OS:
+        * On linux boxes, you have use https://github.com/lpechacek/cpuset.
+        * On Windows, you can "Set Processor Affinity" using https://www.thewindowsclub.com/processor-affinity-windows
+        * On Mac, you can try to use their dedicated affinity API https://developer.apple.com/library/archive/releasenotes/Performance/RN-AffinityAPI/#//apple_ref/doc/uid/TP40006635-CH1-DontLinkElementID_2
+3. To benchmark, you will likely end up writing a separate c/c++ program that will link libzstd.
+Dynamically linking your library will introduce some added variation (not a large amount but
+definitely some). Statically linking libzstd will be more stable. Static libraries should
+be enabled by default when building zstd.
+4. Use a profiler with a good high resolution timer. See the section below on profiling for
+details on this.
+5. Disable frequency scaling, turbo boost and address space randomization (this will vary by OS)
+6. Try to avoid storage. On some systems you can use tmpfs. Putting the program, inputs and outputs on
+tmpfs avoids touching a real storage system, which can have a pretty big variability.
+
+Also check our LLVM's guide on benchmarking here: https://llvm.org/docs/Benchmarking.html
+
+### Zstd benchmark
+The fastest signal you can get regarding your performance changes is via the in-build zstd cli
+bench option. You can run Zstd as you typically would for your scenario using some set of options
+and then additionally also specify the `-b#` option. Doing this will run our benchmarking pipeline
+for that options you have just provided. If you want to look at the internals of how this
+benchmarking script works, you can check out programs/benchzstd.c
+
+For example: say you have made a change that you believe improves the speed of zstd level 1. The
+very first thing you should use to asses whether you actually achieved any sort of improvement
+is `zstd -b`. You might try to do something like this. Note: you can use the `-i` option to
+specify a running time for your benchmark in seconds (default is 3 seconds).
+Usually, the longer the running time, the more stable your results will be.
+
+```
+$ git checkout <commit-before-your-change>
+$ make && cp zstd zstd-old
+$ git checkout <commit-after-your-change>
+$ make && cp zstd zstd-new
+$ zstd-old -i5 -b1 <your-test-data>
+ 1<your-test-data>         :      8990 ->      3992 (2.252), 302.6 MB/s , 626.4 MB/s
+$ zstd-new -i5 -b1 <your-test-data>
+ 1<your-test-data>         :      8990 ->      3992 (2.252), 302.8 MB/s , 628.4 MB/s
+```
+
+Unless your performance win is large enough to be visible despite the intrinsic noise
+on your computer, benchzstd alone will likely not be enough to validate the impact of your
+changes. For example, the results of the example above indicate that effectively nothing
+changed but there could be a small <3% improvement that the noise on the host machine
+obscured. So unless you see a large performance win (10-15% consistently) using just
+this method of evaluation will not be sufficient.
+
+### Profiling
+There are a number of great profilers out there. We're going to briefly mention how you can
+profile your code using `instruments` on mac, `perf` on linux and `visual studio profiler`
+on windows.
+
+Say you have an idea for a change that you think will provide some good performance gains
+for level 1 compression on Zstd. Typically this means, you have identified a section of
+code that you think can be made to run faster.
+
+The first thing you will want to do is make sure that the piece of code is actually taking up
+a notable amount of time to run. It is usually not worth optimzing something which accounts for less than
+0.0001% of the total running time. Luckily, there are tools to help with this.
+Profilers will let you see how much time your code spends inside a particular function.
+If your target code snippit is only part of a function, it might be worth trying to
+isolate that snippit by moving it to its own function (this is usually not necessary but
+might be).
+
+Most profilers (including the profilers dicusssed below) will generate a call graph of
+functions for you. Your goal will be to find your function of interest in this call grapch
+and then inspect the time spent inside of it. You might also want to to look at the
+annotated assembly which most profilers will provide you with.
+
+#### Instruments
+We will once again consider the scenario where you think you've identified a piece of code
+whose performance can be improved upon. Follow these steps to profile your code using
+Instruments.
+
+1. Open Instruments
+2. Select `Time Profiler` from the list of standard templates
+3. Close all other applications except for your instruments window and your terminal
+4. Run your benchmarking script from your terminal window
+    * You will want a benchmark that runs for at least a few seconds (5 seconds will
+    usually be long enough). This way the profiler will have something to work with
+    and you will have ample time to attach your profiler to this process:)
+    * I will just use benchzstd as my bencharmking script for this example:
+```
+$ zstd -b1 -i5 <my-data> # this will run for 5 seconds
+```
+5. Once you run your benchmarking script, switch back over to instruments and attach your
+process to the time profiler. You can do this by:
+    * Clicking on the `All Processes` drop down in the top left of the toolbar.
+    * Selecting your process from the dropdown. In my case, it is just going to be labled
+    `zstd`
+    * Hitting the bright red record circle button on the top left of the toolbar
+6. You profiler will now start collecting metrics from your bencharking script. Once
+you think you have collected enough samples (usually this is the case after 3 seconds of
+recording), stop your profiler.
+7. Make sure that in toolbar of the bottom window, `profile` is selected.
+8. You should be able to see your call graph.
+    * If you don't see the call graph or an incomplete call graph, make sure you have compiled
+    zstd and your benchmarking scripg using debug flags. On mac and linux, this just means
+    you will have to supply the `-g` flag alone with your build script. You might also
+    have to provide the `-fno-omit-frame-pointer` flag
+9. Dig down the graph to find your function call and then inspect it by double clicking
+the list item. You will be able to see the annotated source code and the assembly side by
+side.
+
+#### Perf
+
+This wiki has a pretty detailed tutorial on getting started working with perf so we'll
+leave you to check that out of you're getting started:
+
+https://perf.wiki.kernel.org/index.php/Tutorial
+
+Some general notes on perf:
+* Use `perf stat -r # <bench-program>` to quickly get some relevant timing and
+counter statistics. Perf uses a high resolution timer and this is likely one
+of the first things your team will run when assessing your PR.
+* Perf has a long list of hardware counters that can be viewed with `perf --list`.
+When measuring optimizations, something worth trying is to make sure the handware
+counters you expect to be impacted by your change are in fact being so. For example,
+if you expect the L1 cache misses to decrease with your change, you can look at the
+counter `L1-dcache-load-misses`
+* Perf hardware counters will not work on a virtual machine.
+
+#### Visual Studio
+
+TODO
+
+
+## Setting up continuous integration (CI) on your fork
+Zstd uses a number of different continuous integration (CI) tools to ensure that new changes
+are well tested before they make it to an official release. Specifically, we use the platforms
+travis-ci, circle-ci, and appveyor.
+
+Changes cannot be merged into the main dev branch unless they pass all of our CI tests.
+The easiest way to run these CI tests on your own before submitting a PR to our dev branch
+is to configure your personal fork of zstd with each of the CI platforms. Below, you'll find
+instructions for doing this.
+
+### travis-ci
+Follow these steps to link travis-ci with your github fork of zstd
+
+1. Make sure you are logged into your github account
+2. Go to https://travis-ci.org/
+3. Click 'Sign in with Github' on the top right
+4. Click 'Authorize travis-ci'
+5. Click 'Activate all repositories using Github Apps'
+6. Select 'Only select repositories' and select your fork of zstd from the drop down
+7. Click 'Approve and Install'
+8. Click 'Sign in with Github' again. This time, it will be for travis-pro (which will let you view your tests on the web dashboard)
+9. Click 'Authorize travis-pro'
+10. You should have travis set up on your fork now.
+
+### circle-ci
+TODO
+
+### appveyor
+Follow these steps to link circle-ci with your girhub fork of zstd
+
+1. Make sure you are logged into your github account
+2. Go to https://www.appveyor.com/
+3. Click 'Sign in' on the top right
+4. Select 'Github' on the left panel
+5. Click 'Authorize appveyor'
+6. You might be asked to select which repositories you want to give appveyor permission to. Select your fork of zstd if you're prompted
+7. You should have appveyor set up on your fork now.
+
+### General notes on CI
+CI tests run every time a pull request (PR) is created or updated. The exact tests
+that get run will depend on the destination branch you specify. Some tests take
+longer to run than others. Currently, our CI is set up to run a short
+series of tests when creating a PR to the dev branch and a longer series of tests
+when creating a PR to the master branch. You can look in the configuration files
+of the respective CI platform for more information on what gets run when.
+
+Most people will just want to create a PR with the destination set to their local dev
+branch of zstd. You can then find the status of the tests on the PR's page. You can also
+re-run tests and cancel running tests from the PR page or from the respective CI's dashboard.
+
 ## Issues
 We use GitHub issues to track public bugs. Please ensure your description is
 clear and has sufficient instructions to be able to reproduce the issue.
@@ -34,7 +398,7 @@ Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe
 disclosure of security bugs. In those cases, please go through the process
 outlined on that page and do not file a public issue.

-## Coding Style  
+## Coding Style
 * 4 spaces for indentation rather than tabs

 ## License
@@ -1,12 +1,16 @@
 # ################################################################
-# Copyright (c) 2015-present, Yann Collet, Facebook, Inc.
+# Copyright (c) 2015-2020, Yann Collet, Facebook, Inc.
 # All rights reserved.
 #
 # This source code is licensed under both the BSD-style license (found in the
 # LICENSE file in the root directory of this source tree) and the GPLv2 (found
 # in the COPYING file in the root directory of this source tree).
+# You may select, at your option, one of the above-listed licenses.
 # ################################################################

+# verbose mode (print commands) on V=1 or VERBOSE=1
+Q = $(if $(filter 1,$(V) $(VERBOSE)),,@)
+
 PRGDIR   = programs
 ZSTDDIR  = lib
 BUILDIR  = build
@@ -17,7 +21,16 @@ FUZZDIR  = $(TESTDIR)/fuzz
 # Define nul output
 VOID = /dev/null

-ifneq (,$(filter Windows%,$(OS)))
+# When cross-compiling from linux to windows, you might
+# need to specify this as "Windows." Fedora build fails
+# without it.
+#
+# Note: mingw-w64 build from linux to windows does not
+# fail on other tested distros (ubuntu, debian) even
+# without manually specifying the TARGET_SYSTEM.
+TARGET_SYSTEM ?= $(OS)
+
+ifneq (,$(filter Windows%,$(TARGET_SYSTEM)))
 EXT =.exe
 else
 EXT =
@@ -35,9 +48,9 @@ allmost: allzstd zlibwrapper

 # skip zwrapper, can't build that on alternate architectures without the proper zlib installed
 .PHONY: allzstd
-allzstd: lib
-	$(MAKE) -C $(PRGDIR) all
-	$(MAKE) -C $(TESTDIR) all
+allzstd: lib-all
+	$(Q)$(MAKE) -C $(PRGDIR) all
+	$(Q)$(MAKE) -C $(TESTDIR) all

 .PHONY: all32
 all32:
@@ -45,18 +58,19 @@ all32:
 	$(MAKE) -C $(TESTDIR) all32

 .PHONY: lib lib-release libzstd.a
-lib lib-release :
-	@$(MAKE) -C $(ZSTDDIR) $@
+lib-all : lib
+lib lib-release lib-all :
+	$(Q)$(MAKE) -C $(ZSTDDIR) $@

 .PHONY: zstd zstd-release
 zstd zstd-release:
-	@$(MAKE) -C $(PRGDIR) $@
-	cp $(PRGDIR)/zstd$(EXT) .
+	$(Q)$(MAKE) -C $(PRGDIR) $@
+	$(Q)cp $(PRGDIR)/zstd$(EXT) .

 .PHONY: zstdmt
 zstdmt:
-	@$(MAKE) -C $(PRGDIR) $@
-	cp $(PRGDIR)/zstd$(EXT) ./zstdmt$(EXT)
+	$(Q)$(MAKE) -C $(PRGDIR) $@
+	$(Q)cp $(PRGDIR)/zstd$(EXT) ./zstdmt$(EXT)

 .PHONY: zlibwrapper
 zlibwrapper: lib
@@ -69,12 +83,12 @@ test: MOREFLAGS += -g -DDEBUGLEVEL=$(DEBUGLEVEL) -Werror
 test:
 	MOREFLAGS="$(MOREFLAGS)" $(MAKE) -j -C $(PRGDIR) allVariants
 	$(MAKE) -C $(TESTDIR) $@
-	ZSTD=../../programs/zstd $(MAKE) -C doc/educational_decoder test
+	ZSTD=../../programs/zstd $(MAKE) -C doc/educational_decoder $@

 ## shortest: same as `make check`
 .PHONY: shortest
 shortest:
-	$(MAKE) -C $(TESTDIR) $@
+	$(Q)$(MAKE) -C $(TESTDIR) $@

 ## check: run basic tests for `zstd` cli
 .PHONY: check
@@ -108,7 +122,8 @@ contrib: lib
 	$(MAKE) -C contrib/pzstd all
 	$(MAKE) -C contrib/seekable_format/examples all
 	$(MAKE) -C contrib/largeNbDicts all
-	cd contrib/single_file_decoder/ ; ./build_test.sh
+	cd contrib/single_file_libs/ ; ./build_decoder_test.sh
+	cd contrib/single_file_libs/ ; ./build_library_test.sh

 .PHONY: cleanTabs
 cleanTabs:
@@ -116,17 +131,17 @@ cleanTabs:

 .PHONY: clean
 clean:
-	@$(MAKE) -C $(ZSTDDIR) $@ > $(VOID)
-	@$(MAKE) -C $(PRGDIR) $@ > $(VOID)
-	@$(MAKE) -C $(TESTDIR) $@ > $(VOID)
-	@$(MAKE) -C $(ZWRAPDIR) $@ > $(VOID)
-	@$(MAKE) -C examples/ $@ > $(VOID)
-	@$(MAKE) -C contrib/gen_html $@ > $(VOID)
-	@$(MAKE) -C contrib/pzstd $@ > $(VOID)
-	@$(MAKE) -C contrib/seekable_format/examples $@ > $(VOID)
-	@$(MAKE) -C contrib/largeNbDicts $@ > $(VOID)
-	@$(RM) zstd$(EXT) zstdmt$(EXT) tmp*
-	@$(RM) -r lz4
+	$(Q)$(MAKE) -C $(ZSTDDIR) $@ > $(VOID)
+	$(Q)$(MAKE) -C $(PRGDIR) $@ > $(VOID)
+	$(Q)$(MAKE) -C $(TESTDIR) $@ > $(VOID)
+	$(Q)$(MAKE) -C $(ZWRAPDIR) $@ > $(VOID)
+	$(Q)$(MAKE) -C examples/ $@ > $(VOID)
+	$(Q)$(MAKE) -C contrib/gen_html $@ > $(VOID)
+	$(Q)$(MAKE) -C contrib/pzstd $@ > $(VOID)
+	$(Q)$(MAKE) -C contrib/seekable_format/examples $@ > $(VOID)
+	$(Q)$(MAKE) -C contrib/largeNbDicts $@ > $(VOID)
+	$(Q)$(RM) zstd$(EXT) zstdmt$(EXT) tmp*
+	$(Q)$(RM) -r lz4
 	@echo Cleaning completed

 #------------------------------------------------------------------------------
@@ -150,7 +165,7 @@ EGREP = egrep $(EGREP_OPTIONS)
 ## list: Print all targets and their descriptions (if provided)
 .PHONY: list
 list:
-	@TARGETS=$$($(MAKE) -pRrq -f $(lastword $(MAKEFILE_LIST)) : 2>/dev/null \
+	$(Q)TARGETS=$$($(MAKE) -pRrq -f $(lastword $(MAKEFILE_LIST)) : 2>/dev/null \
 		| awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' \
 		| $(EGREP) -v  -e '^[^[:alnum:]]' | sort); \
 	{ \
@@ -165,13 +180,13 @@ list:

 .PHONY: install armtest usan asan uasan
 install:
-	@$(MAKE) -C $(ZSTDDIR) $@
-	@$(MAKE) -C $(PRGDIR) $@
+	$(Q)$(MAKE) -C $(ZSTDDIR) $@
+	$(Q)$(MAKE) -C $(PRGDIR) $@

 .PHONY: uninstall
 uninstall:
-	@$(MAKE) -C $(ZSTDDIR) $@
-	@$(MAKE) -C $(PRGDIR) $@
+	$(Q)$(MAKE) -C $(ZSTDDIR) $@
+	$(Q)$(MAKE) -C $(PRGDIR) $@

 .PHONY: travis-install
 travis-install:
@@ -360,7 +375,7 @@ cmakebuild:
 	$(MAKE) -C $(BUILDIR)/cmake/build -j4;
 	$(MAKE) -C $(BUILDIR)/cmake/build install;
 	$(MAKE) -C $(BUILDIR)/cmake/build uninstall;
-	cd $(BUILDIR)/cmake/build; ctest -V
+	cd $(BUILDIR)/cmake/build; ctest -V -L Medium

 c89build: clean
 	$(CC) -v
@@ -31,10 +31,10 @@ a list of known ports and bindings is provided on [Zstandard homepage](http://ww
 ## Benchmarks

 For reference, several fast compression algorithms were tested and compared
-on a server running Arch Linux (`Linux version 5.0.5-arch1-1`),
+on a server running Arch Linux (`Linux version 5.5.11-arch1-1`),
 with a Core i9-9900K CPU @ 5.0GHz,
 using [lzbench], an open-source in-memory benchmark by @inikep
-compiled with [gcc] 8.2.1,
+compiled with [gcc] 9.3.0,
 on the [Silesia compression corpus].

 [lzbench]: https://github.com/inikep/lzbench
@@ -43,18 +43,26 @@ on the [Silesia compression corpus].

 | Compressor name         | Ratio | Compression| Decompress.|
 | ---------------         | ------| -----------| ---------- |
-| **zstd 1.4.4 -1**       | 2.884 |   520 MB/s |  1600 MB/s |
-| zlib 1.2.11 -1          | 2.743 |   110 MB/s |   440 MB/s |
-| brotli 1.0.7 -0         | 2.701 |   430 MB/s |   470 MB/s |
-| quicklz 1.5.0 -1        | 2.238 |   600 MB/s |   800 MB/s |
-| lzo1x 2.09 -1           | 2.106 |   680 MB/s |   950 MB/s |
-| lz4 1.8.3               | 2.101 |   800 MB/s |  4220 MB/s |
-| snappy 1.1.4            | 2.073 |   580 MB/s |  2020 MB/s |
-| lzf 3.6 -1              | 2.077 |   440 MB/s |   930 MB/s |
+| **zstd 1.4.5 -1**       | 2.884 |   500 MB/s |  1660 MB/s |
+| zlib 1.2.11 -1          | 2.743 |    90 MB/s |   400 MB/s |
+| brotli 1.0.7 -0         | 2.703 |   400 MB/s |   450 MB/s |
+| **zstd 1.4.5 --fast=1** | 2.434 |   570 MB/s |  2200 MB/s |
+| **zstd 1.4.5 --fast=3** | 2.312 |   640 MB/s |  2300 MB/s |
+| quicklz 1.5.0 -1        | 2.238 |   560 MB/s |   710 MB/s |
+| **zstd 1.4.5 --fast=5** | 2.178 |   700 MB/s |  2420 MB/s |
+| lzo1x 2.10 -1           | 2.106 |   690 MB/s |   820 MB/s |
+| lz4 1.9.2               | 2.101 |   740 MB/s |  4530 MB/s |
+| **zstd 1.4.5 --fast=7** | 2.096 |   750 MB/s |  2480 MB/s |
+| lzf 3.6 -1              | 2.077 |   410 MB/s |   860 MB/s |
+| snappy 1.1.8            | 2.073 |   560 MB/s |  1790 MB/s |

 [zlib]: http://www.zlib.net/
 [LZ4]: http://www.lz4.org/

+The negative compression levels, specified with `--fast=#`,
+offer faster compression and decompression speed in exchange for some loss in
+compression ratio compared to level 1, as seen in the table above.
+
 Zstd can also offer stronger compression ratios at the cost of compression speed.
 Speed vs Compression trade-off is configurable by small increments.
 Decompression speed is preserved and remains roughly the same at all settings,
@@ -143,6 +151,18 @@ example about how Meson is used to build this project.

 Note that default build type is **release**.

+### VCPKG
+You can build and install zstd [vcpkg](https://github.com/Microsoft/vcpkg/) dependency manager:
+
+    git clone https://github.com/Microsoft/vcpkg.git
+    cd vcpkg
+    ./bootstrap-vcpkg.sh
+    ./vcpkg integrate install
+    ./vcpkg install zstd
+
+The zstd port in vcpkg is kept up to date by Microsoft team members and community contributors.
+If the version is out of date, please [create an issue or pull request](https://github.com/Microsoft/vcpkg) on the vcpkg repository.
+
 ### Visual Studio (Windows)

 Going into `build` directory, you will find additional possibilities:
@@ -156,6 +176,12 @@ Going into `build` directory, you will find additional possibilities:
 You can build the zstd binary via buck by executing: `buck build programs:zstd` from the root of the repo.
 The output binary will be in `buck-out/gen/programs/`.

+## Testing
+
+You can run quick local smoke tests by executing the `playTest.sh` script from the `src/tests` directory.
+Two env variables `$ZSTD_BIN` and `$DATAGEN_BIN` are needed for the test script to locate the zstd and datagen binary.
+For information on CI testing, please refer to TESTING.md
+
 ## Status

 Zstandard is currently deployed within Facebook. It is used continuously to compress large amounts of data in multiple formats and use cases.
@@ -11,7 +11,7 @@ They consist of the following tests:
 - Compilation on all supported targets (x86, x86_64, ARM, AArch64, PowerPC, and PowerPC64)
 - Compilation on various versions of gcc, clang, and g++
 - `tests/playTests.sh` on x86_64, without the tests on long data (CLI tests)
- Small tests (`tests/legacy.c`, `tests/longmatch.c`, `tests/symbols.c`) on x64_64
+- Small tests (`tests/legacy.c`, `tests/longmatch.c`) on x64_64

 Medium Tests
 ------------
@@ -19,7 +19,7 @@ Medium tests run on every commit and pull request to `dev` branch, on TravisCI.
 They consist of the following tests:
 - The following tests run with UBsan and Asan on x86_64 and x86, as well as with
  Msan on x86_64
-  - `tests/playTests.sh --test-long-data`
+  - `tests/playTests.sh --test-large-data`
  - Fuzzer tests: `tests/fuzzer.c`, `tests/zstreamtest.c`, and `tests/decodecorpus.c`
 - `tests/zstreamtest.c` under Tsan (streaming mode, including multithreaded mode)
 - Valgrind Test (`make -C tests valgrindTest`) (testing CLI and fuzzer under valgrind)
@@ -14,7 +14,7 @@
    - COMPILER: "gcc"
      HOST:     "mingw"
      PLATFORM: "x64"
-      SCRIPT:   "make allzstd MOREFLAGS=-static && make -C tests test-symbols fullbench-lib"
+      SCRIPT:   "make allzstd MOREFLAGS=-static && make -C tests fullbench-lib"
      ARTIFACT: "true"
      BUILD:    "true"
    - COMPILER: "gcc"
@@ -169,7 +169,8 @@
  - SET "FUZZERTEST=-T30s"
  - if [%HOST%]==[visual] if [%CONFIGURATION%]==[Release] (
      CD tests &&
-      SET ZSTD=./zstd.exe &&
+      SET ZSTD_BIN=./zstd.exe&&
+      SET DATAGEN_BIN=./datagen.exe&&
      sh -e playTests.sh --test-large-data &&
      fullbench.exe -i1 &&
      fullbench.exe -i1 -P0 &&
@@ -249,9 +250,9 @@
      C:\cygwin64\bin\bash --login -c "
        set -e;
        cd build/cmake;
-        CFLAGS='-Werror' cmake -G 'Unix Makefiles' -DCMAKE_BUILD_TYPE=Debug -DZSTD_BUILD_TESTS:BOOL=ON .;
+        CFLAGS='-Werror' cmake -G 'Unix Makefiles' -DCMAKE_BUILD_TYPE=Debug -DZSTD_BUILD_TESTS:BOOL=ON -DZSTD_FUZZER_FLAGS=-T30s -DZSTD_ZSTREAM_FLAGS=-T30s .;
        make -j4;
-        ctest -V;
+        ctest -V -L Medium;
      "
    )
  - if [%HOST%]==[mingw] (
@@ -103,7 +103,14 @@ endif ()

 option(ZSTD_BUILD_PROGRAMS "BUILD PROGRAMS" ON)
 option(ZSTD_BUILD_CONTRIB "BUILD CONTRIB" OFF)
-option(ZSTD_BUILD_TESTS "BUILD TESTS" OFF)
+
+# Respect the conventional CMake option for enabling tests if it was specified on the first configure
+if (BUILD_TESTING)
+    set(ZSTD_BUILD_TESTS_default ON)
+else()
+    set(ZSTD_BUILD_TESTS_default OFF)
+endif()
+option(ZSTD_BUILD_TESTS "BUILD TESTS" ${ZSTD_BUILD_TESTS_default})
 if (MSVC)
    option(ZSTD_USE_STATIC_RUNTIME "LINK TO STATIC RUN-TIME LIBRARIES" OFF)
 endif ()
@@ -158,3 +165,39 @@ add_custom_target(clean-all
   COMMAND ${CMAKE_BUILD_TOOL} clean
   COMMAND rm -rf ${CMAKE_BINARY_DIR}/
 )
+
+#-----------------------------------------------------------------------------
+# Generate Package Config files
+#
+# This section is based on the boiler plate code from:
+# https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html#creating-packages
+#-----------------------------------------------------------------------------
+include(CMakePackageConfigHelpers)
+write_basic_package_version_file(
+    "${CMAKE_CURRENT_BINARY_DIR}/zstdConfigVersion.cmake"
+    VERSION ${zstd_VERSION}
+    COMPATIBILITY SameMajorVersion
+    )
+
+# A Package Config file that works from the build directory
+export(EXPORT zstdExports
+    FILE "${CMAKE_CURRENT_BINARY_DIR}/zstdTargets.cmake"
+    NAMESPACE zstd::
+    )
+configure_file(zstdConfig.cmake
+    "${CMAKE_CURRENT_BINARY_DIR}/zstdConfig.cmake"
+    COPYONLY
+    )
+
+# A Package Config file that works from the installation directory
+set(ConfigPackageLocation ${CMAKE_INSTALL_LIBDIR}/cmake/zstd)
+install(EXPORT zstdExports
+    FILE zstdTargets.cmake
+    NAMESPACE zstd::
+    DESTINATION ${ConfigPackageLocation}
+    )
+install(FILES
+    zstdConfig.cmake
+    "${CMAKE_CURRENT_BINARY_DIR}/zstdConfigVersion.cmake"
+    DESTINATION ${ConfigPackageLocation}
+    )
@@ -60,7 +60,6 @@ macro(ADD_ZSTD_COMPILATION_FLAGS)
             CMAKE_CXX_FLAGS_MINSIZEREL CMAKE_CXX_FLAGS_RELWITHDEBINFO)
        if( ${flag_var} )
            separate_arguments(${flag_var})
-            list(REMOVE_DUPLICATES ${flag_var})
            string(REPLACE ";" " " ${flag_var} "${${flag_var}}")
        endif()
    endforeach ()
@@ -77,8 +77,10 @@ if (MSVC)
 endif ()

 # Split project to static and shared libraries build
+set(library_targets)
 if (ZSTD_BUILD_SHARED)
    add_library(libzstd_shared SHARED ${Sources} ${Headers} ${PlatformDependResources})
+    list(APPEND library_targets libzstd_shared)
    if (ZSTD_MULTITHREAD_SUPPORT)
        set_property(TARGET libzstd_shared APPEND PROPERTY COMPILE_DEFINITIONS "ZSTD_MULTITHREAD")
        if (UNIX)
@@ -88,6 +90,7 @@ if (ZSTD_BUILD_SHARED)
 endif ()
 if (ZSTD_BUILD_STATIC)
    add_library(libzstd_static STATIC ${Sources} ${Headers})
+    list(APPEND library_targets libzstd_static)
    if (ZSTD_MULTITHREAD_SUPPORT)
        set_property(TARGET libzstd_static APPEND PROPERTY COMPILE_DEFINITIONS "ZSTD_MULTITHREAD")
        if (UNIX)
@@ -131,58 +134,35 @@ if (ZSTD_BUILD_STATIC)
            OUTPUT_NAME ${STATIC_LIBRARY_BASE_NAME})
 endif ()

-if (UNIX)
+if (UNIX OR MINGW)
    # pkg-config
    set(PREFIX "${CMAKE_INSTALL_PREFIX}")
-    set(LIBDIR "${CMAKE_INSTALL_FULL_LIBDIR}")
-    set(VERSION "${zstd_VERSION_MAJOR}.${zstd_VERSION_MINOR}.${zstd_VERSION_PATCH}")
+    set(LIBDIR "${CMAKE_INSTALL_LIBDIR}")
+    set(INCLUDEDIR "${CMAKE_INSTALL_INCLUDEDIR}")
+    set(VERSION "${zstd_VERSION}")
    add_custom_target(libzstd.pc ALL
            ${CMAKE_COMMAND} -DIN="${LIBRARY_DIR}/libzstd.pc.in" -DOUT="libzstd.pc"
-            -DPREFIX="${PREFIX}" -DVERSION="${VERSION}"
+            -DPREFIX="${PREFIX}" -DLIBDIR="${LIBDIR}" -DINCLUDEDIR="${INCLUDEDIR}" -DVERSION="${VERSION}"
            -P "${CMAKE_CURRENT_SOURCE_DIR}/pkgconfig.cmake"
            COMMENT "Creating pkg-config file")

-    install(FILES "${CMAKE_CURRENT_BINARY_DIR}/libzstd.pc" DESTINATION "${LIBDIR}/pkgconfig")
+    install(FILES "${CMAKE_CURRENT_BINARY_DIR}/libzstd.pc" DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig")
 endif ()

 # install target
 install(FILES
    "${LIBRARY_DIR}/zstd.h"
-    "${LIBRARY_DIR}/deprecated/zbuff.h"
    "${LIBRARY_DIR}/dictBuilder/zdict.h"
-    "${LIBRARY_DIR}/dictBuilder/cover.h"
    "${LIBRARY_DIR}/common/zstd_errors.h"
    DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}")

-if (ZSTD_BUILD_SHARED)
-    install(TARGETS libzstd_shared EXPORT zstdExports
-                                   INCLUDES DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
-                                   ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
-                                   LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}"
+install(TARGETS ${library_targets}
+    EXPORT zstdExports
+    INCLUDES DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
+    ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
+    LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}"
+    RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}"
    )
-endif()
-if (ZSTD_BUILD_STATIC)
-    install(TARGETS libzstd_static EXPORT zstdExports
-                                   INCLUDES DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}"
-                                   ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}"
-                                   LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}"
-    )
-endif ()
-
-# export targets + find config
-configure_file(
-  "config.cmake.in"
-  "${CMAKE_CURRENT_BINARY_DIR}/zstdConfig.cmake"
-  @ONLY
-)
-install(FILES
-  "${CMAKE_CURRENT_BINARY_DIR}/zstdConfig.cmake"
-  DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/zstd"
-)
-install(EXPORT zstdExports
-  NAMESPACE zstd::
-  DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/zstd"
-)

 # uninstall target
 if (NOT TARGET uninstall)
@@ -1,7 +0,0 @@
-include(FindPackageHandleStandardArgs)
-set(${CMAKE_FIND_PACKAGE_NAME}_CONFIG ${CMAKE_CURRENT_LIST_FILE})
-find_package_handle_standard_args(zstd CONFIG_MODE)
-
-if(NOT TARGET zstd::libzstd_shared)
-    include("${CMAKE_CURRENT_LIST_DIR}/zstdExports.cmake")
-endif()
@@ -32,6 +32,16 @@

 project(tests)

+# name: Cache variable name. The value is expected to be a semicolon-separated
+# list of command line flags
+# default_value: Value to initialize the option with. Can be space separated.
+function(AddTestFlagsOption name default_value doc)
+    string(STRIP "${default_value}" default_value)
+    string(REGEX REPLACE " +" ";" default_value "${default_value}")
+    set(${name} ${default_value} CACHE STRING "${doc}")
+    mark_as_advanced(${name})
+endfunction()
+
 set(CMAKE_INCLUDE_CURRENT_DIR TRUE)

 # Define programs directory, where sources and header files are located
@@ -43,18 +53,52 @@ include_directories(${TESTS_DIR} ${PROGRAMS_DIR} ${LIBRARY_DIR} ${LIBRARY_DIR}/c
 add_executable(datagen ${PROGRAMS_DIR}/datagen.c ${TESTS_DIR}/datagencli.c)
 target_link_libraries(datagen libzstd_static)

+#
+# fullbench
+#
 add_executable(fullbench ${PROGRAMS_DIR}/datagen.c ${PROGRAMS_DIR}/util.c ${PROGRAMS_DIR}/timefn.c ${PROGRAMS_DIR}/benchfn.c ${PROGRAMS_DIR}/benchzstd.c ${TESTS_DIR}/fullbench.c)
 target_link_libraries(fullbench libzstd_static)
+add_test(NAME fullbench COMMAND fullbench)

+#
+# fuzzer
+#
 add_executable(fuzzer ${PROGRAMS_DIR}/datagen.c ${PROGRAMS_DIR}/util.c ${PROGRAMS_DIR}/timefn.c ${TESTS_DIR}/fuzzer.c)
 target_link_libraries(fuzzer libzstd_static)
+AddTestFlagsOption(ZSTD_FUZZER_FLAGS "$ENV{FUZZERTEST} $ENV{FUZZER_FLAGS}"
+    "Semicolon-separated list of flags to pass to the fuzzer test (see `fuzzer -h` for usage)")
+add_test(NAME fuzzer COMMAND fuzzer ${ZSTD_FUZZER_FLAGS})
+# Disable the timeout since the run time is too long for the default timeout of
+# 1500 seconds and varies considerably between low-end and high-end CPUs.
+# set_tests_properties(fuzzer PROPERTIES TIMEOUT 0)

+#
+# zstreamtest
+#
 add_executable(zstreamtest ${PROGRAMS_DIR}/datagen.c ${PROGRAMS_DIR}/util.c ${PROGRAMS_DIR}/timefn.c ${TESTS_DIR}/seqgen.c ${TESTS_DIR}/zstreamtest.c)
 target_link_libraries(zstreamtest libzstd_static)
+AddTestFlagsOption(ZSTD_ZSTREAM_FLAGS "$ENV{ZSTREAM_TESTTIME} $ENV{FUZZER_FLAGS}"
+    "Semicolon-separated list of flags to pass to the zstreamtest test (see `zstreamtest -h` for usage)")
+add_test(NAME zstreamtest COMMAND zstreamtest ${ZSTD_ZSTREAM_FLAGS})

-add_test(
-    NAME playTests
-    COMMAND sh -c "ZSTD_BIN='$<TARGET_FILE:zstd>' DATAGEN_BIN='$<TARGET_FILE:datagen>' '${TESTS_DIR}/playTests.sh'")
+#
+# playTests.sh
+#
+AddTestFlagsOption(ZSTD_PLAYTESTS_FLAGS "--test-large-data"
+    "Semicolon-separated list of flags to pass to the playTests.sh test")
+add_test(NAME playTests COMMAND sh -c "${TESTS_DIR}/playTests.sh" ${ZSTD_PLAYTESTS_FLAGS})
+if (ZSTD_BUILD_PROGRAMS)
+    set_property(TEST playTests APPEND PROPERTY ENVIRONMENT
+        "ZSTD_BIN=$<TARGET_FILE:zstd>"
+        "DATAGEN_BIN=$<TARGET_FILE:datagen>"
+        )
+else()
+    message(STATUS "Disabling playTests.sh test because ZSTD_BUILD_PROGRAMS is not enabled")
+    set_tests_properties(playTests PROPERTIES DISABLED YES)
+endif()
+
+# Label the "Medium" set of tests (see TESTING.md)
+set_property(TEST fuzzer zstreamtest playTests APPEND PROPERTY LABELS Medium)

 if (UNIX)
    add_executable(paramgrill ${PROGRAMS_DIR}/benchfn.c ${PROGRAMS_DIR}/benchzstd.c ${PROGRAMS_DIR}/datagen.c ${PROGRAMS_DIR}/util.c ${PROGRAMS_DIR}/timefn.c ${TESTS_DIR}/paramgrill.c)
@@ -0,0 +1 @@
+include("${CMAKE_CURRENT_LIST_DIR}/zstdTargets.cmake")
@@ -127,6 +127,5 @@ pkgconfig.generate(libzstd,
  url: 'http://www.zstd.net/')

 install_headers(join_paths(zstd_rootdir, 'lib/zstd.h'),
-  join_paths(zstd_rootdir, 'lib/deprecated/zbuff.h'),
  join_paths(zstd_rootdir, 'lib/dictBuilder/zdict.h'),
  join_paths(zstd_rootdir, 'lib/common/zstd_errors.h'))
@@ -68,6 +68,7 @@ zstd_docdir = join_paths(zstd_datadir, 'doc', meson.project_name())
 # Built-in options
 use_debug = get_option('debug')
 buildtype = get_option('buildtype')
+default_library_type = get_option('default_library')

 # Custom options
 debug_level = get_option('debug_level')
@@ -121,7 +122,7 @@ elif cc_id == compiler_msvc
  if use_multi_thread
    msvc_compile_flags += '/MP'
  endif
-  if enable_static_runtime
+  if use_static_runtime
    msvc_compile_flags += '/MT'
  endif
  add_project_arguments(msvc_compile_flags, language: ['c', 'cpp'])
@@ -60,17 +60,6 @@ fuzzer = executable('fuzzer',
  dependencies: libzstd_dep,
  install: false)

-zbufftest_sources = [join_paths(zstd_rootdir, 'programs/datagen.c'),
-  join_paths(zstd_rootdir, 'programs/util.c'),
-  join_paths(zstd_rootdir, 'programs/timefn.c'),
-  join_paths(zstd_rootdir, 'tests/zbufftest.c')]
-zbufftest = executable('zbufftest',
-  zbufftest_sources,
-  c_args: ['-Wno-deprecated-declarations'],
-  include_directories: test_includes,
-  dependencies: libzstd_dep,
-  install: false)
-
 zstreamtest_sources = [join_paths(zstd_rootdir, 'programs/datagen.c'),
  join_paths(zstd_rootdir, 'programs/util.c'),
  join_paths(zstd_rootdir, 'programs/timefn.c'),
@@ -131,14 +120,6 @@ decodecorpus = executable('decodecorpus',
  dependencies: [ libzstd_dep, libm_dep ],
  install: false)

-symbols_sources = [join_paths(zstd_rootdir, 'tests/symbols.c')]
-symbols = executable('symbols',
-  symbols_sources,
-  include_directories: test_includes,
-  c_args: host_machine_os == os_windows ? '-DZSTD_DLL_IMPORT=1' : [],
-  dependencies: [ libzstd_dep ],
-  install: false)
-
 poolTests_sources = [join_paths(zstd_rootdir, 'programs/util.c'),
  join_paths(zstd_rootdir, 'programs/timefn.c'),
  join_paths(zstd_rootdir, 'tests/poolTests.c'),
@@ -177,7 +158,7 @@ if host_machine_os != os_windows
  test('test-zstd',
    playTests_sh,
    args: ZSTDRTTEST,
-    env: ['ZSTD=' + zstd.full_path()],
+    env: ['ZSTD_BIN=' + zstd.full_path(), 'DATAGEN_BIN=./datagen'],
    depends: [datagen],
    workdir: meson.current_build_dir(),
    timeout: 2800) # Timeout should work on HDD drive
@@ -201,10 +182,6 @@ if use_zlib
    timeout: 480)
 endif

-test('test-zbuff',
-  zbufftest,
-  args: [ZSTREAM_TESTTIME],
-  timeout: 120)
 test('test-zstream-1',
  zstreamtest,
  args: ['-v', ZSTREAM_TESTTIME] + FUZZER_FLAGS,
@@ -219,7 +196,6 @@ test('test-zstream-3',
  timeout: 120)
 test('test-longmatch', longmatch, timeout: 36)
 test('test-invalidDictionaries', invalidDictionaries) # should be fast
-test('test-symbols', symbols) # should be fast
 if 0 < legacy_level and legacy_level <= 4
  test('test-legacy', legacy) # should be fast
 endif
@@ -100,11 +100,14 @@ static char* readFile(const char* filename, size_t* size) {
  buf = malloc(*size);
  if (buf == NULL) {
    fprintf(stderr, "malloc failed\n");
+    fclose(f);
+    return NULL;
  }

  bytes_read = fread(buf, 1, *size, f);
  if (bytes_read != *size) {
    fprintf(stderr, "failed to read whole file\n");
+    fclose(f);
    free(buf);
    return NULL;
  }
@@ -334,6 +334,42 @@ createBufferCollection_fromSliceCollectionSizes(slice_collection_t sc)
    return result;
 }

+static buffer_collection_t
+createBufferCollection_fromSliceCollection(slice_collection_t sc)
+{
+    size_t const bufferSize = sliceCollection_totalCapacity(sc);
+
+    buffer_t buffer = createBuffer(bufferSize);
+    CONTROL(buffer.ptr != NULL);
+
+    size_t const nbSlices = sc.nbSlices;
+    void** const slices = (void**)malloc(nbSlices * sizeof(*slices));
+    CONTROL(slices != NULL);
+
+    size_t* const capacities = (size_t*)malloc(nbSlices * sizeof(*capacities));
+    CONTROL(capacities != NULL);
+
+    char* const ptr = (char*)buffer.ptr;
+    size_t pos = 0;
+    for (size_t n=0; n < nbSlices; n++) {
+        capacities[n] = sc.capacities[n];
+        slices[n] = ptr + pos;
+        pos += capacities[n];
+    }
+
+    for (size_t i = 0; i < nbSlices; i++) {
+        memcpy(slices[i], sc.slicePtrs[i], sc.capacities[i]);
+        capacities[i] = sc.capacities[i];
+    }
+
+    buffer_collection_t result;
+    result.buffer = buffer;
+    result.slices.nbSlices = nbSlices;
+    result.slices.capacities = capacities;
+    result.slices.slicePtrs = slices;
+
+    return result;
+}

 /* @return : kBuffNull if any error */
 static buffer_collection_t
@@ -397,6 +433,36 @@ typedef struct {
    size_t nbDDict;
 } ddict_collection_t;

+typedef struct {
+    ZSTD_CDict** cdicts;
+    size_t nbCDict;
+} cdict_collection_t;
+
+static const cdict_collection_t kNullCDictCollection = { NULL, 0 };
+
+static void freeCDictCollection(cdict_collection_t cdictc)
+{
+    for (size_t dictNb=0; dictNb < cdictc.nbCDict; dictNb++) {
+        ZSTD_freeCDict(cdictc.cdicts[dictNb]);
+    }
+    free(cdictc.cdicts);
+}
+
+/* returns .buffers=NULL if operation fails */
+static cdict_collection_t createCDictCollection(const void* dictBuffer, size_t dictSize, size_t nbCDict, int cLevel)
+{
+    ZSTD_CDict** const cdicts = malloc(nbCDict * sizeof(ZSTD_CDict*));
+    if (cdicts==NULL) return kNullCDictCollection;
+    for (size_t dictNb=0; dictNb < nbCDict; dictNb++) {
+        cdicts[dictNb] = ZSTD_createCDict(dictBuffer, dictSize, cLevel);
+        CONTROL(cdicts[dictNb] != NULL);
+    }
+    cdict_collection_t cdictc;
+    cdictc.cdicts = cdicts;
+    cdictc.nbCDict = nbCDict;
+    return cdictc;
+}
+
 static const ddict_collection_t kNullDDictCollection = { NULL, 0 };

 static void freeDDictCollection(ddict_collection_t ddictc)
@@ -425,7 +491,26 @@ static ddict_collection_t createDDictCollection(const void* dictBuffer, size_t d


 /* mess with addresses, so that linear scanning dictionaries != linear address scanning */
-void shuffleDictionaries(ddict_collection_t dicts)
+void shuffleCDictionaries(cdict_collection_t dicts)
+{
+    size_t const nbDicts = dicts.nbCDict;
+    for (size_t r=0; r<nbDicts; r++) {
+        size_t const d = (size_t)rand() % nbDicts;
+        ZSTD_CDict* tmpd = dicts.cdicts[d];
+        dicts.cdicts[d] = dicts.cdicts[r];
+        dicts.cdicts[r] = tmpd;
+    }
+    for (size_t r=0; r<nbDicts; r++) {
+        size_t const d1 = (size_t)rand() % nbDicts;
+        size_t const d2 = (size_t)rand() % nbDicts;
+        ZSTD_CDict* tmpd = dicts.cdicts[d1];
+        dicts.cdicts[d1] = dicts.cdicts[d2];
+        dicts.cdicts[d2] = tmpd;
+    }
+}
+
+/* mess with addresses, so that linear scanning dictionaries != linear address scanning */
+void shuffleDDictionaries(ddict_collection_t dicts)
 {
    size_t const nbDicts = dicts.nbDDict;
    for (size_t r=0; r<nbDicts; r++) {
@@ -485,6 +570,29 @@ static size_t compressBlocks(size_t* cSizes,   /* optional (can be NULL). If pre

 /* ---  Benchmark  --- */

+typedef struct {
+    ZSTD_CCtx* cctx;
+    size_t nbDicts;
+    size_t dictNb;
+    cdict_collection_t dictionaries;
+} compressInstructions;
+
+compressInstructions createCompressInstructions(cdict_collection_t dictionaries)
+{
+    compressInstructions ci;
+    ci.cctx = ZSTD_createCCtx();
+    CONTROL(ci.cctx != NULL);
+    ci.nbDicts = dictionaries.nbCDict;
+    ci.dictNb = 0;
+    ci.dictionaries = dictionaries;
+    return ci;
+}
+
+void freeCompressInstructions(compressInstructions ci)
+{
+    ZSTD_freeCCtx(ci.cctx);
+}
+
 typedef struct {
    ZSTD_DCtx* dctx;
    size_t nbDicts;
@@ -508,6 +616,23 @@ void freeDecompressInstructions(decompressInstructions di)
    ZSTD_freeDCtx(di.dctx);
 }

+/* benched function */
+size_t compress(const void* src, size_t srcSize, void* dst, size_t dstCapacity, void* payload)
+{
+    compressInstructions* const ci = (compressInstructions*) payload;
+    (void)dstCapacity;
+
+    ZSTD_compress_usingCDict(ci->cctx,
+                    dst, srcSize,
+                    src, srcSize,
+                    ci->dictionaries.cdicts[ci->dictNb]);
+
+    ci->dictNb = ci->dictNb + 1;
+    if (ci->dictNb >= ci->nbDicts) ci->dictNb = 0;
+
+    return srcSize;
+}
+
 /* benched function */
 size_t decompress(const void* src, size_t srcSize, void* dst, size_t dstCapacity, void* payload)
 {
@@ -527,8 +652,9 @@ size_t decompress(const void* src, size_t srcSize, void* dst, size_t dstCapacity

 static int benchMem(slice_collection_t dstBlocks,
                    slice_collection_t srcBlocks,
-                    ddict_collection_t dictionaries,
-                    unsigned nbRounds)
+                    ddict_collection_t ddictionaries,
+                    cdict_collection_t cdictionaries,
+                    unsigned nbRounds, int benchCompression)
 {
    assert(dstBlocks.nbSlices == srcBlocks.nbSlices);

@@ -539,10 +665,13 @@ static int benchMem(slice_collection_t dstBlocks,

    BMK_timedFnState_t* const benchState =
            BMK_createTimedFnState(total_time_ms, ms_per_round);
-    decompressInstructions di = createDecompressInstructions(dictionaries);
+
+    decompressInstructions di = createDecompressInstructions(ddictionaries);
+    compressInstructions ci = createCompressInstructions(cdictionaries);
+    void* payload = benchCompression ? (void*)&ci : (void*)&di;
    BMK_benchParams_t const bp = {
-        .benchFn = decompress,
-        .benchPayload = &di,
+        .benchFn = benchCompression ? compress : decompress,
+        .benchPayload = payload,
        .initFn = NULL,
        .initPayload = NULL,
        .errorFn = ZSTD_isError,
@@ -562,15 +691,20 @@ static int benchMem(slice_collection_t dstBlocks,
        double const dTime_ns = result.nanoSecPerRun;
        double const dTime_sec = (double)dTime_ns / 1000000000;
        size_t const srcSize = result.sumOfReturn;
-        double const dSpeed_MBps = (double)srcSize / dTime_sec / (1 MB);
-        if (dSpeed_MBps > bestSpeed) bestSpeed = dSpeed_MBps;
-        DISPLAY("Decompression Speed : %.1f MB/s \r", bestSpeed);
+        double const speed_MBps = (double)srcSize / dTime_sec / (1 MB);
+        if (speed_MBps > bestSpeed) bestSpeed = speed_MBps;
+        if (benchCompression)
+            DISPLAY("Compression Speed : %.1f MB/s \r", bestSpeed);
+        else
+            DISPLAY("Decompression Speed : %.1f MB/s \r", bestSpeed);
+
        fflush(stdout);
        if (BMK_isCompleted_TimedFn(benchState)) break;
    }
    DISPLAY("\n");

    freeDecompressInstructions(di);
+    freeCompressInstructions(ci);
    BMK_freeTimedFnState(benchState);

    return 0;   /* success */
@@ -586,7 +720,7 @@ int bench(const char** fileNameTable, unsigned nbFiles,
          const char* dictionary,
          size_t blockSize, int clevel,
          unsigned nbDictMax, unsigned nbBlocks,
-          unsigned nbRounds)
+          unsigned nbRounds, int benchCompression)
 {
    int result = 0;

@@ -662,25 +796,47 @@ int bench(const char** fileNameTable, unsigned nbFiles,
    /* now dstSlices contain the real compressed size of each block, instead of the maximum capacity */
    shrinkSizes(dstSlices, cSizes);

-    size_t const dictMem = ZSTD_estimateDDictSize(dictBuffer.size, ZSTD_dlm_byCopy);
    unsigned const nbDicts = nbDictMax ? nbDictMax : nbBlocks;
-    size_t const allDictMem = dictMem * nbDicts;
-    DISPLAYLEVEL(3, "generating %u dictionaries, using %.1f MB of memory \n",
-                    nbDicts, (double)allDictMem / (1 MB));

-    ddict_collection_t const dictionaries = createDDictCollection(dictBuffer.ptr, dictBuffer.size, nbDicts);
-    CONTROL(dictionaries.ddicts != NULL);
+    cdict_collection_t const cdictionaries = createCDictCollection(dictBuffer.ptr, dictBuffer.size, nbDicts, clevel);
+    CONTROL(cdictionaries.cdicts != NULL);

-    shuffleDictionaries(dictionaries);
+    ddict_collection_t const ddictionaries = createDDictCollection(dictBuffer.ptr, dictBuffer.size, nbDicts);
+    CONTROL(ddictionaries.ddicts != NULL);

-    buffer_collection_t resultCollection = createBufferCollection_fromSliceCollectionSizes(srcSlices);
-    CONTROL(resultCollection.buffer.ptr != NULL);
+    if (benchCompression) {
+        size_t const dictMem = ZSTD_estimateCDictSize(dictBuffer.size, ZSTD_dlm_byCopy);
+        size_t const allDictMem = dictMem * nbDicts;
+        DISPLAYLEVEL(3, "generating %u dictionaries, using %.1f MB of memory \n",
+                        nbDicts, (double)allDictMem / (1 MB));

-    result = benchMem(resultCollection.slices, dstSlices, dictionaries, nbRounds);
+        shuffleCDictionaries(cdictionaries);
+
+        buffer_collection_t resultCollection = createBufferCollection_fromSliceCollection(srcSlices);
+        CONTROL(resultCollection.buffer.ptr != NULL);
+
+        result = benchMem(dstSlices, resultCollection.slices, ddictionaries, cdictionaries, nbRounds, benchCompression);
+
+        freeBufferCollection(resultCollection);
+    } else {
+        size_t const dictMem = ZSTD_estimateDDictSize(dictBuffer.size, ZSTD_dlm_byCopy);
+        size_t const allDictMem = dictMem * nbDicts;
+        DISPLAYLEVEL(3, "generating %u dictionaries, using %.1f MB of memory \n",
+                        nbDicts, (double)allDictMem / (1 MB));
+
+        shuffleDDictionaries(ddictionaries);
+
+        buffer_collection_t resultCollection = createBufferCollection_fromSliceCollectionSizes(srcSlices);
+        CONTROL(resultCollection.buffer.ptr != NULL);
+
+        result = benchMem(resultCollection.slices, dstSlices, ddictionaries, cdictionaries, nbRounds, benchCompression);
+
+        freeBufferCollection(resultCollection);
+    }

    /* free all heap objects in reverse order */
-    freeBufferCollection(resultCollection);
-    freeDDictCollection(dictionaries);
+    freeCDictCollection(cdictionaries);
+    freeDDictCollection(ddictionaries);
    free(cSizes);
    ZSTD_freeCDict(cdict);
    freeBuffer(dictBuffer);
@@ -744,6 +900,8 @@ int usage(const char* exeName)
    DISPLAY (" %s [Options] filename(s) \n", exeName);
    DISPLAY (" \n");
    DISPLAY ("Options : \n");
+    DISPLAY ("-z          : benchmark compression (default) \n");
+    DISPLAY ("-d          : benchmark decompression \n");
    DISPLAY ("-r          : recursively load all files in subdirectories (default: off) \n");
    DISPLAY ("-B#         : split input into blocks of size # (default: no split) \n");
    DISPLAY ("-#          : use compression level # (default: %u) \n", CLEVEL_DEFAULT);
@@ -765,6 +923,7 @@ int bad_usage(const char* exeName)
 int main (int argc, const char** argv)
 {
    int recursiveMode = 0;
+    int benchCompression = 1;
    unsigned nbRounds = BENCH_TIME_DEFAULT_S;
    const char* const exeName = argv[0];

@@ -783,6 +942,8 @@ int main (int argc, const char** argv)
    for (int argNb = 1; argNb < argc ; argNb++) {
        const char* argument = argv[argNb];
        if (!strcmp(argument, "-h")) { free(nameTable); return usage(exeName); }
+        if (!strcmp(argument, "-d")) { benchCompression = 0; continue; }
+        if (!strcmp(argument, "-z")) { benchCompression = 1; continue; }
        if (!strcmp(argument, "-r")) { recursiveMode = 1; continue; }
        if (!strcmp(argument, "-D")) { argNb++; assert(argNb < argc); dictionary = argv[argNb]; continue; }
        if (longCommandWArg(&argument, "-i")) { nbRounds = readU32FromChar(&argument); continue; }
@@ -809,7 +970,7 @@ int main (int argc, const char** argv)
        nameTable = NULL;  /* UTIL_createFileNamesTable() takes ownership of nameTable */
    }

-    int result = bench(filenameTable->fileNames, (unsigned)filenameTable->tableSize, dictionary, blockSize, cLevel, nbDicts, nbBlocks, nbRounds);
+    int result = bench(filenameTable->fileNames, (unsigned)filenameTable->tableSize, dictionary, blockSize, cLevel, nbDicts, nbBlocks, nbRounds, benchCompression);

    UTIL_freeFileNamesTable(filenameTable);
    free(nameTable);
@@ -0,0 +1,42 @@
+## Edit Distance Match Finder
+
+```
+/* This match finder leverages techniques used in file comparison algorithms
+ * to find matches between a dictionary and a source file.
+ * 
+ * The original motivation for studying this approach was to try and optimize 
+ * Zstandard for the use case of patching: the most common scenario being 
+ * updating an existing software package with the next version. When patching,
+ * the difference between the old version of the package and the new version 
+ * is generally tiny (most of the new file will be identical to 
+ * the old one). In more technical terms, the edit distance (the minimal number 
+ * of changes required to take one sequence of bytes to another) between the 
+ * files would be small relative to the size of the file. 
+ * 
+ * Various 'diffing' algorithms utilize this notion of edit distance and 
+ * the corrensponding concept of a minimal edit script between two 
+ * sequences to identify the regions within two files where they differ. 
+ * The core algorithm used in this match finder is described in: 
+ * 
+ * "An O(ND) Difference Algorithm and its Variations", Eugene W. Myers,
+ *    Algorithmica Vol. 1, 1986, pp. 251-266,
+ *    <https://doi.org/10.1007/BF01840446>.
+ * 
+ * Additional algorithmic heuristics for speed improvement have also been included.
+ * These we inspired from implementations of various regular and binary diffing 
+ * algorithms such as GNU diff, bsdiff, and Xdelta. 
+ * 
+ * Note: after some experimentation, this approach proved to not provide enough 
+ * utility to justify the additional CPU used in finding matches. The one area
+ * where this approach consistenly outperforms Zstandard even on level 19 is 
+ * when compressing small files (<10 KB) using a equally small dictionary that 
+ * is very similar to the source file. For the use case that this was intended,
+ * (large similar files) this approach by itself took 5-10X longer than zstd-19 and 
+ * generally resulted in 2-3X larger files. The core advantage that zstd-19 has 
+ * over this appraoch for match finding is the overlapping matches. This approach 
+ * cannot find any. 
+ * 
+ * I'm leaving this in the contrib section in case this ever becomes interesting 
+ * to explore again.
+ * */
+```
@@ -0,0 +1,558 @@
+/*
+ * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+
+/*-*************************************
+*  Dependencies
+***************************************/
+
+/* Currently relies on qsort when combining contiguous matches. This can probabily 
+ * be avoided but would require changes to the algorithm. The qsort is far from 
+ * the bottleneck in this algorithm even for medium sized files so it's probably 
+ * not worth trying to address */ 
+#include <stdlib.h>
+#include <assert.h>
+
+#include "zstd_edist.h"
+#include "mem.h"
+
+/*-*************************************
+*  Constants
+***************************************/
+
+/* Just a sential for the entires of the diagnomal matrix */
+#define ZSTD_EDIST_DIAG_MAX (S32)(1 << 30)
+
+/* How large should a snake be to be considered a 'big' snake. 
+ * For an explanation of what a 'snake' is with respect to the 
+ * edit distance matrix, see the linked paper in zstd_edist.h */
+#define ZSTD_EDIST_SNAKE_THRESH 20
+
+/* After how many iterations should we start to use the heuristic
+ * based on 'big' snakes */
+#define ZSTD_EDIST_SNAKE_ITER_THRESH 200
+
+/* After how many iterations should be just give up and take 
+ * the best availabe edit script for this round */ 
+#define ZSTD_EDIST_EXPENSIVE_THRESH 1024
+
+/*-*************************************
+*  Structures
+***************************************/
+
+typedef struct {
+    U32 dictIdx;
+    U32 srcIdx;
+    U32 matchLength;
+} ZSTD_eDist_match;
+
+typedef struct {
+    const BYTE* dict;
+    const BYTE* src;
+    size_t dictSize;
+    size_t srcSize;
+    S32* forwardDiag;            /* Entires of the forward diagonal stored here */ 
+    S32* backwardDiag;           /* Entires of the backward diagonal stored here.
+                                  *   Note: this buffer and the 'forwardDiag' buffer 
+                                  *   are contiguous. See the ZSTD_eDist_genSequences */ 
+    ZSTD_eDist_match* matches;   /* Accumulate matches of length 1 in this buffer. 
+                                  *   In a subsequence post-processing step, we combine 
+                                  *   contiguous matches. */
+    U32 nbMatches;
+} ZSTD_eDist_state;
+
+typedef struct {
+    S32 dictMid;           /* The mid diagonal for the dictionary */
+    S32 srcMid;            /* The mid diagonal for the source */ 
+    int lowUseHeuristics;  /* Should we use heuristics for the low part */
+    int highUseHeuristics; /* Should we use heuristics for the high part */ 
+} ZSTD_eDist_partition;
+
+/*-*************************************
+*  Internal
+***************************************/
+
+static void ZSTD_eDist_diag(ZSTD_eDist_state* state,
+                    ZSTD_eDist_partition* partition,
+                    S32 dictLow, S32 dictHigh, S32 srcLow, 
+                    S32 srcHigh, int useHeuristics)
+{
+    S32* const forwardDiag = state->forwardDiag;
+    S32* const backwardDiag = state->backwardDiag;
+    const BYTE* const dict = state->dict;
+    const BYTE* const src = state->src;
+
+    S32 const diagMin = dictLow - srcHigh;
+    S32 const diagMax = dictHigh - srcLow;
+    S32 const forwardMid = dictLow - srcLow;
+    S32 const backwardMid = dictHigh - srcHigh;
+
+    S32 forwardMin = forwardMid;
+    S32 forwardMax = forwardMid;
+    S32 backwardMin = backwardMid;
+    S32 backwardMax = backwardMid;
+    int odd = (forwardMid - backwardMid) & 1;
+    U32 iterations;
+
+    forwardDiag[forwardMid] = dictLow;
+    backwardDiag[backwardMid] = dictHigh;
+
+    /* Main loop for updating diag entries. Unless useHeuristics is 
+     * set to false, this loop will run until it finds the minimal 
+     * edit script */ 
+    for (iterations = 1;;iterations++) {
+        S32 diag;
+        int bigSnake = 0;
+        
+        if (forwardMin > diagMin) {
+            forwardMin--;
+            forwardDiag[forwardMin - 1] = -1;
+        } else {
+            forwardMin++;
+        }
+
+        if (forwardMax < diagMax) {
+            forwardMax++;
+            forwardDiag[forwardMax + 1] = -1;
+        } else {
+            forwardMax--;
+        }
+
+        for (diag = forwardMax; diag >= forwardMin; diag -= 2) {
+            S32 dictIdx;
+            S32 srcIdx;
+            S32 low = forwardDiag[diag - 1];
+            S32 high = forwardDiag[diag + 1];
+            S32 dictIdx0 = low < high ? high : low + 1;
+
+            for (dictIdx = dictIdx0, srcIdx = dictIdx0 - diag;
+                dictIdx < dictHigh && srcIdx < srcHigh && dict[dictIdx] == src[srcIdx];
+                dictIdx++, srcIdx++) continue;
+
+            if (dictIdx - dictIdx0 > ZSTD_EDIST_SNAKE_THRESH)
+                bigSnake = 1;
+
+            forwardDiag[diag] = dictIdx;
+
+            if (odd && backwardMin <= diag && diag <= backwardMax && backwardDiag[diag] <= dictIdx) {
+                partition->dictMid = dictIdx;
+                partition->srcMid = srcIdx;
+                partition->lowUseHeuristics = 0;
+                partition->highUseHeuristics = 0;
+                return;
+            }
+        }
+
+        if (backwardMin > diagMin) {
+            backwardMin--;
+            backwardDiag[backwardMin - 1] = ZSTD_EDIST_DIAG_MAX;
+        } else {
+            backwardMin++;
+        }
+
+        if (backwardMax < diagMax) {
+            backwardMax++;
+            backwardDiag[backwardMax + 1] = ZSTD_EDIST_DIAG_MAX;
+        } else {
+            backwardMax--;
+        }
+
+
+        for (diag = backwardMax; diag >= backwardMin; diag -= 2) {
+            S32 dictIdx;
+            S32 srcIdx;
+            S32 low = backwardDiag[diag - 1];
+            S32 high = backwardDiag[diag + 1];
+            S32 dictIdx0 = low < high ? low : high - 1;
+
+            for (dictIdx = dictIdx0, srcIdx = dictIdx0 - diag;
+                dictLow < dictIdx && srcLow < srcIdx && dict[dictIdx - 1] == src[srcIdx - 1];
+                dictIdx--, srcIdx--) continue;
+
+            if (dictIdx0 - dictIdx > ZSTD_EDIST_SNAKE_THRESH)
+                bigSnake = 1;
+
+            backwardDiag[diag] = dictIdx;
+
+            if (!odd && forwardMin <= diag && diag <= forwardMax && dictIdx <= forwardDiag[diag]) {
+                partition->dictMid = dictIdx;
+                partition->srcMid = srcIdx;
+                partition->lowUseHeuristics = 0;
+                partition->highUseHeuristics = 0;
+                return;
+            }
+        }
+
+        if (!useHeuristics)
+            continue;
+
+        /* Everything under this point is a heuritic. Using these will 
+         * substantially speed up the match finding. In some cases, taking 
+         * the total match finding time from several minutes to seconds.
+         * Of course, the caveat is that the edit script found may no longer 
+         * be optimal */ 
+
+        /* Big snake heuristic */ 
+        if (iterations > ZSTD_EDIST_SNAKE_ITER_THRESH && bigSnake) {
+            {
+                S32 best = 0;
+                
+                for (diag = forwardMax; diag >= forwardMin; diag -= 2) {
+                    S32 diagDiag = diag - forwardMid;
+                    S32 dictIdx = forwardDiag[diag];
+                    S32 srcIdx = dictIdx - diag;
+                    S32 v = (dictIdx - dictLow) * 2 - diagDiag;
+
+                    if (v > 12 * (iterations + (diagDiag < 0 ? -diagDiag : diagDiag))) {
+                        if (v > best 
+                          && dictLow + ZSTD_EDIST_SNAKE_THRESH <= dictIdx && dictIdx <= dictHigh
+                          && srcLow + ZSTD_EDIST_SNAKE_THRESH <= srcIdx && srcIdx <= srcHigh) {
+                            S32 k;
+                            for (k = 1; dict[dictIdx - k] == src[srcIdx - k]; k++) {
+                                if (k == ZSTD_EDIST_SNAKE_THRESH) {
+                                    best = v;
+                                    partition->dictMid = dictIdx;
+                                    partition->srcMid = srcIdx;
+                                    break;
+                                }
+                            }
+                        }
+                    }
+                }
+
+                if (best > 0) {
+                    partition->lowUseHeuristics = 0;
+                    partition->highUseHeuristics = 1;
+                    return;
+                }
+            }
+
+            {
+                S32 best = 0;
+
+                for (diag = backwardMax; diag >= backwardMin; diag -= 2) {
+                    S32 diagDiag = diag - backwardMid;
+                    S32 dictIdx = backwardDiag[diag];
+                    S32 srcIdx = dictIdx - diag;
+                    S32 v = (dictHigh - dictIdx) * 2 + diagDiag;
+
+                    if (v > 12 * (iterations + (diagDiag < 0 ? -diagDiag : diagDiag))) {
+                        if (v > best 
+                          && dictLow < dictIdx && dictIdx <= dictHigh - ZSTD_EDIST_SNAKE_THRESH
+                          && srcLow < srcIdx && srcIdx <= srcHigh - ZSTD_EDIST_SNAKE_THRESH) {
+                            int k;
+                            for (k = 0; dict[dictIdx + k] == src[srcIdx + k]; k++) {
+                                if (k == ZSTD_EDIST_SNAKE_THRESH - 1) { 
+                                    best = v;
+                                    partition->dictMid = dictIdx;
+                                    partition->srcMid = srcIdx;
+                                    break; 
+                                }
+                            }
+                        }
+                    }
+                }
+
+                if (best > 0) {
+                    partition->lowUseHeuristics = 1;
+                    partition->highUseHeuristics = 0;
+                    return;
+                }
+            }
+        }
+
+        /* More general 'too expensive' heuristic */ 
+        if (iterations >= ZSTD_EDIST_EXPENSIVE_THRESH) {
+            S32 forwardDictSrcBest;
+            S32 forwardDictBest = 0;
+            S32 backwardDictSrcBest;
+            S32 backwardDictBest = 0;
+
+            forwardDictSrcBest = -1;
+            for (diag = forwardMax; diag >= forwardMin; diag -= 2) {
+                S32 dictIdx = MIN(forwardDiag[diag], dictHigh);
+                S32 srcIdx = dictIdx - diag;
+
+                if (srcHigh < srcIdx) {
+                    dictIdx = srcHigh + diag;
+                    srcIdx = srcHigh;
+                }
+
+                if (forwardDictSrcBest < dictIdx + srcIdx) {
+                    forwardDictSrcBest = dictIdx + srcIdx;
+                    forwardDictBest = dictIdx;
+                }
+            }
+
+            backwardDictSrcBest = ZSTD_EDIST_DIAG_MAX;
+            for (diag = backwardMax; diag >= backwardMin; diag -= 2) {
+                S32 dictIdx = MAX(dictLow, backwardDiag[diag]);
+                S32 srcIdx = dictIdx - diag;
+
+                if (srcIdx < srcLow) {
+                    dictIdx = srcLow + diag;
+                    srcIdx = srcLow;
+                }
+
+                if (dictIdx + srcIdx < backwardDictSrcBest) {
+                    backwardDictSrcBest = dictIdx + srcIdx;
+                    backwardDictBest = dictIdx;
+                }
+            }
+
+            if ((dictHigh + srcHigh) - backwardDictSrcBest < forwardDictSrcBest - (dictLow + srcLow)) {
+                partition->dictMid = forwardDictBest;
+                partition->srcMid = forwardDictSrcBest - forwardDictBest;
+                partition->lowUseHeuristics = 0;
+                partition->highUseHeuristics = 1;
+            } else {
+                partition->dictMid = backwardDictBest;
+                partition->srcMid = backwardDictSrcBest - backwardDictBest;
+                partition->lowUseHeuristics = 1;
+                partition->highUseHeuristics = 0;
+            }
+            return;
+        }
+    }
+}
+
+static void ZSTD_eDist_insertMatch(ZSTD_eDist_state* state, 
+                    S32 const dictIdx, S32 const srcIdx)
+{
+    state->matches[state->nbMatches].dictIdx = dictIdx;
+    state->matches[state->nbMatches].srcIdx = srcIdx;
+    state->matches[state->nbMatches].matchLength = 1;
+    state->nbMatches++;
+}
+
+static int ZSTD_eDist_compare(ZSTD_eDist_state* state,
+                    S32 dictLow, S32 dictHigh, S32 srcLow,
+                    S32 srcHigh, int useHeuristics)
+{
+    const BYTE* const dict = state->dict;
+    const BYTE* const src = state->src;
+
+    /* Found matches while traversing from the low end */ 
+    while (dictLow < dictHigh && srcLow < srcHigh && dict[dictLow] == src[srcLow]) {
+        ZSTD_eDist_insertMatch(state, dictLow, srcLow);
+        dictLow++;
+        srcLow++;
+    }
+
+    /* Found matches while traversing from the high end */
+    while (dictLow < dictHigh && srcLow < srcHigh && dict[dictHigh - 1] == src[srcHigh - 1]) {
+        ZSTD_eDist_insertMatch(state, dictHigh - 1, srcHigh - 1);
+        dictHigh--;
+        srcHigh--;
+    }
+    
+    /* If the low and high end end up touching. If we wanted to make 
+     * note of the differences like most diffing algorithms do, we would 
+     * do so here. In our case, we're only concerned with matches 
+     * Note: if you wanted to find the edit distance of the algorithm, 
+     *   you could just accumulate the cost for an insertion/deletion 
+     *   below. */ 
+    if (dictLow == dictHigh) {
+        while (srcLow < srcHigh) {
+            /* Reaching this point means inserting src[srcLow] into 
+             * the current position of dict */ 
+            srcLow++;
+        }
+    } else if (srcLow == srcHigh) {
+        while (dictLow < dictHigh) {
+            /* Reaching this point means deleteing dict[dictLow] from 
+             * the current positino of dict */ 
+            dictLow++;
+        }
+    } else {
+        ZSTD_eDist_partition partition;
+        partition.dictMid = 0;
+        partition.srcMid = 0;
+        ZSTD_eDist_diag(state, &partition, dictLow, dictHigh, 
+            srcLow, srcHigh, useHeuristics);
+        if (ZSTD_eDist_compare(state, dictLow, partition.dictMid, 
+          srcLow, partition.srcMid, partition.lowUseHeuristics))
+            return 1;
+        if (ZSTD_eDist_compare(state, partition.dictMid, dictHigh,
+          partition.srcMid, srcHigh, partition.highUseHeuristics))
+            return 1;
+    }
+
+    return 0;
+}
+
+static int ZSTD_eDist_matchComp(const void* p, const void* q)
+{
+    S32 const l = ((ZSTD_eDist_match*)p)->srcIdx;
+    S32 const r = ((ZSTD_eDist_match*)q)->srcIdx;
+    return (l - r);
+}
+
+/* The matches from the approach above will all be of the form 
+ * (dictIdx, srcIdx, 1). this method combines contiguous matches 
+ * of length MINMATCH or greater. Matches less than MINMATCH 
+ * are discarded */ 
+static void ZSTD_eDist_combineMatches(ZSTD_eDist_state* state)
+{
+    /* Create a new buffer to put the combined matches into 
+     * and memcpy to state->matches after */ 
+    ZSTD_eDist_match* combinedMatches = 
+        ZSTD_malloc(state->nbMatches * sizeof(ZSTD_eDist_match), 
+        ZSTD_defaultCMem);
+
+    U32 nbCombinedMatches = 1;
+    size_t i;
+
+    /* Make sure that the srcIdx and dictIdx are in sorted order.
+     * The combination step won't work otherwise */ 
+    qsort(state->matches, state->nbMatches, sizeof(ZSTD_eDist_match), ZSTD_eDist_matchComp);
+
+    memcpy(combinedMatches, state->matches, sizeof(ZSTD_eDist_match));
+    for (i = 1; i < state->nbMatches; i++) {
+        ZSTD_eDist_match const match = state->matches[i];
+        ZSTD_eDist_match const combinedMatch = 
+            combinedMatches[nbCombinedMatches - 1];
+        if (combinedMatch.srcIdx + combinedMatch.matchLength == match.srcIdx && 
+          combinedMatch.dictIdx + combinedMatch.matchLength == match.dictIdx) {
+            combinedMatches[nbCombinedMatches - 1].matchLength++;
+        } else {
+            /* Discard matches that are less than MINMATCH */
+            if (combinedMatches[nbCombinedMatches - 1].matchLength < MINMATCH) {
+                nbCombinedMatches--;
+            }
+
+            memcpy(combinedMatches + nbCombinedMatches, 
+                state->matches + i, sizeof(ZSTD_eDist_match));
+            nbCombinedMatches++;
+        }
+    }
+    memcpy(state->matches, combinedMatches, nbCombinedMatches * sizeof(ZSTD_eDist_match));
+    state->nbMatches = nbCombinedMatches;
+    ZSTD_free(combinedMatches, ZSTD_defaultCMem);
+}
+
+static size_t ZSTD_eDist_convertMatchesToSequences(ZSTD_Sequence* sequences, 
+    ZSTD_eDist_state* state)
+{
+    const ZSTD_eDist_match* matches = state->matches;
+    size_t const nbMatches = state->nbMatches;
+    size_t const dictSize = state->dictSize;
+    size_t nbSequences = 0;
+    size_t i;
+    for (i = 0; i < nbMatches; i++) {
+        ZSTD_eDist_match const match = matches[i];
+        U32 const litLength = !i ? match.srcIdx : 
+            match.srcIdx - (matches[i - 1].srcIdx + matches[i - 1].matchLength);
+        U32 const offset = (match.srcIdx + dictSize) - match.dictIdx;
+        U32 const matchLength = match.matchLength;
+        sequences[nbSequences].offset = offset;
+        sequences[nbSequences].litLength = litLength;
+        sequences[nbSequences].matchLength = matchLength;
+        nbSequences++;
+    }
+    return nbSequences;
+}
+
+/*-*************************************
+*  Interal utils
+***************************************/
+
+static size_t ZSTD_eDist_hamingDist(const BYTE* const a,
+                        const BYTE* const b, size_t n)
+{
+    size_t i;
+    size_t dist = 0;
+    for (i = 0; i < n; i++)
+        dist += a[i] != b[i];
+    return dist; 
+}
+
+/* This is a pretty naive recursive implementation that should only
+ * be used for quick tests obviously. Don't try and run this on a 
+ * GB file or something. There are faster implementations. Use those
+ * if you need to run it for large files. */
+static size_t ZSTD_eDist_levenshteinDist(const BYTE* const s,
+                        size_t const sn, const BYTE* const t,
+                        size_t const tn)
+{
+    size_t a, b, c;
+
+    if (!sn)
+        return tn;
+    if (!tn)
+        return sn;
+    
+    if (s[sn - 1] == t[tn - 1])
+        return ZSTD_eDist_levenshteinDist(
+            s, sn - 1, t, tn - 1);
+    
+    a = ZSTD_eDist_levenshteinDist(s, sn - 1, t, tn - 1);
+    b = ZSTD_eDist_levenshteinDist(s, sn, t, tn - 1);
+    c = ZSTD_eDist_levenshteinDist(s, sn - 1, t, tn);
+
+    if (a > b)
+        a = b;
+    if (a > c)
+        a = c;
+    
+    return a + 1;
+}
+
+static void ZSTD_eDist_validateMatches(ZSTD_eDist_match* matches,
+                        size_t const nbMatches, const BYTE* const dict,
+                        size_t const dictSize, const BYTE* const src,
+                        size_t const srcSize)
+{
+    size_t i;
+    for (i = 0; i < nbMatches; i++) {
+        ZSTD_eDist_match match = matches[i];
+        U32 const dictIdx = match.dictIdx;
+        U32 const srcIdx = match.srcIdx;
+        U32 const matchLength = match.matchLength;
+        
+        assert(dictIdx + matchLength < dictSize);
+        assert(srcIdx + matchLength < srcSize);
+        assert(!memcmp(dict + dictIdx, src + srcIdx, matchLength));
+    }
+}
+
+/*-*************************************
+*  API
+***************************************/
+
+size_t ZSTD_eDist_genSequences(ZSTD_Sequence* sequences, 
+                        const void* dict, size_t dictSize,
+                        const void* src, size_t srcSize,
+                        int useHeuristics)
+{
+    size_t const nbDiags = dictSize + srcSize + 3;
+    S32* buffer = ZSTD_malloc(nbDiags * 2 * sizeof(S32), ZSTD_defaultCMem);
+    ZSTD_eDist_state state;
+    size_t nbSequences = 0;
+
+    state.dict = (const BYTE*)dict;
+    state.src = (const BYTE*)src;
+    state.dictSize = dictSize;
+    state.srcSize = srcSize;
+    state.forwardDiag = buffer;
+    state.backwardDiag = buffer + nbDiags;
+    state.forwardDiag += srcSize + 1;
+    state.backwardDiag += srcSize + 1;
+    state.matches = ZSTD_malloc(srcSize * sizeof(ZSTD_eDist_match), ZSTD_defaultCMem);
+    state.nbMatches = 0;
+
+    ZSTD_eDist_compare(&state, 0, dictSize, 0, srcSize, 1);
+    ZSTD_eDist_combineMatches(&state);
+    nbSequences = ZSTD_eDist_convertMatchesToSequences(sequences, &state);
+
+    ZSTD_free(buffer, ZSTD_defaultCMem);
+    ZSTD_free(state.matches, ZSTD_defaultCMem);
+
+    return nbSequences;
+}
@@ -0,0 +1,70 @@
+/*
+ * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+
+/* This match finder leverages techniques used in file comparison algorithms
+ * to find matches between a dictionary and a source file.
+ * 
+ * The original motivation for studying this approach was to try and optimize 
+ * Zstandard for the use case of patching: the most common scenario being 
+ * updating an existing software package with the next version. When patching,
+ * the difference between the old version of the package and the new version 
+ * is generally tiny (most of the new file will be identical to 
+ * the old one). In more technical terms, the edit distance (the minimal number 
+ * of changes required to take one sequence of bytes to another) between the 
+ * files would be small relative to the size of the file. 
+ * 
+ * Various 'diffing' algorithms utilize this notion of edit distance and 
+ * the corrensponding concept of a minimal edit script between two 
+ * sequences to identify the regions within two files where they differ. 
+ * The core algorithm used in this match finder is described in: 
+ * 
+ * "An O(ND) Difference Algorithm and its Variations", Eugene W. Myers,
+ *    Algorithmica Vol. 1, 1986, pp. 251-266,
+ *    <https://doi.org/10.1007/BF01840446>.
+ * 
+ * Additional algorithmic heuristics for speed improvement have also been included.
+ * These we inspired from implementations of various regular and binary diffing 
+ * algorithms such as GNU diff, bsdiff, and Xdelta. 
+ * 
+ * Note: after some experimentation, this approach proved to not provide enough 
+ * utility to justify the additional CPU used in finding matches. The one area
+ * where this approach consistenly outperforms Zstandard even on level 19 is 
+ * when compressing small files (<10 KB) using a equally small dictionary that 
+ * is very similar to the source file. For the use case that this was intended,
+ * (large similar files) this approach by itself took 5-10X longer than zstd-19 and 
+ * generally resulted in 2-3X larger files. The core advantage that zstd-19 has 
+ * over this appraoch for match finding is the overlapping matches. This approach 
+ * cannot find any. 
+ * 
+ * I'm leaving this in the contrib section in case this ever becomes interesting 
+ * to explore again.
+ * */
+
+#ifndef ZSTD_EDIST_H
+#define ZSTD_EDIST_H
+
+/*-*************************************
+*  Dependencies
+***************************************/
+
+#include <stddef.h>
+#include "zstd_internal.h" /* ZSTD_Sequence */
+
+/*! ZSTD_eDist_genSequences() : 
+ * Will populate the provided ZSTD_Sequence buffer with sequences 
+ * based on the optimal or near-optimal (depending on 'useHeuristics')
+ * edit script between 'dict' and 'src.' 
+ * @return : the number of sequences found */ 
+size_t ZSTD_eDist_genSequences(ZSTD_Sequence* sequences, 
+                        const void* dict, size_t dictSize,
+                        const void* src, size_t srcSize,
+                        int useHeuristics);
+
+#endif
@@ -13,10 +13,10 @@

 namespace pzstd {

-constexpr int ERROR = 1;
-constexpr int INFO = 2;
-constexpr int DEBUG = 3;
-constexpr int VERBOSE = 4;
+constexpr int kLogError = 1;
+constexpr int kLogInfo = 2;
+constexpr int kLogDebug = 3;
+constexpr int kLogVerbose = 4;

 class Logger {
  std::mutex mutex_;
@@ -94,12 +94,12 @@ static std::uint64_t handleOneInput(const Options &options,
    if (!options.decompress) {
      double ratio = static_cast<double>(bytesWritten) /
                     static_cast<double>(bytesRead + !bytesRead);
-      state.log(INFO, "%-20s :%6.2f%%   (%6" PRIu64 " => %6" PRIu64
+      state.log(kLogInfo, "%-20s :%6.2f%%   (%6" PRIu64 " => %6" PRIu64
                   " bytes, %s)\n",
                   inputFileName.c_str(), ratio * 100, bytesRead, bytesWritten,
                   outputFileName.c_str());
    } else {
-      state.log(INFO, "%-20s: %" PRIu64 " bytes \n",
+      state.log(kLogInfo, "%-20s: %" PRIu64 " bytes \n",
                   inputFileName.c_str(),bytesWritten);
    }
  }
@@ -139,12 +139,12 @@ static FILE *openOutputFile(const Options &options,
    auto outputFd = std::fopen(outputFile.c_str(), "rb");
    if (outputFd != nullptr) {
      std::fclose(outputFd);
-      if (!state.log.logsAt(INFO)) {
+      if (!state.log.logsAt(kLogInfo)) {
        state.errorHolder.setError("Output file exists");
        return nullptr;
      }
      state.log(
-          INFO,
+          kLogInfo,
          "pzstd: %s already exists; do you wish to overwrite (y/n) ? ",
          outputFile.c_str());
      int c = getchar();
@@ -170,7 +170,7 @@ int pzstdMain(const Options &options) {
    auto printErrorGuard = makeScopeGuard([&] {
      if (state.errorHolder.hasError()) {
        returnCode = 1;
-        state.log(ERROR, "pzstd: %s: %s.\n", input.c_str(),
+        state.log(kLogError, "pzstd: %s: %s.\n", input.c_str(),
                  state.errorHolder.getError().c_str());
      }
    });
@@ -388,7 +388,7 @@ std::uint64_t asyncCompressChunks(
  // Break the input up into chunks of size `step` and compress each chunk
  // independently.
  size_t step = calculateStep(size, numThreads, params);
-  state.log(DEBUG, "Chosen frame size: %zu\n", step);
+  state.log(kLogDebug, "Chosen frame size: %zu\n", step);
  auto status = FileStatus::Continue;
  while (status == FileStatus::Continue && !state.errorHolder.hasError()) {
    // Make a new input queue that we will put the chunk's input data into.
@@ -403,7 +403,7 @@ std::uint64_t asyncCompressChunks(
    });
    // Pass the output queue to the writer thread.
    chunks.push(std::move(out));
-    state.log(VERBOSE, "%s\n", "Starting a new frame");
+    state.log(kLogVerbose, "%s\n", "Starting a new frame");
    // Fill the input queue for the compression job we just started
    status = readData(*in, ZSTD_CStreamInSize(), step, fd, &bytesRead);
  }
@@ -540,14 +540,14 @@ std::uint64_t asyncDecompressFrames(
    if (frameSize == 0) {
      // We hit a non SkippableFrame ==> not compressed by pzstd or corrupted
      // Pass the rest of the source to this decompression task
-      state.log(VERBOSE, "%s\n",
+      state.log(kLogVerbose, "%s\n",
          "Input not in pzstd format, falling back to serial decompression");
      while (status == FileStatus::Continue && !state.errorHolder.hasError()) {
        status = readData(*in, chunkSize, chunkSize, fd, &totalBytesRead);
      }
      break;
    }
-    state.log(VERBOSE, "Decompressing a frame of size %zu", frameSize);
+    state.log(kLogVerbose, "Decompressing a frame of size %zu", frameSize);
    // Fill the input queue for the decompression job we just started
    status = readData(*in, chunkSize, frameSize, fd, &totalBytesRead);
  }
@@ -573,7 +573,7 @@ std::uint64_t writeFile(
    bool decompress) {
  auto& errorHolder = state.errorHolder;
  auto lineClearGuard = makeScopeGuard([&state] {
-    state.log.clear(INFO);
+    state.log.clear(kLogInfo);
  });
  std::uint64_t bytesWritten = 0;
  std::shared_ptr<BufferWorkQueue> out;
@@ -602,7 +602,7 @@ std::uint64_t writeFile(
        return bytesWritten;
      }
      bytesWritten += buffer.size();
-      state.log.update(INFO, "Written: %u MB   ",
+      state.log.update(kLogInfo, "Written: %u MB   ",
                static_cast<std::uint32_t>(bytesWritten >> 20));
    }
  }
@@ -41,7 +41,7 @@ class SharedState {
      auto parameters = options.determineParameters();
      cStreamPool.reset(new ResourcePool<ZSTD_CStream>{
          [this, parameters]() -> ZSTD_CStream* {
-            this->log(VERBOSE, "%s\n", "Creating new ZSTD_CStream");
+            this->log(kLogVerbose, "%s\n", "Creating new ZSTD_CStream");
            auto zcs = ZSTD_createCStream();
            if (zcs) {
              auto err = ZSTD_initCStream_advanced(
@@ -59,7 +59,7 @@ class SharedState {
    } else {
      dStreamPool.reset(new ResourcePool<ZSTD_DStream>{
          [this]() -> ZSTD_DStream* {
-            this->log(VERBOSE, "%s\n", "Creating new ZSTD_DStream");
+            this->log(kLogVerbose, "%s\n", "Creating new ZSTD_DStream");
            auto zds = ZSTD_createDStream();
            if (zds) {
              auto err = ZSTD_initDStream(zds);
@@ -1,2 +0,0 @@
-zstddeclib.c
-
@@ -1,18 +0,0 @@
-# Single File Zstandard Decompression Library
-
-The script `combine.sh` creates an _amalgamated_ source file that can be used with or without `zstd.h`. This isn't a _header-only_ file but it does offer a similar level of simplicity when integrating into a project.
-
-Create `zstddeclib.c` from the Zstd source using:
-```
-cd zstd/contrib/declib
-./combine.sh -r ../../lib -r ../../lib/common -r ../../lib/decompress -o zstddeclib.c zstddeclib-in.c
-```
-Then add the resulting file to your project (see the [example files](examples)).
-
-`create_single_file_decoder.sh` will run the above script, creating file `zstddeclib.c`.
-`build_test.sh` will create the decoder, then compile and test the library.
-
-Why?
----
-
-Because all it now takes to support decompressing Zstd is the addition of a single file, two if using the header, with no configuration or further build steps. The library is small, adding, for example, 26kB to an Emscripten compiled WebAssembly project. Native implementations add a little more, 40-70kB depending on the compiler and platform.
@@ -1,58 +0,0 @@
-#!/bin/sh
-
-# Where to find the sources
-ZSTD_SRC_ROOT="../../lib"
-
-# Temporary compiled binary
-OUT_FILE="tempbin"
-
-# Optional temporary compiled WebAssembly
-OUT_WASM="temp.wasm"
-
-# Amalgamate the sources
-./create_single_file_decoder.sh
-# Did combining work?
-if [ $? -ne 0 ]; then
-  echo "Single file decoder creation script: FAILED"
-  exit 1
-fi
-echo "Single file decoder creation script: PASSED"
-
-# Compile the generated output
-cc -Wall -Wextra -Werror -Os -g0 -o $OUT_FILE examples/simple.c
-# Did compilation work?
-if [ $? -ne 0 ]; then
-  echo "Compiling simple.c: FAILED"
-  exit 1
-fi
-echo "Compiling simple.c: PASSED"
-
-# Run then delete the compiled output
-./$OUT_FILE
-retVal=$?
-rm -f $OUT_FILE
-# Did the test work?
-if [ $retVal -ne 0 ]; then
-  echo "Running simple.c: FAILED"
-  exit 1
-fi
-echo "Running simple.c: PASSED"
-
-# Is Emscripten available?
-which emcc > /dev/null
-if [ $? -ne 0 ]; then
-  echo "(Skipping Emscripten test)"
-else
-  # Compile the Emscripten example
-  CC_FLAGS="-Wall -Wextra -Werror -Os -g0 -flto --llvm-lto 3 -lGL -DNDEBUG=1"
-  emcc $CC_FLAGS -s WASM=1 -o $OUT_WASM examples/emscripten.c
-  # Did compilation work?
-  if [ $? -ne 0 ]; then
-    echo "Compiling emscripten.c: FAILED"
-    exit 1
-  fi
-  echo "Compiling emscripten.c: PASSED"
-  rm -f $OUT_WASM
-fi
-
-exit 0
@@ -1,124 +0,0 @@
-#!/bin/sh -e
-
-# Tool to bundle multiple C/C++ source files, inlining any includes.
-# 
-# Note: this POSIX-compliant script is many times slower than the original bash
-# implementation (due to the grep calls) but it runs and works everywhere.
-# 
-# TODO: ROOTS and FOUND as arrays (since they fail on paths with spaces)
-# 
-# Author: Carl Woffenden, Numfum GmbH (released under a CC0 license)
-
-# Common file roots
-ROOTS="./"
-
-# Files previously visited
-FOUND=""
-
-# Optional destination file (empty string to write to stdout)
-DESTN=""
-
-# Prints the script usage then exits
-usage() {
-  echo "Usage: $0 [-r <path>] [-o <outfile>] infile"
-  echo "  -r file root search paths"
-  echo "  -o output file (otherwise stdout)"
-  echo "Example: $0 -r ../my/path - r ../other/path -o out.c in.c"
-  exit 1
-}
-
-# Tests that the grep implementation works as expected (older OSX grep fails)
-test_grep() {
-	if ! echo '#include "foo"' | grep -Eq '^\s*#\s*include\s*".+"'; then
-		echo "Aborting: the grep implementation fails to parse include lines"
-		exit 1
-	fi
-}
-
-# Tests if list $1 has item $2 (returning zero on a match)
-list_has_item() {
-  if echo "$1" | grep -Eq "(^|\s*)$2(\$|\s*)"; then
-    return 0
-  else
-    return 1
-  fi
-}
-
-# Adds a new line with the supplied arguments to $DESTN (or stdout)
-write_line() {
-  if [ -n "$DESTN" ]; then
-    printf '%s\n' "$@" >> "$DESTN"
-  else
-    printf '%s\n' "$@"
-  fi
-}
-
-# Adds the contents of $1 with any of its includes inlined
-add_file() {
-  # Match the path
-  local file=
-  if [ -f "$1" ]; then
-    file="$1"
-  else
-    for root in $ROOTS; do
-      if test -f "$root/$1"; then
-        file="$root/$1"
-      fi
-    done
-  fi
-  if [ -n "$file" ]; then
-    # Read the file
-    local line=
-    while IFS= read -r line; do
-      if echo "$line" | grep -Eq '^\s*#\s*include\s*".+"'; then
-        # We have an include directive so strip the (first) file
-        local inc=$(echo "$line" | grep -Eo '".*"' | grep -Eo '\w*(\.?\w+)+' | head -1)
-        if ! list_has_item "$FOUND" "$inc"; then
-          # And we've not previously encountered it
-          FOUND="$FOUND $inc"
-          write_line "/**** start inlining $inc ****/"
-          add_file "$inc"
-          write_line "/**** ended inlining $inc ****/"
-        else
-          write_line "/**** skipping file: $inc ****/"
-        fi
-      else
-        # Otherwise write the source line
-        write_line "$line"
-      fi
-    done < "$file"
-  else
-    write_line "#error Unable to find \"$1\""
-  fi
-}
-
-while getopts ":r:o:" opts; do
-  case $opts in
-  r)
-    ROOTS="$OPTARG $ROOTS"
-    ;;
-  o)
-    DESTN="$OPTARG"
-    ;;
-  *)
-    usage
-    ;;
-  esac
-done
-shift $((OPTIND-1))
-
-if [ -n "$1" ]; then
-  if [ -f "$1" ]; then
-    if [ -n "$DESTN" ]; then
-      printf "" > "$DESTN"
-    fi
-    test_grep
-    add_file $1
-  else
-    echo "Input file not found: \"$1\""
-    exit 1
-  fi
-else
-  usage
-fi
-exit 0
@@ -1,600 +0,0 @@
-/**
- * \file emscripten.c
- * Emscripten example of using the single-file \c zstddeclib. Draws a rotating
- * textured quad with data from the in-line Zstd compressed DXT1 texture (DXT1
- * being hardware compression, further compressed with Zstd).
- * \n
- * Compile using:
- * \code
- *	export CC_FLAGS="-Wall -Wextra -Werror -Os -g0 -flto --llvm-lto 3 -lGL -DNDEBUG=1"
- *	export EM_FLAGS="-s WASM=1 -s ENVIRONMENT=web --shell-file shell.html --closure 1"
- *	emcc $CC_FLAGS $EM_FLAGS -o out.html emscripten.c
- * \endcode
- * 
- * \author Carl Woffenden, Numfum GmbH (released under a CC0 license)
- */
-
-#include <stddef.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-#include <emscripten/emscripten.h>
-#include <emscripten/html5.h>
-
-#include <GLES2/gl2.h>
-#include <GLES2/gl2ext.h>
-
-#include "../zstddeclib.c"
-
-//************************* Test Data (DXT texture) **************************/
-
-/**
- * Zstd compressed DXT1 256x256 texture source.
- * \n
- * See \c testcard.png for the original.
- */
-static uint8_t const srcZstd[] = {
-	0x28, 0xb5, 0x2f, 0xfd, 0x60, 0x00, 0x7f, 0x6d, 0x61, 0x00, 0x0a, 0x66,
-	0xec, 0x17, 0x48, 0x60, 0x1c, 0x5a, 0xc9, 0x5d, 0x1a, 0x38, 0x07, 0xe8,
-	0xc5, 0x82, 0x99, 0x68, 0xe6, 0x95, 0x45, 0x58, 0x0d, 0x0c, 0xf3, 0x36,
-	0xc8, 0xd9, 0x0f, 0x46, 0x2d, 0x68, 0x11, 0xf8, 0x31, 0x10, 0xa1, 0x1a,
-	0x2f, 0x99, 0x5c, 0x84, 0xfd, 0x92, 0x02, 0xe6, 0x3b, 0x44, 0x9b, 0x01,
-	0x5d, 0x92, 0xff, 0x38, 0x26, 0x00, 0x6a, 0x6b, 0xc3, 0x53, 0xb2, 0x0c,
-	0x25, 0xf3, 0xd8, 0x59, 0x68, 0x9b, 0x14, 0x8a, 0x89, 0x75, 0x18, 0x03,
-	0x1d, 0xc9, 0x0f, 0x63, 0x01, 0x73, 0x01, 0x72, 0x01, 0x4f, 0x66, 0x31,
-	0x58, 0x0f, 0x97, 0x4b, 0x0c, 0x4c, 0x06, 0xac, 0x07, 0x0b, 0x68, 0xd4,
-	0xad, 0x80, 0x64, 0x13, 0x74, 0xa1, 0x12, 0x16, 0x58, 0xcf, 0x1a, 0x95,
-	0x5f, 0x0d, 0x26, 0x55, 0xd0, 0x9c, 0xf4, 0x52, 0x35, 0x2e, 0x20, 0xc1,
-	0x06, 0x69, 0x03, 0x0a, 0x93, 0x83, 0x5e, 0x27, 0x9b, 0x4c, 0x6d, 0xee,
-	0x87, 0x03, 0x30, 0x6c, 0x46, 0xd7, 0x50, 0x5c, 0xca, 0xe6, 0xa6, 0x4d,
-	0xa8, 0xf6, 0xab, 0xd7, 0x0e, 0x27, 0x27, 0x90, 0xc4, 0xb2, 0xd1, 0x10,
-	0xfa, 0x43, 0x82, 0xc8, 0xf2, 0xe5, 0xff, 0xff, 0xd5, 0x52, 0x62, 0x43,
-	0x87, 0x26, 0x2a, 0x05, 0x70, 0x0e, 0xb0, 0x2f, 0xc4, 0x56, 0xef, 0xb5,
-	0xca, 0xb8, 0x53, 0xb7, 0x96, 0x0e, 0xe7, 0x00, 0x2c, 0xa8, 0xda, 0x3b,
-	0x07, 0x70, 0xa7, 0x78, 0x38, 0x60, 0x87, 0x7a, 0x01, 0x3b, 0x75, 0xec,
-	0xfa, 0x77, 0xe2, 0x46, 0x94, 0x61, 0x8e, 0x0d, 0x0c, 0xfb, 0xe7, 0x8b,
-	0x13, 0x50, 0x31, 0xa9, 0x27, 0xcd, 0x27, 0xef, 0x6b, 0xa6, 0xab, 0x9c,
-	0x4d, 0x95, 0x6c, 0x3a, 0xbb, 0x8e, 0x96, 0x92, 0x18, 0x5a, 0x7c, 0x4f,
-	0xff, 0x7b, 0x38, 0xf2, 0xdb, 0x86, 0xde, 0xff, 0x1f, 0x2f, 0x21, 0x86,
-	0x7d, 0xbf, 0x45, 0xd0, 0x6e, 0x77, 0x0a, 0xee, 0x0a, 0xee, 0x14, 0x9a,
-	0xb8, 0x84, 0xf3, 0xac, 0xbe, 0xc8, 0x7f, 0x8d, 0xff, 0xff, 0xcf, 0x2a,
-	0xfb, 0x69, 0xfc, 0xfb, 0xfd, 0x7a, 0x10, 0x22, 0x36, 0xfc, 0xff, 0x3f,
-	0xcf, 0xd0, 0xf1, 0x7f, 0xfe, 0xff, 0x3d, 0x24, 0xdf, 0x78, 0x4a, 0xff,
-	0xda, 0x9c, 0x39, 0xcf, 0xef, 0xe7, 0xfd, 0x52, 0x98, 0xb5, 0x40, 0x92,
-	0xee, 0xdd, 0x99, 0xf5, 0x53, 0x5b, 0x65, 0x6b, 0xb5, 0xd8, 0x7b, 0xae,
-	0xfa, 0xc1, 0x0f, 0x0c, 0x7f, 0x4f, 0x55, 0xa3, 0xad, 0x2c, 0xa0, 0xbd,
-	0xf7, 0x2a, 0x0e, 0xe8, 0xbd, 0xc7, 0x5e, 0xf5, 0xd8, 0x54, 0x9e, 0x56,
-	0xa3, 0xd6, 0x59, 0xd5, 0xfe, 0x1f, 0xc0, 0x30, 0x8c, 0xfc, 0x46, 0x04,
-	0xae, 0x60, 0xbc, 0xe8, 0xcf, 0xec, 0x3d, 0xde, 0xf9, 0xf0, 0xfe, 0xef,
-	0x7d, 0xcc, 0xf7, 0x2b, 0xe5, 0x1b, 0x70, 0xff, 0xff, 0x7e, 0x3f, 0x6e,
-	0xe4, 0x02, 0x07, 0xfc, 0x1b, 0x7a, 0xff, 0xe7, 0x58, 0xfc, 0x7e, 0x3a,
-	0xdc, 0x97, 0xfd, 0x57, 0xef, 0xa3, 0xfc, 0x2a, 0xc7, 0x4d, 0xf3, 0xcb,
-	0x9d, 0xce, 0xac, 0xfe, 0xeb, 0x2e, 0x86, 0xb9, 0x69, 0x54, 0xef, 0xf9,
-	0x55, 0xcf, 0xff, 0x48, 0x24, 0x72, 0x3a, 0x9d, 0x72, 0x2f, 0x2f, 0x2f,
-	0xff, 0x3f, 0xe7, 0x01, 0x6c, 0x4d, 0x6c, 0xcd, 0x2d, 0x5b, 0x53, 0xb7,
-	0x59, 0x22, 0x08, 0x0b, 0xa7, 0x92, 0x15, 0x75, 0x93, 0xb0, 0x5d, 0xaf,
-	0x2a, 0x63, 0x95, 0x1d, 0x25, 0xd2, 0xd2, 0xa8, 0x1c, 0x84, 0xc9, 0xdc,
-	0x72, 0xba, 0xd7, 0xfc, 0x69, 0xf5, 0xc7, 0x19, 0xa9, 0xbe, 0xfa, 0x26,
-	0x55, 0x25, 0x75, 0xb7, 0x60, 0xa3, 0xd8, 0x68, 0x54, 0xb7, 0x1b, 0xa5,
-	0x54, 0x62, 0xb1, 0x49, 0xde, 0xe2, 0xac, 0xa2, 0xe8, 0x7b, 0xff, 0x5f,
-	0x75, 0x4e, 0xb8, 0xa2, 0xdd, 0x6a, 0xb7, 0xda, 0x6e, 0x2e, 0x04, 0xcd,
-	0x08, 0x2f, 0xec, 0x8e, 0x49, 0xaf, 0x49, 0x6f, 0x8b, 0x4f, 0x2e, 0x1a,
-	0xc5, 0x62, 0x7b, 0x6b, 0x3e, 0x32, 0x3e, 0x32, 0xbe, 0x08, 0x35, 0x4d,
-	0x63, 0x93, 0xa6, 0xc8, 0x42, 0xe6, 0x21, 0xcc, 0x59, 0xc8, 0x4c, 0xe5,
-	0x86, 0xe1, 0x03, 0x06, 0xa4, 0xec, 0xff, 0xb7, 0x78, 0x7e, 0x62, 0x43,
-	0xc7, 0x2c, 0x50, 0x30, 0x4a, 0xc8, 0x9b, 0xf3, 0xbf, 0xe6, 0x62, 0xa0,
-	0x50, 0xa6, 0x9c, 0xe3, 0x6e, 0x5b, 0xaf, 0x77, 0x8b, 0xbb, 0xe1, 0x70,
-	0xaa, 0xaa, 0xaa, 0x92, 0xb4, 0x52, 0xad, 0x14, 0x87, 0x93, 0x0b, 0xe6,
-	0x82, 0x39, 0x11, 0xb9, 0x20, 0x9a, 0x16, 0x34, 0x22, 0x68, 0xb2, 0x68,
-	0xb2, 0x76, 0xd3, 0xe8, 0x6e, 0xda, 0x6b, 0x62, 0x34, 0x2e, 0x8d, 0xbd,
-	0x2d, 0x3d, 0x6b, 0x4c, 0x26, 0x33, 0xda, 0x33, 0xf3, 0x91, 0x51, 0x46,
-	0x79, 0xbd, 0x3e, 0x39, 0x9f, 0xcf, 0xd2, 0xb8, 0x5c, 0xfa, 0x22, 0xf8,
-	0x8e, 0xb6, 0xbe, 0x08, 0x40, 0x14, 0x49, 0x40, 0x14, 0xf2, 0x0c, 0x2d,
-	0x30, 0x85, 0x3c, 0x63, 0x29, 0xd3, 0x98, 0x85, 0x6c, 0xb5, 0xdb, 0xad,
-	0x5c, 0x63, 0x9e, 0x72, 0xcf, 0x43, 0xe6, 0xaf, 0x77, 0xa6, 0xe2, 0x21,
-	0x0c, 0x4d, 0xd3, 0x49, 0x1e, 0xc2, 0x14, 0x6f, 0xee, 0xdb, 0x7b, 0x7b,
-	0x08, 0xb3, 0xa4, 0x60, 0x3b, 0x9d, 0x6e, 0x8b, 0x37, 0x4b, 0x0a, 0x74,
-	0x35, 0x33, 0xbc, 0xf9, 0x64, 0x85, 0x63, 0x32, 0x29, 0x20, 0x59, 0x0c,
-	0x3c, 0x96, 0x67, 0x62, 0xb7, 0x8a, 0x92, 0x4d, 0xa0, 0xd3, 0xf3, 0xd1,
-	0x85, 0x80, 0x38, 0xcb, 0x64, 0x60, 0xc9, 0xb5, 0xaf, 0x97, 0x8d, 0x20,
-	0x45, 0x28, 0xb8, 0xab, 0xe8, 0xc9, 0x0a, 0x88, 0x1f, 0xd6, 0x47, 0x54,
-	0xf1, 0xd3, 0xfb, 0x62, 0xa7, 0xfd, 0xf2, 0x8b, 0xfd, 0xb6, 0xe4, 0x2e,
-	0xb6, 0x91, 0x73, 0x1c, 0xd0, 0x7b, 0xba, 0x83, 0xc9, 0xac, 0x51, 0x39,
-	0x92, 0xc5, 0x4f, 0x30, 0x1e, 0x2e, 0xd5, 0xf1, 0xa8, 0xa6, 0xa5, 0x80,
-	0x70, 0xb9, 0xbc, 0xb7, 0xc2, 0x52, 0x32, 0x6c, 0xe3, 0x3d, 0xed, 0x41,
-	0xa4, 0x4b, 0x31, 0x2a, 0xe6, 0x62, 0x11, 0x19, 0x95, 0x73, 0x1d, 0xbf,
-	0xe1, 0x6c, 0xfc, 0x47, 0x75, 0x6c, 0x37, 0x63, 0x02, 0xf8, 0x34, 0x40,
-	0x9a, 0x00, 0x1d, 0xf7, 0x32, 0x56, 0x77, 0xda, 0x5b, 0x9f, 0x9f, 0x0f,
-	0xbb, 0x91, 0x5b, 0xbd, 0xe7, 0x58, 0x82, 0x4a, 0x20, 0xcd, 0x4f, 0x47,
-	0x15, 0xf3, 0x51, 0xf1, 0x43, 0x51, 0x10, 0x96, 0xae, 0xba, 0xf7, 0x21,
-	0x50, 0xef, 0x55, 0x27, 0x0c, 0x1f, 0xe0, 0x54, 0xf8, 0xc9, 0x69, 0xef,
-	0xb9, 0x53, 0xf7, 0x83, 0x73, 0x9d, 0xce, 0x86, 0x07, 0x83, 0x44, 0x61,
-	0x37, 0x35, 0x35, 0x33, 0x4e, 0x33, 0x33, 0x3e, 0x9f, 0x50, 0x48, 0x24,
-	0xe6, 0xd0, 0x79, 0x49, 0xc3, 0x2d, 0xa0, 0x46, 0x31, 0x9a, 0x72, 0xc3,
-	0x84, 0xff, 0x7a, 0x95, 0xbb, 0x00, 0x22, 0xcc, 0x14, 0x00, 0x04, 0xac,
-	0x60, 0x64, 0x86, 0xe4, 0x6f, 0xb1, 0x58, 0xf4, 0xdc, 0xb0, 0x05, 0x00,
-	0x72, 0x38, 0xf8, 0xce, 0xce, 0x8e, 0xcf, 0x37, 0x33, 0x43, 0x24, 0x0a,
-	0x85, 0x33, 0x35, 0x35, 0x37, 0xc2, 0xa8, 0x28, 0x27, 0x1b, 0x9d, 0xce,
-	0x29, 0x18, 0xe4, 0xfc, 0x09, 0x53, 0xa5, 0x51, 0xad, 0x74, 0x79, 0x7b,
-	0xb5, 0x4a, 0x65, 0x94, 0x36, 0x89, 0xf5, 0x62, 0x9b, 0xd8, 0x9a, 0xe8,
-	0x2b, 0xff, 0xa1, 0x73, 0xfe, 0x2e, 0x2c, 0x41, 0x45, 0x37, 0xef, 0x6e,
-	0x7b, 0x38, 0xca, 0xa5, 0xd2, 0xb8, 0x1c, 0x3b, 0x96, 0x21, 0xbb, 0x5d,
-	0xad, 0xd1, 0xdb, 0x1c, 0xf6, 0x5e, 0x4b, 0xd9, 0x59, 0x1b, 0x67, 0xf5,
-	0x7a, 0x7c, 0x9a, 0x91, 0x3e, 0x8e, 0xe3, 0xee, 0x7b, 0xb9, 0xa4, 0xb9,
-	0xf5, 0x70, 0xee, 0x1d, 0x4e, 0x4f, 0xcc, 0xd6, 0x7b, 0x07, 0x71, 0x48,
-	0xf2, 0x06, 0xd0, 0x10, 0x82, 0x21, 0xe4, 0x55, 0x2e, 0xa5, 0x1d, 0xbe,
-	0x48, 0x8c, 0x69, 0xbb, 0x24, 0x40, 0x68, 0x9b, 0xba, 0x5a, 0x5a, 0xa7,
-	0xe2, 0xd1, 0xac, 0xc2, 0xd8, 0x87, 0x9c, 0xe3, 0x78, 0xee, 0xa6, 0xcd,
-	0xdd, 0x68, 0x6e, 0xdd, 0xdb, 0x2e, 0xc6, 0xbb, 0x8b, 0xe9, 0xc1, 0xd9,
-	0xf6, 0x62, 0x7e, 0x72, 0x7e, 0x72, 0x34, 0xe4, 0x68, 0xc8, 0x11, 0x32,
-	0x10, 0x32, 0x20, 0x32, 0xf0, 0x12, 0x19, 0x90, 0xe0, 0x45, 0x91, 0xe0,
-	0x25, 0x83, 0xba, 0xc9, 0x20, 0x26, 0x87, 0xa5, 0x72, 0xa9, 0x64, 0x72,
-	0x80, 0x21, 0x54, 0x13, 0xeb, 0x29, 0x18, 0x42, 0x34, 0x84, 0x94, 0x34,
-	0x84, 0xa4, 0x1d, 0xa4, 0x1d, 0x82, 0x1c, 0xef, 0xaf, 0x0e, 0x63, 0x47,
-	0x6d, 0x10, 0xb9, 0xec, 0xd8, 0x2d, 0x3b, 0x9e, 0x21, 0xb1, 0x67, 0x48,
-	0x34, 0x24, 0x1a, 0x52, 0xdb, 0xa4, 0x6d, 0x62, 0xd3, 0xfa, 0xff, 0xfa,
-	0x03, 0x34, 0xef, 0x9d, 0xc9, 0xe1, 0x5d, 0xec, 0xf5, 0xf1, 0x79, 0x8c,
-	0x97, 0xd2, 0x24, 0xc1, 0x2d, 0xe0, 0x39, 0x16, 0x1e, 0xa9, 0x41, 0xc3,
-	0xbf, 0x4a, 0xd9, 0x3c, 0xea, 0x77, 0x96, 0x55, 0xe6, 0x95, 0xc3, 0xf1,
-	0x8e, 0x7b, 0x4f, 0xad, 0x61, 0xf8, 0xe7, 0x01, 0xad, 0x46, 0xf5, 0x2c,
-	0xac, 0x55, 0x2c, 0x94, 0xaa, 0x46, 0xfb, 0x5e, 0xcd, 0xaa, 0x1f, 0x78,
-	0x4f, 0x2f, 0xd1, 0xc9, 0x02, 0xd6, 0x2c, 0x67, 0xef, 0x3f, 0x54, 0xab,
-	0xda, 0x03, 0x79, 0x1f, 0xab, 0xfd, 0x0c, 0x38, 0x3c, 0xbc, 0xe1, 0xd5,
-	0x01, 0xf6, 0xfb, 0xfb, 0xf1, 0x70, 0xee, 0xfd, 0x90, 0x13, 0x97, 0xc4,
-	0xbc, 0x08, 0xe7, 0x4b, 0x88, 0x34, 0xf7, 0x56, 0x1e, 0x0c, 0xdb, 0xe4,
-	0x9c, 0x78, 0xf1, 0xf4, 0x62, 0x4c, 0xb5, 0xf7, 0xdd, 0xd9, 0x4c, 0x5a,
-	0x69, 0xa6, 0x36, 0x27, 0x03, 0xbe, 0x86, 0xc2, 0x72, 0xa2, 0x60, 0x73,
-	0xf1, 0xe0, 0x17, 0x50, 0xb5, 0x93, 0x81, 0xac, 0xf1, 0xc9, 0xd4, 0x66,
-	0x73, 0x71, 0xce, 0x63, 0xa8, 0x60, 0x98, 0xda, 0x86, 0x46, 0x72, 0xec,
-	0x54, 0xc1, 0xe6, 0x8a, 0x10, 0x23, 0x2d, 0x0c, 0xdd, 0x44, 0x0b, 0xa0,
-	0x44, 0xa4, 0x9d, 0x0e, 0x64, 0x31, 0x30, 0x45, 0xb1, 0x97, 0x4c, 0x6d,
-	0x9f, 0x43, 0x99, 0x71, 0xa8, 0x9a, 0xe6, 0xbd, 0x3a, 0xe1, 0xff, 0x7f,
-	0x33, 0x61, 0xf1, 0x48, 0xda, 0x48, 0x28, 0x10, 0x23, 0x9b, 0x24, 0xeb,
-	0xee, 0xbd, 0x35, 0x0e, 0x6e, 0x75, 0x23, 0x20, 0xd6, 0x95, 0x8c, 0xa5,
-	0x24, 0xec, 0x44, 0x67, 0x52, 0x2e, 0x78, 0x2a, 0xba, 0x3e, 0x3a, 0x4e,
-	0xc9, 0x5c, 0x23, 0xb0, 0xd5, 0xfb, 0x29, 0xaf, 0x9a, 0x0b, 0x90, 0x89,
-	0xd8, 0xec, 0xa9, 0xa8, 0x13, 0xfc, 0x22, 0xfa, 0xf2, 0x74, 0x2f, 0x4e,
-	0x35, 0xb0, 0x6d, 0x6c, 0xfd, 0xc4, 0xfe, 0xd0, 0x98, 0x3d, 0xe5, 0x43,
-	0x0a, 0xd0, 0x33, 0x26, 0x3b, 0x12, 0x7d, 0x65, 0xa1, 0xff, 0xff, 0x6f,
-	0x53, 0x0e, 0x28, 0x84, 0xa7, 0xa2, 0x2e, 0xf8, 0x4a, 0xb6, 0xa1, 0x47,
-	0xf5, 0xd0, 0x13, 0x9d, 0xd9, 0x50, 0xef, 0x9f, 0x31, 0xb4, 0x13, 0x67,
-	0xf9, 0x0a, 0x99, 0xb5, 0x80, 0xec, 0x4a, 0x1a, 0x59, 0x21, 0x3b, 0xce,
-	0xc5, 0x7e, 0x96, 0x85, 0x92, 0xc5, 0xb0, 0x75, 0xaa, 0x41, 0x16, 0xa9,
-	0xd3, 0xde, 0x13, 0x7d, 0xd9, 0x61, 0x30, 0x1c, 0x73, 0x61, 0x54, 0x90,
-	0xcf, 0xd4, 0xe8, 0xfe, 0xbf, 0xbf, 0x36, 0x57, 0x26, 0x38, 0xab, 0x06,
-	0xc4, 0x7e, 0x3c, 0x5b, 0x35, 0xd2, 0x7c, 0x2c, 0x0a, 0x82, 0xf2, 0xa8,
-	0xf2, 0x8b, 0x48, 0xa9, 0x90, 0x00, 0x01, 0x10, 0x10, 0x14, 0x04, 0x00,
-	0x32, 0xa2, 0x06, 0x82, 0x28, 0x90, 0xc2, 0xb4, 0x85, 0xda, 0x01, 0x52,
-	0xe9, 0x18, 0x85, 0x60, 0x00, 0x00, 0x20, 0x0c, 0x00, 0x14, 0x41, 0x00,
-	0x40, 0xa0, 0x04, 0x40, 0x00, 0x80, 0x50, 0x03, 0x01, 0x10, 0x18, 0x01,
-	0x80, 0xd4, 0x14, 0x99, 0x01, 0xfd, 0x07, 0xf8, 0x16, 0x0e, 0xd9, 0x5d,
-	0xa3, 0x70, 0xfe, 0xda, 0x17, 0xfa, 0xce, 0x46, 0x9a, 0x99, 0x81, 0x1a,
-	0x39, 0xba, 0x63, 0xb1, 0x18, 0x11, 0x58, 0xd7, 0xc7, 0xba, 0x03, 0x3e,
-	0x01, 0xf2, 0xf9, 0x4e, 0x12, 0xa3, 0x50, 0x7b, 0xaf, 0x7b, 0x60, 0x5c,
-	0x83, 0x23, 0xd2, 0x60, 0x27, 0x84, 0xad, 0xb8, 0x02, 0xed, 0xfe, 0xb4,
-	0x9c, 0x08, 0x9f, 0x49, 0xae, 0x55, 0x02, 0x94, 0xc0, 0x1b, 0x90, 0x75,
-	0x0b, 0x90, 0xc4, 0xc7, 0x43, 0x9e, 0x67, 0x25, 0x70, 0x61, 0x0d, 0xb8,
-	0x7a, 0x97, 0x43, 0xfc, 0xd1, 0x7e, 0x68, 0xed, 0x03, 0xb7, 0x1e, 0x75,
-	0xe9, 0x4d, 0x7a, 0x23, 0x18, 0x37, 0x63, 0x6f, 0xab, 0x5f, 0x7c, 0x5b,
-	0x1c, 0x05, 0xdf, 0x3f, 0x00, 0x86, 0x37, 0xa0, 0xfa, 0x0c, 0xe0, 0xed,
-	0x35, 0x35, 0x2f, 0xd8, 0xd1, 0x75, 0xba, 0x37, 0x34, 0x7e, 0xb0, 0x84,
-	0x2a, 0x01, 0x0c, 0x98, 0xed, 0x47, 0xf9, 0x86, 0x81, 0x74, 0x00, 0x5d,
-	0x8b, 0x4c, 0x18, 0x8a, 0x31, 0xcd, 0xae, 0x07, 0x44, 0xb5, 0xd5, 0x07,
-	0xa0, 0xdf, 0xf4, 0xfa, 0xa6, 0x42, 0xd0, 0x4f, 0x17, 0xd8, 0xdf, 0xb6,
-	0x34, 0x44, 0xe3, 0x01, 0xc4, 0xb6, 0x2d, 0xb5, 0x56, 0xc6, 0x2a, 0x1f,
-	0x05, 0x6c, 0x35, 0xe0, 0x09, 0x31, 0xef, 0x60, 0xfe, 0xaf, 0x07, 0x80,
-	0x32, 0xa0, 0xe9, 0xd3, 0x96, 0x45, 0xa7, 0xaa, 0xb6, 0xfb, 0x03, 0x10,
-	0xe3, 0x97, 0x96, 0x8d, 0x3a, 0x01, 0xdd, 0x58, 0x58, 0x78, 0x00, 0xab,
-	0xff, 0x06, 0xa0, 0xd6, 0x01, 0x58, 0x08, 0xb7, 0xdc, 0x2d, 0xa7, 0xfb,
-	0x22, 0xa8, 0x67, 0x00, 0xe3, 0xcf, 0x82, 0x43, 0xfc, 0x96, 0x1b, 0x40,
-	0x63, 0xcf, 0x9d, 0x42, 0x5d, 0x66, 0x40, 0xaa, 0xaf, 0x28, 0x94, 0xd3,
-	0x2a, 0xd4, 0x02, 0x13, 0xd2, 0xdf, 0x03, 0x9c, 0x60, 0x6b, 0x16, 0x94,
-	0xb4, 0xbe, 0x62, 0xc2, 0x35, 0x60, 0x45, 0x09, 0x23, 0x5a, 0xe0, 0x85,
-	0xb3, 0x03, 0x50, 0x68, 0x0c, 0x20, 0xa5, 0xf9, 0x94, 0xd2, 0x35, 0x80,
-	0xad, 0x4c, 0x4e, 0x40, 0x41, 0x97, 0x92, 0x75, 0xbe, 0x0c, 0x03, 0x50,
-	0x85, 0x08, 0xaf, 0x36, 0x00, 0x68, 0xaf, 0x09, 0xb3, 0x0c, 0x20, 0x4f,
-	0x81, 0x6a, 0x6a, 0xf5, 0x0d, 0x70, 0x69, 0x00, 0x4c, 0xb4, 0x0f, 0x59,
-	0xe7, 0x31, 0x0a, 0x45, 0x9f, 0xde, 0x90, 0xd6, 0x38, 0x80, 0x6b, 0x2c,
-	0xb9, 0x2f, 0xd4, 0x01, 0x40, 0x14, 0xd5, 0xed, 0x8e, 0x01, 0x53, 0xbf,
-	0x03, 0x18, 0x1e, 0xb0, 0xc1, 0x85, 0x32, 0xec, 0x78, 0x2b, 0xf0, 0xbb,
-	0xbb, 0x6c, 0xf3, 0x4d, 0xdc, 0x73, 0x40, 0xfd, 0x10, 0x09, 0x9e, 0x20,
-	0xe2, 0x12, 0x8c, 0xe0, 0xd2, 0xed, 0x80, 0x6b, 0xcc, 0x78, 0x20, 0x03,
-	0xd0, 0x5e, 0x06, 0xf4, 0xb0, 0xc4, 0x0e, 0x15, 0x1d, 0x80, 0xb4, 0x76,
-	0xdf, 0x49, 0x03, 0x50, 0x82, 0xad, 0xda, 0x8b, 0x5a, 0x61, 0xc2, 0x5e,
-	0xb5, 0x1e, 0x46, 0xc0, 0xde, 0xaa, 0x0e, 0x15, 0x06, 0xd2, 0xf4, 0xb2,
-	0xd1, 0xed, 0x38, 0x0a, 0x03, 0x18, 0x33, 0x1a, 0x80, 0x61, 0x3e, 0xec,
-	0x7c, 0x74, 0xa8, 0x1d, 0x80, 0x1a, 0xce, 0x25, 0x1d, 0x41, 0xd1, 0xc1,
-	0x03, 0x28, 0xb5, 0xaf, 0x72, 0x9c, 0x59, 0x7a, 0xe1, 0x7d, 0xc0, 0xa5,
-	0x08, 0x1e, 0x18, 0x24, 0xfa, 0xbd, 0x99, 0x4a, 0x31, 0xa0, 0xea, 0xee,
-	0xf8, 0x36, 0x60, 0x98, 0xc9, 0x10, 0xd1, 0xa7, 0x35, 0x00, 0x8d, 0x40,
-	0x8e, 0x5a, 0x35, 0x0f, 0x80, 0xb1, 0xd4, 0x32, 0x79, 0x40, 0x34, 0x05,
-	0x7e, 0x98, 0xc6, 0x80, 0x3e, 0x90, 0x01, 0x65, 0xf4, 0x80, 0x73, 0x08,
-	0x64, 0xd7, 0x36, 0xc1, 0x7c, 0xc0, 0x5c, 0x75, 0x00, 0xc5, 0x09, 0x58,
-	0x9c, 0x13, 0x01, 0x72, 0x37, 0x9b, 0x79, 0xe4, 0x05, 0xd1, 0x01, 0x04,
-	0x98, 0x08, 0x74, 0xfd, 0xfc, 0x3f, 0x1c, 0x00, 0x73, 0x01, 0xfc, 0x1c,
-	0xcc, 0x16, 0x43, 0x19, 0x1d, 0xac, 0x61, 0x4b, 0x11, 0xc2, 0xa0, 0xf2,
-	0x01, 0x0b, 0x7b, 0x3b, 0xf4, 0xfc, 0x58, 0x5d, 0x2d, 0x5c, 0x01, 0x8c,
-	0x62, 0x17, 0x78, 0xbe, 0x60, 0x8c, 0x01, 0x6f, 0x91, 0x49, 0x65, 0x54,
-	0x92, 0xe9, 0x01, 0x1e, 0x10, 0x77, 0x35, 0x00, 0xa8, 0xd4, 0xc7, 0x71,
-	0x07, 0xd8, 0xcd, 0xa3, 0x7d, 0x69, 0x20, 0xac, 0x07, 0x00, 0x35, 0xc7,
-	0x62, 0xee, 0x8c, 0x7d, 0x0c, 0xb8, 0x43, 0x0e, 0x00, 0x08, 0xfb, 0xe7,
-	0xec, 0x33, 0x37, 0x04, 0x80, 0x2d, 0x1d, 0xa6, 0x13, 0x34, 0x1b, 0x1d,
-	0xc0, 0xca, 0x00, 0x92, 0xed, 0x2e, 0x56, 0xbe, 0x91, 0x80, 0x0c, 0x88,
-	0xa6, 0x01, 0xdf, 0x7f, 0x90, 0x49, 0xed, 0x0c, 0xe0, 0x08, 0x73, 0x28,
-	0x74, 0xc7, 0xe1, 0xb1, 0x03, 0x5d, 0xc5, 0xab, 0x61, 0x42, 0xdf, 0x03,
-	0x43, 0xf3, 0x35, 0x04, 0xcf, 0xc6, 0x1d, 0x79, 0x07, 0x40, 0x22, 0xe4,
-	0x68, 0x0d, 0x01, 0x95, 0xad, 0x72, 0x69, 0x00, 0x39, 0x9d, 0x53, 0x8f,
-	0x13, 0x0d, 0xb0, 0x29, 0x79, 0x1a, 0x39, 0x20, 0x12, 0x28, 0x9b, 0x02,
-	0x8f, 0x74, 0x90, 0x4c, 0xe8, 0xd1, 0x57, 0xf4, 0x01, 0x44, 0x04, 0xe0,
-	0x0c, 0x82, 0x91, 0xc5, 0x4f, 0x8f, 0xc6, 0x00, 0x43, 0x85, 0x65, 0xc8,
-	0xe6, 0x34, 0x1d, 0x80, 0xc0, 0xca, 0xdb, 0x57, 0x6c, 0x00, 0x72, 0x42,
-	0x5f, 0xd0, 0x49, 0x57, 0x47, 0xd4, 0x97, 0x18, 0x18, 0x80, 0x68, 0x8e,
-	0x0a, 0xf1, 0x6b, 0x34, 0xf1, 0x60, 0x2c, 0x41, 0x29, 0xd3, 0x3d, 0x55,
-	0x95, 0xb1, 0x3c, 0xd4, 0x95, 0x42, 0xef, 0xe7, 0xca, 0x00, 0x2e, 0xce,
-	0x25, 0xc2, 0xca, 0xf5, 0x00, 0x17, 0x3b, 0x8c, 0x42, 0x88, 0x03, 0xde,
-	0x97, 0xe1, 0x3a, 0x74, 0xb0, 0x33, 0xe0, 0x8f, 0x47, 0xeb, 0x2a, 0x5f,
-	0x36, 0x3e, 0x5a, 0xff, 0xc5, 0x80, 0xb9, 0x13, 0xa9, 0x1f, 0xf8, 0x86,
-	0xc9, 0x51, 0xf8, 0x4c, 0xaa, 0xe1, 0x65, 0x80, 0xb0, 0x8b, 0x91, 0xec,
-	0xcc, 0xbf, 0x70, 0x19, 0x98, 0x03, 0x10, 0xf0, 0x38, 0x40, 0xc4, 0x65,
-	0xbe, 0x41, 0xb2, 0x58, 0x3f, 0xe0, 0xcc, 0x0e, 0x08, 0x2b, 0x73, 0xf4,
-	0xdd, 0x86, 0x06, 0xa0, 0xc6, 0x8f, 0x1a, 0x32, 0x66, 0x50, 0x8e, 0xe1,
-	0x59, 0x67, 0x00, 0xed, 0x66, 0x1d, 0xdd, 0xfa, 0x7b, 0xe2, 0x56, 0x89,
-	0xd9, 0xa0, 0x4f, 0x41, 0x94, 0x28, 0xb8, 0xc6, 0xc7, 0x64, 0xde, 0x9b,
-	0x64, 0x44, 0x33, 0x39, 0xb5, 0x6c, 0xb9, 0x42, 0xe7, 0x7e, 0x16, 0xd2,
-	0x01, 0x12, 0x03, 0xb3, 0x48, 0x47, 0x6b, 0x75, 0x26, 0x19, 0x8c, 0xac,
-	0x6f, 0xb1, 0x6f, 0xdc, 0x04, 0x27, 0x3a, 0x00, 0xd6, 0xae, 0xfa, 0xe1,
-	0xf7, 0x30, 0xa4, 0xdb, 0xd5, 0x86, 0x5a, 0x07, 0x11, 0xde, 0xea, 0xf4,
-	0xb0, 0x83, 0x16, 0xbb, 0xc6, 0x00, 0x6e, 0xf2, 0x6b, 0x40, 0x81, 0x01,
-	0x67, 0x0e, 0xa9, 0x82, 0x23, 0x04, 0x34, 0xed, 0x02, 0xf5, 0xe4, 0x0e,
-	0x58, 0xe8, 0x8a, 0x58, 0x57, 0xb0, 0x56, 0x65, 0x3d, 0x40, 0x64, 0x03,
-	0x6e, 0x7b, 0x07, 0x20, 0x99, 0x90, 0x36, 0x95, 0x9f, 0xdf, 0x3d, 0xe8,
-	0x00, 0xa0, 0x57, 0x8f, 0x6d, 0xa4, 0xb3, 0x1d, 0x7a, 0x06, 0xa8, 0x26,
-	0x41, 0xb0, 0x8c, 0x9c, 0x10, 0x85, 0x6c, 0xb4, 0x31, 0xa6, 0x5b, 0x7a,
-	0x10, 0x51, 0x15, 0x3c, 0xa2, 0x42, 0xd3, 0x23, 0x02, 0xc0, 0x17, 0x7e,
-	0x03, 0x28, 0xba, 0xce, 0x9b, 0xae, 0xdd, 0x1a, 0x19, 0xd0, 0x15, 0xac,
-	0xeb, 0x20, 0x3c, 0x3a, 0x00, 0xc6, 0xbb, 0x8a, 0xd5, 0x64, 0xc2, 0x21,
-	0x1d, 0x6c, 0x15, 0x5a, 0xd3, 0x44, 0x98, 0x14, 0x95, 0xb3, 0xb7, 0xdd,
-	0xa6, 0xea, 0x06, 0x54, 0x78, 0xc3, 0xe8, 0x79, 0x9b, 0x86, 0x29, 0x76,
-	0x8b, 0x6b, 0xaa, 0x0d, 0xa8, 0x2f, 0x22, 0x2a, 0xeb, 0x68, 0x81, 0x6c,
-	0x56, 0xfd, 0x79, 0xac, 0x79, 0x4b, 0xa0, 0x01, 0x3f, 0x17, 0x43, 0x82,
-	0xb4, 0xd5, 0x00, 0x14, 0xb7, 0xf5, 0x00, 0xf4, 0x15, 0xa8, 0xd7, 0x4b,
-	0xb1, 0xbc, 0xa8, 0x36, 0x98, 0xf0, 0x8c, 0xe7, 0xf4, 0x7b, 0x35, 0xd8,
-	0xad, 0x0d, 0x5f, 0x9d, 0x96, 0xab, 0xed, 0x48, 0xe2, 0xdc, 0x1c, 0xbe,
-	0x12, 0xfa, 0x41, 0x6f, 0xf5, 0x1e, 0xb6, 0x9f, 0xee, 0xac, 0x21, 0xf4,
-	0xf6, 0x00, 0x38, 0xb1, 0x1f, 0xfd, 0xd0, 0x0e, 0xc7, 0xdd, 0xa0, 0x39,
-	0x07, 0x8c, 0x35, 0x1f, 0x7e, 0xcc, 0xbf, 0xf6, 0xe0, 0x06, 0x66, 0x7d,
-	0x10, 0x3f, 0xc5, 0x3e, 0xde, 0x42, 0xf9, 0x3d, 0x00, 0x54, 0x81, 0x67,
-	0x8a, 0xe6, 0x63, 0x0d, 0x01, 0xd0, 0x31, 0xe0, 0x6e, 0xd0, 0xe1, 0x59,
-	0xf6, 0x1b, 0xf7, 0x0d, 0x52, 0x06, 0x80, 0x61, 0x4f, 0xe8, 0x77, 0xdd,
-	0x6f, 0x48, 0x20, 0x1d, 0xbb, 0x2a, 0x16, 0x8b, 0x54, 0x87, 0x92, 0x83,
-	0xe6, 0x8f, 0x55, 0x59, 0x06, 0x00, 0xe9, 0xc5, 0xce, 0x21, 0x63, 0x87,
-	0xaf, 0x86, 0xcc, 0xba, 0xd6, 0xe7, 0x00, 0xf6, 0x91, 0x92, 0x92, 0xea,
-	0xe8, 0x42, 0x06, 0x69, 0x13, 0xf5, 0x00, 0xd0, 0xb0, 0xa7, 0xcb, 0x4c,
-	0xb0, 0xd2, 0x2d, 0x28, 0x63, 0xf0, 0x6a, 0xc7, 0x80, 0x6a, 0x19, 0xb2,
-	0x66, 0x51, 0xf3, 0xb1, 0x21, 0xa0, 0x48, 0xad, 0x1e, 0x80, 0x62, 0xaf,
-	0x00, 0xf4, 0xa5, 0x4e, 0x83, 0x75, 0x1b, 0xfe, 0x00, 0xc4, 0xcf, 0x55,
-	0xb2, 0x50, 0xa6, 0xeb, 0x38, 0xed, 0x8f, 0xd3, 0x1d, 0x00, 0xf6, 0xe3,
-	0x90, 0x1c, 0x60, 0x9e, 0x8e, 0xeb, 0x0b, 0xba, 0x44, 0x06, 0x68, 0xbb,
-	0xd3, 0x10, 0x63, 0x35, 0xe1, 0x86, 0x5c, 0x5c, 0x2b, 0x85, 0xa6, 0xe7,
-	0x38, 0x2c, 0x18, 0x83, 0x1f, 0x8f, 0x9b, 0x8e, 0x4d, 0x26, 0xcd, 0x34,
-	0x0c, 0x66, 0x1d, 0x70, 0xb7, 0x01, 0xe6, 0x02, 0xa8, 0x51, 0x63, 0xcf,
-	0xbb, 0x03, 0xca, 0x85, 0xc6, 0x9c, 0xf6, 0xf1, 0x51, 0xe0, 0x60, 0x07,
-	0x40, 0x86, 0xf0, 0x1e, 0x6e, 0xef, 0x61, 0x10, 0xd9, 0x36, 0xcc, 0xfc,
-	0x58, 0xe2, 0x37, 0x0d, 0x58, 0xb7, 0xbe, 0xca, 0xc9, 0xd8, 0xcd, 0xaa,
-	0xd5, 0x5b, 0x77, 0x83, 0xcb, 0x0e, 0x30, 0xce, 0xc8, 0xb8, 0xcd, 0xbf,
-	0x1e, 0x63, 0x04, 0xad, 0xb7, 0xcd, 0x43, 0x62, 0x4c, 0xe0, 0x1a, 0xd4,
-	0x21, 0xe2, 0xdd, 0x33, 0xdf, 0xb1, 0xdd, 0xdc, 0x01, 0x22, 0x18, 0xce,
-	0xa1, 0xd8, 0xcb, 0x67, 0xd5, 0x38, 0x4a, 0xbc, 0xd5, 0x81, 0x3d, 0x03,
-	0x98, 0x35, 0x60, 0x41, 0x85, 0x0c, 0x1d, 0xe7, 0x76, 0xf8, 0x11, 0x52,
-	0x76, 0xf6, 0x06, 0x16, 0x02, 0x45, 0xc8, 0xd8, 0x2f, 0x5e, 0x57, 0xbc,
-	0x3b, 0x89, 0x97, 0x09, 0x3e, 0x03, 0x34, 0x1a, 0x9d, 0x37, 0x87, 0x48,
-	0x0a, 0xe0, 0xa7, 0x4f, 0x8c, 0x3a, 0xa2, 0xaf, 0xfd, 0x7b, 0x80, 0xcf,
-	0xe5, 0x18, 0x61, 0x68, 0xba, 0x61, 0x8b, 0x09, 0xaa, 0xa3, 0x0c, 0x47,
-	0x3c, 0x43, 0x03, 0xac, 0xa3, 0x2e, 0x5e, 0x72, 0x0c, 0x80, 0x19, 0x61,
-	0xe6, 0x6e, 0x0e, 0xd9, 0xe8, 0xe8, 0xaf, 0x11, 0x9b, 0x4a, 0x73, 0x7a,
-	0x61, 0x66, 0xf0, 0x54, 0x1d, 0x18, 0xc8, 0x23, 0x36, 0xbf, 0xb5, 0xf4,
-	0x86, 0x54, 0xed, 0xb5, 0x91, 0xee, 0xb8, 0xbc, 0xde, 0xc3, 0x87, 0x9b,
-	0x2f, 0x81, 0xf2, 0xee, 0xa3, 0xec, 0x02
-};
-
-/**
- * Uncompressed size of \c #srcZstd.
- */
-#define DXT1_256x256 32768
-
-/**
- * Destination for decoding \c #srcZstd.
- */
-static uint8_t dstDxt1[DXT1_256x256] = {};
-
-#ifndef ZSTD_VERSION_MAJOR
-/**
- * For the case where the decompression library hasn't been included we add a
- * dummy function to fake the process and stop the buffers being optimised out.
- */
-size_t ZSTD_decompress(void* dst, size_t dstLen, const void* src, size_t srcLen) {
-	return (memcmp(dst, src, (srcLen < dstLen) ? srcLen : dstLen)) ? dstLen : 0;
-}
-#endif
-
-//*************************** Program and Shaders ***************************/
-
-/**
- * Program object ID.
- */
-static GLuint progId = 0;
-
-/**
- * Vertex shader ID.
- */
-static GLuint vertId = 0;
-
-/**
- * Fragment shader ID.
- */
-static GLuint fragId = 0;
-
-//********************************* Uniforms *********************************/
-
-/**
- * Quad rotation angle ID.
- */
-static GLint uRotId = -1;
-
-/**
- * Draw colour ID.
- */
-static GLint uTx0Id = -1;
-
-//******************************* Shader Source ******************************/
-
-/**
- * Vertex shader to draw texture mapped polys with an applied rotation.
- */
-static GLchar const vertShader2D[] =
-#if GL_ES_VERSION_2_0
-	"#version 100\n"
-	"precision mediump float;\n"
-#else
-	"#version 120\n"
-#endif
-	"uniform   float uRot;"	// rotation
-	"attribute vec2  aPos;"	// vertex position coords
-	"attribute vec2  aUV0;"	// vertex texture UV0
-	"varying   vec2  vUV0;"	// (passed to fragment shader)
-	"void main() {"
-	"	float cosA = cos(radians(uRot));"
-	"	float sinA = sin(radians(uRot));"
-	"	mat3 rot = mat3(cosA, -sinA, 0.0,"
-	"					sinA,  cosA, 0.0,"
-	"					0.0,   0.0,  1.0);"
-	"	gl_Position = vec4(rot * vec3(aPos, 1.0), 1.0);"
-	"	vUV0 = aUV0;"
-	"}";
-
-/**
- * Fragment shader for the above polys.
- */
-static GLchar const fragShader2D[] =
-#if GL_ES_VERSION_2_0
-	"#version 100\n"
-	"precision mediump float;\n"
-#else
-	"#version 120\n"
-#endif
-	"uniform sampler2D uTx0;"
-	"varying vec2      vUV0;" // (passed from fragment shader)
-	"void main() {"
-	"	gl_FragColor = texture2D(uTx0, vUV0);"
-	"}";
-
-/**
- * Helper to compile a shader.
- * 
- * \param type shader type
- * \param text shader source
- * \return the shader ID (or zero if compilation failed)
- */
-static GLuint compileShader(GLenum const type, const GLchar* text) {
-	GLuint shader = glCreateShader(type);
-	if (shader) {
-		glShaderSource (shader, 1, &text, NULL);
-		glCompileShader(shader);
-		GLint compiled;
-		glGetShaderiv(shader, GL_COMPILE_STATUS, &compiled);
-		if (compiled) {
-			return shader;
-		} else {
-			GLint logLen;
-			glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &logLen);
-			if (logLen > 1) {
-				GLchar*  logStr = malloc(logLen);
-				glGetShaderInfoLog(shader, logLen, NULL, logStr);
-			#ifndef NDEBUG
-				printf("Shader compilation error: %s\n", logStr);
-			#endif
-				free(logStr);
-			}
-			glDeleteShader(shader);
-		}
-	}
-	return 0;
-}
-
-//********************************** Helpers *********************************/
-
-/**
- * Vertex position index.
- */
-#define GL_VERT_POSXY_ID 0
-
-/**
- * Vertex UV0 index.
- */
-#define GL_VERT_TXUV0_ID 1
-
- /**
-  * \c GL vec2 storage type.
-  */
-struct vec2 {
-	float x;
-	float y;
-};
-
-/**
- * Combined 2D vertex and 2D texture coordinates.
- */
-struct posTex2d {
-	struct vec2 pos;
-	struct vec2 uv0;
-};
-
-//****************************************************************************/
-
-/**
- * Current quad rotation angle (in degrees, updated per frame).
- */
-static float rotDeg = 0.0f;
-
-/**
- * Emscripten (single) GL context.
- */
-static EMSCRIPTEN_WEBGL_CONTEXT_HANDLE glCtx = 0;
-
-/**
- * Emscripten resize handler.
- */
-static EM_BOOL resize(int type, const EmscriptenUiEvent* e, void* data) {
-	double surfaceW;
-	double surfaceH;
-	if (emscripten_get_element_css_size   ("#canvas", &surfaceW, &surfaceH) == EMSCRIPTEN_RESULT_SUCCESS) {
-		emscripten_set_canvas_element_size("#canvas",  surfaceW,  surfaceH);
-		if (glCtx) {
-			glViewport(0, 0, (int) surfaceW, (int) surfaceH);
-		}
-	}
-	(void) type;
-	(void) data;
-	(void) e;
-	return EM_FALSE;
-}
-
-/**
- * Boilerplate to create a WebGL context.
- */
-static EM_BOOL initContext() {
-	// Default attributes
-	EmscriptenWebGLContextAttributes attr;
-	emscripten_webgl_init_context_attributes(&attr);
-	if ((glCtx = emscripten_webgl_create_context("#canvas", &attr))) {
-		// Bind the context and fire a resize to get the initial size
-		emscripten_webgl_make_context_current(glCtx);
-		emscripten_set_resize_callback(EMSCRIPTEN_EVENT_TARGET_DOCUMENT, NULL, EM_FALSE, resize);
-		resize(0, NULL, NULL);
-		return EM_TRUE;
-	}
-	return EM_FALSE;
-}
-
-/**
- * Called once per frame (clears the screen and draws the rotating quad).
- */
-static void tick() {
-	glClearColor(1.0f, 0.0f, 1.0f, 1.0f);
-	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
-
-	if (uRotId >= 0) {
-		glUniform1f(uRotId, rotDeg);
-		rotDeg += 0.1f;
-		if (rotDeg >= 360.0f) {
-			rotDeg -= 360.0f;
-		}
-	}
-
-	glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, 0);
-	glFlush();
-}
-
-/**
- * Creates the GL context, shaders and quad data, decompresses the Zstd data
- * and 'uploads' the resulting texture.
- * 
- * As a (naive) comparison, removing Zstd and building with "-Os -g0 s WASM=1
- * -lGL emscripten.c" results in a 15kB WebAssembly file; re-adding Zstd
- * increases the Wasm by 26kB.
- */
-int main() {
-	if (initContext()) {
-		// Compile shaders and set the initial GL state
-		if ((progId = glCreateProgram())) {
-			 vertId = compileShader(GL_VERTEX_SHADER,   vertShader2D);
-			 fragId = compileShader(GL_FRAGMENT_SHADER, fragShader2D);
-			 
-			 glBindAttribLocation(progId, GL_VERT_POSXY_ID, "aPos");
-			 glBindAttribLocation(progId, GL_VERT_TXUV0_ID, "aUV0");
-			 
-			 glAttachShader(progId, vertId);
-			 glAttachShader(progId, fragId);
-			 glLinkProgram (progId);
-			 glUseProgram  (progId);
-			 uRotId = glGetUniformLocation(progId, "uRot");
-			 uTx0Id = glGetUniformLocation(progId, "uTx0");
-			 if (uTx0Id >= 0) {
-				 glUniform1i(uTx0Id, 0);
-			 }
-			
-			 glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
-			 glEnable(GL_BLEND);
-			 glDisable(GL_DITHER);
-
-			 glCullFace(GL_BACK);
-			 glEnable(GL_CULL_FACE);
-		}
-		
-		GLuint vertsBuf = 0;
-		GLuint indexBuf = 0;
-		GLuint txName   = 0;
-		// Create the textured quad (vert positions then UVs)
-		struct posTex2d verts2d[] = {
-			{{-0.85f, -0.85f}, {0.0f, 0.0f}}, // BL
-			{{ 0.85f, -0.85f}, {1.0f, 0.0f}}, // BR
-			{{-0.85f,  0.85f}, {0.0f, 1.0f}}, // TL
-			{{ 0.85f,  0.85f}, {1.0f, 1.0f}}, // TR
-		};
-		uint16_t index2d[] = {
-			0, 1, 2,
-			2, 1, 3,
-		};
-		glGenBuffers(1, &vertsBuf);
-		glBindBuffer(GL_ARRAY_BUFFER, vertsBuf);
-		glBufferData(GL_ARRAY_BUFFER,
-			sizeof(verts2d), verts2d, GL_STATIC_DRAW);
-		glVertexAttribPointer(GL_VERT_POSXY_ID, 2,
-			GL_FLOAT, GL_FALSE, sizeof(struct posTex2d), 0);
-		glVertexAttribPointer(GL_VERT_TXUV0_ID, 2,
-			GL_FLOAT, GL_FALSE, sizeof(struct posTex2d),
-				(void*) offsetof(struct posTex2d, uv0));
-		glGenBuffers(1, &indexBuf);
-		glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuf);
-		glBufferData(GL_ELEMENT_ARRAY_BUFFER,
-			sizeof(index2d), index2d, GL_STATIC_DRAW);
-		glEnableVertexAttribArray(GL_VERT_POSXY_ID);
-		glEnableVertexAttribArray(GL_VERT_TXUV0_ID);
-		
-		// Decode the Zstd data and create the texture
-		if (ZSTD_decompress(dstDxt1, DXT1_256x256, srcZstd, sizeof srcZstd) == DXT1_256x256) {
-			glGenTextures(1, &txName);
-			glBindTexture(GL_TEXTURE_2D, txName);
-			glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
-			glCompressedTexImage2D(GL_TEXTURE_2D, 0,
-				GL_COMPRESSED_RGB_S3TC_DXT1_EXT,
-					256, 256, 0, DXT1_256x256, dstDxt1);
-		} else {
-			printf("Failed to decode Zstd data\n");
-		}
-		emscripten_set_main_loop(tick, 0, EM_FALSE);
-		emscripten_exit_with_live_runtime();
-	}
-	return EXIT_FAILURE;
-}
@@ -1,72 +0,0 @@
-/**
- * \file zstddeclib.c
- * Single-file Zstandard decompressor.
- * 
- * Generate using:
- * \code
- *	combine.sh -r ../../lib -r ../../lib/common -r ../../lib/decompress -o zstddeclib.c zstddeclib-in.c
- * \endcode
- */
-/* 
- * BSD License
- * 
- * For Zstandard software
- * 
- * Copyright (c) 2016-present, Facebook, Inc. All rights reserved.
- * 
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- * 
- * * Redistributions of source code must retain the above copyright notice,
- *   this list of conditions and the following disclaimer.
- * 
- * * Redistributions in binary form must reproduce the above copyright notice,
- *   this list of conditions and the following disclaimer in the documentation
- *   and/or other materials provided with the distribution.
- *
- * * Neither the name Facebook nor the names of its contributors may be used
- *   to endorse or promote products derived from this software without
- *   specific prior written permission.
- * 
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-/*
- * Settings to bake for the standalone decompressor.
- * 
- * Note: It's important that none of these affects 'zstd.h' (only the
- * implementation files we're amalgamating).
- * 
- * Note: MEM_MODULE stops xxhash redefining BYTE, U16, etc., which are also
- * defined in mem.h (breaking C99 compatibility).
- */
-#define DEBUGLEVEL 0
-#define MEM_MODULE
-#define XXH_NAMESPACE ZSTD_
-#define XXH_PRIVATE_API
-#define XXH_INLINE_ALL
-#define ZSTD_LEGACY_SUPPORT 0
-#define ZSTD_LIB_COMPRESSION 0
-#define ZSTD_LIB_DEPRECATED 0
-#define ZSTD_NOBENCH
-#define ZSTD_STRIP_ERROR_STRINGS
-
-#include "debug.c"
-#include "entropy_common.c"
-#include "error_private.c"
-#include "fse_decompress.c"
-#include "xxhash.c"
-#include "zstd_common.c"
-#include "huf_decompress.c"
-#include "zstd_ddict.c"
-#include "zstd_decompress.c"
-#include "zstd_decompress_block.c"
@@ -0,0 +1,9 @@
+
+# build artifacts
+zstddeclib.c
+zstdenclib.c
+zstd.c
+zstd.h
+
+# test artifacts
+temp*
@@ -0,0 +1,33 @@
+# Single File Zstandard Libraries
+
+The script `combine.sh` creates an _amalgamated_ source file that can be used with or without `zstd.h`. This isn't a _header-only_ file but it does offer a similar level of simplicity when integrating into a project.
+
+All it now takes to support Zstd in your own projects is the addition of a single file, two if using the header, with no configuration or further build steps.
+
+Decompressor
+------------
+
+This is the most common use case. The decompression library is small, adding, for example, 26kB to an Emscripten compiled WebAssembly project. Native implementations add a little more, 40-70kB depending on the compiler and platform.
+
+Create `zstddeclib.c` from the Zstd source using:
+```
+cd zstd/contrib/single_file_libs
+./combine.sh -r ../../lib -o zstddeclib.c zstddeclib-in.c
+```
+Then add the resulting file to your project (see the [example files](examples)).
+
+`create_single_file_decoder.sh` will run the above script, creating the file `zstddeclib.c` (`build_decoder_test.sh` will also create `zstddeclib.c`, then compile and test the result).
+
+Full Library
+------------
+
+The same tool can amalgamate the entire Zstd library for ease of adding both compression and decompression to a project. The [roundtrip example](examples/roundtrip.c) uses the original `zstd.h` with the remaining source files combined into `zstd.c` (currently just over 1.2MB) created from `zstd-in.c`. As with the standalone decoder the most useful compile flags have already been rolled-in and the resulting file can be added to a project as-is.
+
+Create `zstd.c` from the Zstd source using:
+```
+cd zstd/contrib/single_file_libs
+./combine.sh -r ../../lib -o zstd.c zstd-in.c
+```
+It's possible to create a compressor-only library but since the decompressor is so small in comparison this doesn't bring much of a gain (but for the curious, simply remove the files in the _decompress_ section at the end of `zstd-in.c`).
+
+`create_single_file_library.sh` will run the script to create `zstd.c` (`build_library_test.sh` will also create `zstd.c`, then compile and test the result).
@@ -0,0 +1,91 @@
+#!/bin/sh
+
+# Temporary compiled binary
+OUT_FILE="tempbin"
+
+# Optional temporary compiled WebAssembly
+OUT_WASM="temp.wasm"
+
+# Source files to compile using Emscripten.
+IN_FILES="examples/emscripten.c"
+
+# Emscripten build using emcc.
+emscripten_emcc_build() {
+  # Compile the the same example as above
+  CC_FLAGS="-Wall -Wextra -Werror -Os -g0 -flto"
+  emcc $CC_FLAGS -s WASM=1 -I. -o $OUT_WASM $IN_FILES
+  # Did compilation work?
+  if [ $? -ne 0 ]; then
+    echo "Compiling ${IN_FILES}: FAILED"
+    exit 1
+  fi
+  echo "Compiling ${IN_FILES}: PASSED"
+  rm -f $OUT_WASM
+}
+
+# Emscripten build using docker.
+emscripten_docker_build() {
+  docker container run --rm \
+    --volume $PWD:/code \
+    --workdir /code \
+    emscripten/emsdk:latest \
+    emcc $CC_FLAGS -s WASM=1 -I. -o $OUT_WASM $IN_FILES
+  # Did compilation work?
+  if [ $? -ne 0 ]; then
+      echo "Compiling ${IN_FILES} (using docker): FAILED"
+    exit 1
+  fi
+  echo "Compiling ${IN_FILES} (using docker): PASSED"
+  rm -f $OUT_WASM
+}
+
+# Try Emscripten build using emcc or docker.
+try_emscripten_build() {
+  which emcc > /dev/null
+  if [ $? -eq 0 ]; then
+    emscripten_emcc_build
+    return $?
+  fi
+
+  which docker > /dev/null
+  if [ $? -eq 0 ]; then
+    emscripten_docker_build
+    return $?
+  fi
+
+  echo "(Skipping Emscripten test)"
+}
+
+# Amalgamate the sources
+./create_single_file_decoder.sh
+# Did combining work?
+if [ $? -ne 0 ]; then
+  echo "Single file decoder creation script: FAILED"
+  exit 1
+fi
+echo "Single file decoder creation script: PASSED"
+
+# Compile the generated output
+cc -Wall -Wextra -Werror -Os -g0 -o $OUT_FILE examples/simple.c
+# Did compilation work?
+if [ $? -ne 0 ]; then
+  echo "Compiling simple.c: FAILED"
+  exit 1
+fi
+echo "Compiling simple.c: PASSED"
+
+# Run then delete the compiled output
+./$OUT_FILE
+retVal=$?
+rm -f $OUT_FILE
+# Did the test work?
+if [ $retVal -ne 0 ]; then
+  echo "Running simple.c: FAILED"
+  exit 1
+fi
+echo "Running simple.c: PASSED"
+
+# Try Emscripten build if emcc or docker command is available.
+try_emscripten_build
+
+exit 0
@@ -0,0 +1,97 @@
+#!/bin/sh
+
+# Where to find the sources (only used to copy zstd.h)
+ZSTD_SRC_ROOT="../../lib"
+
+# Temporary compiled binary
+OUT_FILE="tempbin"
+
+# Optional temporary compiled WebAssembly
+OUT_WASM="temp.wasm"
+
+# Source files to compile using Emscripten.
+IN_FILES="zstd.c examples/roundtrip.c"
+
+# Emscripten build using emcc.
+emscripten_emcc_build() {
+  # Compile the the same example as above
+  CC_FLAGS="-Wall -Wextra -Werror -Os -g0 -flto"
+  emcc $CC_FLAGS -s WASM=1 -I. -o $OUT_WASM $IN_FILES
+  # Did compilation work?
+  if [ $? -ne 0 ]; then
+    echo "Compiling ${IN_FILES}: FAILED"
+    exit 1
+  fi
+  echo "Compiling ${IN_FILES}: PASSED"
+  rm -f $OUT_WASM
+}
+
+# Emscripten build using docker.
+emscripten_docker_build() {
+  docker container run --rm \
+    --volume $PWD:/code \
+    --workdir /code \
+    emscripten/emsdk:latest \
+    emcc $CC_FLAGS -s WASM=1 -I. -o $OUT_WASM $IN_FILES
+  # Did compilation work?
+  if [ $? -ne 0 ]; then
+      echo "Compiling ${IN_FILES} (using docker): FAILED"
+    exit 1
+  fi
+  echo "Compiling ${IN_FILES} (using docker): PASSED"
+  rm -f $OUT_WASM
+}
+
+# Try Emscripten build using emcc or docker.
+try_emscripten_build() {
+  which emcc > /dev/null
+  if [ $? -eq 0 ]; then
+    emscripten_emcc_build
+    return $?
+  fi
+
+  which docker > /dev/null
+  if [ $? -eq 0 ]; then
+    emscripten_docker_build
+    return $?
+  fi
+
+  echo "(Skipping Emscripten test)"
+}
+
+# Amalgamate the sources
+./create_single_file_library.sh
+# Did combining work?
+if [ $? -ne 0 ]; then
+  echo "Single file library creation script: FAILED"
+  exit 1
+fi
+echo "Single file library creation script: PASSED"
+
+# Copy the header to here (for the tests)
+cp "$ZSTD_SRC_ROOT/zstd.h" zstd.h
+
+# Compile the generated output
+cc -Wall -Wextra -Werror -pthread -I. -Os -g0 -o $OUT_FILE zstd.c examples/roundtrip.c
+# Did compilation work?
+if [ $? -ne 0 ]; then
+  echo "Compiling roundtrip.c: FAILED"
+  exit 1
+fi
+echo "Compiling roundtrip.c: PASSED"
+
+# Run then delete the compiled output
+./$OUT_FILE
+retVal=$?
+rm -f $OUT_FILE
+# Did the test work?
+if [ $retVal -ne 0 ]; then
+  echo "Running roundtrip.c: FAILED"
+  exit 1
+fi
+echo "Running roundtrip.c: PASSED"
+
+# Try Emscripten build if emcc or docker command is available.
+try_emscripten_build
+
+exit 0
@@ -0,0 +1,211 @@
+#!/bin/sh -e
+
+# Tool to bundle multiple C/C++ source files, inlining any includes.
+# 
+# Note: this POSIX-compliant script is many times slower than the original bash
+# implementation (due to the grep calls) but it runs and works everywhere.
+# 
+# TODO: ROOTS, FOUND, etc., as arrays (since they fail on paths with spaces)
+# TODO: revert to Bash-only regex (the grep ones being too slow)
+# 
+# Author: Carl Woffenden, Numfum GmbH (this script is released under a CC0 license/Public Domain)
+
+# Common file roots
+ROOTS="."
+
+# -x option excluded includes
+XINCS=""
+
+# -k option includes to keep as include directives
+KINCS=""
+
+# Files previously visited
+FOUND=""
+
+# Optional destination file (empty string to write to stdout)
+DESTN=""
+
+# Whether the "#pragma once" directives should be written to the output
+PONCE=0
+
+# Prints the script usage then exits
+usage() {
+  echo "Usage: $0 [-r <path>] [-x <header>] [-k <header>] [-o <outfile>] infile"
+  echo "  -r file root search path"
+  echo "  -x file to completely exclude from inlining"
+  echo "  -k file to exclude from inlining but keep the include directive"
+  echo "  -p keep any '#pragma once' directives (removed by default)"
+  echo "  -o output file (otherwise stdout)"
+  echo "Example: $0 -r ../my/path - r ../other/path -o out.c in.c"
+  exit 1
+}
+
+# Tests that the grep implementation works as expected (older OSX grep fails)
+test_deps() {
+  if ! echo '#include "foo"' | grep -Eq '^\s*#\s*include\s*".+"'; then
+    echo "Aborting: the grep implementation fails to parse include lines"
+    exit 1
+  fi
+  if ! echo '"foo.h"' | sed -E 's/"([^"]+)"/\1/' | grep -Eq '^foo\.h$'; then
+    echo "Aborting: sed is unavailable or non-functional"
+    exit 1
+  fi
+}
+
+# Tests if list $1 has item $2 (returning zero on a match)
+list_has_item() {
+  if echo "$1" | grep -Eq "(^|\s*)$2(\$|\s*)"; then
+    return 0
+  else
+    return 1
+  fi
+}
+
+# Adds a new line with the supplied arguments to $DESTN (or stdout)
+write_line() {
+  if [ -n "$DESTN" ]; then
+    printf '%s\n' "$@" >> "$DESTN"
+  else
+    printf '%s\n' "$@"
+  fi
+}
+
+log_line() {
+  echo $@ >&2
+}
+
+# Find this file!
+resolve_include() {
+  local srcdir=$1
+  local inc=$2
+  for root in $srcdir $ROOTS; do
+    if [ -f "$root/$inc" ]; then
+      # Try to reduce the file path into a canonical form (so that multiple)
+      # includes of the same file are successfully deduplicated, even if they
+      # are expressed differently.
+      local relpath="$(realpath --relative-to . "$root/$inc" 2>/dev/null)"
+      if [ "$relpath" != "" ]; then # not all realpaths support --relative-to
+        echo "$relpath"
+        return 0
+      fi
+      local relpath="$(realpath "$root/$inc" 2>/dev/null)"
+      if [ "$relpath" != "" ]; then # not all distros have realpath...
+        echo "$relpath"
+        return 0
+      fi
+      # Fallback on Python to reduce the path if the above fails.
+      local relpath=$(python -c "import os,sys; print os.path.relpath(sys.argv[1])" "$root/$inc" 2>/dev/null)
+      if [ "$relpath" != "" ]; then # not all distros have realpath...
+        echo "$relpath"
+        return 0
+      fi
+      # Worst case, fall back to just the root + relative include path. The
+      # problem with this is that it is possible to emit multiple different
+      # resolved paths to the same file, depending on exactly how its included.
+      # Since the main loop below keeps a list of the resolved paths it's
+      # already included, in order to avoid repeated includes, this failure to
+      # produce a canonical/reduced path can lead to multiple inclusions of the
+      # same file. But it seems like the resulting single file library still
+      # works (hurray include guards!), so I guess it's ok.
+      echo "$root/$inc"
+      return 0
+    fi
+  done
+  return 1
+}
+
+# Adds the contents of $1 with any of its includes inlined
+add_file() {
+  local file=$1
+  if [ -n "$file" ]; then
+    log_line "Processing: $file"
+    # Get directory of the current so we can resolve relative includes
+    local srcdir="$(dirname "$file")"
+    # Read the file
+    local line=
+    while IFS= read -r line; do
+      if echo "$line" | grep -Eq '^\s*#\s*include\s*".+"'; then
+        # We have an include directive so strip the (first) file
+        local inc=$(echo "$line" | grep -Eo '".*"' | sed -E 's/"([^"]+)"/\1/' | head -1)
+        local res_inc="$(resolve_include "$srcdir" "$inc")"
+        if list_has_item "$XINCS" "$inc"; then
+          # The file was excluded so error if the source attempts to use it
+          write_line "#error Using excluded file: $inc"
+          log_line "Excluding: $inc"
+        else
+          if ! list_has_item "$FOUND" "$res_inc"; then
+            # The file was not previously encountered
+            FOUND="$FOUND $res_inc"
+            if list_has_item "$KINCS" "$inc"; then
+              # But the include was flagged to keep as included
+              write_line "/**** *NOT* inlining $inc ****/"
+              write_line "$line"
+              log_line "Not Inlining: $inc"
+            else
+              # The file was neither excluded nor seen before so inline it
+              write_line "/**** start inlining $inc ****/"
+              add_file "$res_inc"
+              write_line "/**** ended inlining $inc ****/"
+            fi
+          else
+            write_line "/**** skipping file: $inc ****/"
+          fi
+        fi
+      else
+        # Skip any 'pragma once' directives, otherwise write the source line
+        local write=$PONCE
+        if [ $write -eq 0 ]; then
+          if echo "$line" | grep -Eqv '^\s*#\s*pragma\s*once\s*'; then
+            write=1
+          fi
+        fi
+        if [ $write -ne 0 ]; then
+          write_line "$line"
+        fi
+      fi
+    done < "$file"
+  else
+    write_line "#error Unable to find \"$1\""
+    log_line "Error: Unable to find: \"$1\""
+  fi
+}
+
+while getopts ":r:x:k:po:" opts; do
+  case $opts in
+  r)
+    ROOTS="$ROOTS $OPTARG"
+    ;;
+  x)
+    XINCS="$XINCS $OPTARG"
+    ;;
+  k)
+    KINCS="$KINCS $OPTARG"
+    ;;
+  p)
+    PONCE=1
+    ;;
+  o)
+    DESTN="$OPTARG"
+    ;;
+  *)
+    usage
+    ;;
+  esac
+done
+shift $((OPTIND-1))
+
+if [ -n "$1" ]; then
+  if [ -f "$1" ]; then
+    if [ -n "$DESTN" ]; then
+      printf "" > "$DESTN"
+    fi
+    test_deps
+    add_file "$1"
+  else
+    echo "Input file not found: \"$1\""
+    exit 1
+  fi
+else
+  usage
+fi
+exit 0
@@ -3,10 +3,9 @@
 # Where to find the sources
 ZSTD_SRC_ROOT="../../lib"

-
 # Amalgamate the sources
 echo "Amalgamating files... this can take a while"
-./combine.sh -r "$ZSTD_SRC_ROOT" -r "$ZSTD_SRC_ROOT/common" -r "$ZSTD_SRC_ROOT/decompress" -o zstddeclib.c zstddeclib-in.c
+./combine.sh -r "$ZSTD_SRC_ROOT" -o zstddeclib.c zstddeclib-in.c
 # Did combining work?
 if [ $? -ne 0 ]; then
  echo "Combine script: FAILED"
@@ -0,0 +1,14 @@
+#!/bin/sh
+
+# Where to find the sources
+ZSTD_SRC_ROOT="../../lib"
+
+# Amalgamate the sources
+echo "Amalgamating files... this can take a while"
+./combine.sh -r "$ZSTD_SRC_ROOT" -o zstd.c zstd-in.c
+# Did combining work?
+if [ $? -ne 0 ]; then
+  echo "Combine script: FAILED"
+  exit 1
+fi
+echo "Combine script: PASSED"
@@ -6,4 +6,6 @@ The examples `#include` the generated `zstddeclib.c` directly but work equally a

 `emscripten.c` is a bare-bones [Emscripten](https://github.com/emscripten-core/emscripten) compiled WebGL demo using Zstd to further compress a DXT1 texture (see the [original PNG image](testcard.png)). The 256x256 texture would normally be 32kB, but even when bundled with the Zstd decompressor the resulting WebAssembly weighs in at 41kB (`shell.html` is a support file to run the Wasm).

-The example files in this directory are released under a [Creative Commons Zero license](https://creativecommons.org/publicdomain/zero/1.0/).
+`roundtrip.c` is an example to use with the optional [amalgamated library](../create_single_file_library.sh) showing compression the decompression.
+
+The example files in this directory are released under a [Creative Commons Zero license](https://creativecommons.org/publicdomain/zero/1.0/) (or Public Domain, whichever is applicable in your jurisdiction).
@@ -0,0 +1,340 @@
+/**
+ * \file emscripten.c
+ * Emscripten example of using the single-file \c zstddeclib. Draws a rotating
+ * textured quad with data from the in-line Zstd compressed DXT1 texture (DXT1
+ * being hardware compression, further compressed with Zstd).
+ * \n
+ * Compile using:
+ * \code
+ *	export CC_FLAGS="-Wall -Wextra -Werror -Os -g0 -flto --llvm-lto 3 -lGL -DNDEBUG=1"
+ *	export EM_FLAGS="-s WASM=1 -s ENVIRONMENT=web --shell-file shell.html --closure 1"
+ *	emcc $CC_FLAGS $EM_FLAGS -o out.html emscripten.c
+ * \endcode
+ * 
+ * \author Carl Woffenden, Numfum GmbH (released under a CC0 license)
+ */
+
+#include <stddef.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <emscripten/emscripten.h>
+#include <emscripten/html5.h>
+
+#include <GLES2/gl2.h>
+#include <GLES2/gl2ext.h>
+
+#include "../zstddeclib.c"
+
+//************************* Test Data (DXT texture) **************************/
+
+/**
+ * Zstd compressed DXT1 256x256 texture source.
+ * \n
+ * See \c testcard.png for the original.
+ */
+static uint8_t const srcZstd[] = {
+#include "testcard-zstd.inl"
+};
+
+/**
+ * Uncompressed size of \c #srcZstd.
+ */
+#define DXT1_256x256 32768
+
+/**
+ * Destination for decoding \c #srcZstd.
+ */
+static uint8_t dstDxt1[DXT1_256x256] = {};
+
+#ifndef ZSTD_VERSION_MAJOR
+/**
+ * For the case where the decompression library hasn't been included we add a
+ * dummy function to fake the process and stop the buffers being optimised out.
+ */
+size_t ZSTD_decompress(void* dst, size_t dstLen, const void* src, size_t srcLen) {
+	return (memcmp(dst, src, (srcLen < dstLen) ? srcLen : dstLen)) ? dstLen : 0;
+}
+#endif
+
+//*************************** Program and Shaders ***************************/
+
+/**
+ * Program object ID.
+ */
+static GLuint progId = 0;
+
+/**
+ * Vertex shader ID.
+ */
+static GLuint vertId = 0;
+
+/**
+ * Fragment shader ID.
+ */
+static GLuint fragId = 0;
+
+//********************************* Uniforms *********************************/
+
+/**
+ * Quad rotation angle ID.
+ */
+static GLint uRotId = -1;
+
+/**
+ * Draw colour ID.
+ */
+static GLint uTx0Id = -1;
+
+//******************************* Shader Source ******************************/
+
+/**
+ * Vertex shader to draw texture mapped polys with an applied rotation.
+ */
+static GLchar const vertShader2D[] =
+#if GL_ES_VERSION_2_0
+	"#version 100\n"
+	"precision mediump float;\n"
+#else
+	"#version 120\n"
+#endif
+	"uniform   float uRot;"	// rotation
+	"attribute vec2  aPos;"	// vertex position coords
+	"attribute vec2  aUV0;"	// vertex texture UV0
+	"varying   vec2  vUV0;"	// (passed to fragment shader)
+	"void main() {"
+	"	float cosA = cos(radians(uRot));"
+	"	float sinA = sin(radians(uRot));"
+	"	mat3 rot = mat3(cosA, -sinA, 0.0,"
+	"					sinA,  cosA, 0.0,"
+	"					0.0,   0.0,  1.0);"
+	"	gl_Position = vec4(rot * vec3(aPos, 1.0), 1.0);"
+	"	vUV0 = aUV0;"
+	"}";
+
+/**
+ * Fragment shader for the above polys.
+ */
+static GLchar const fragShader2D[] =
+#if GL_ES_VERSION_2_0
+	"#version 100\n"
+	"precision mediump float;\n"
+#else
+	"#version 120\n"
+#endif
+	"uniform sampler2D uTx0;"
+	"varying vec2      vUV0;" // (passed from fragment shader)
+	"void main() {"
+	"	gl_FragColor = texture2D(uTx0, vUV0);"
+	"}";
+
+/**
+ * Helper to compile a shader.
+ * 
+ * \param type shader type
+ * \param text shader source
+ * \return the shader ID (or zero if compilation failed)
+ */
+static GLuint compileShader(GLenum const type, const GLchar* text) {
+	GLuint shader = glCreateShader(type);
+	if (shader) {
+		glShaderSource (shader, 1, &text, NULL);
+		glCompileShader(shader);
+		GLint compiled;
+		glGetShaderiv(shader, GL_COMPILE_STATUS, &compiled);
+		if (compiled) {
+			return shader;
+		} else {
+			GLint logLen;
+			glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &logLen);
+			if (logLen > 1) {
+				GLchar*  logStr = malloc(logLen);
+				glGetShaderInfoLog(shader, logLen, NULL, logStr);
+			#ifndef NDEBUG
+				printf("Shader compilation error: %s\n", logStr);
+			#endif
+				free(logStr);
+			}
+			glDeleteShader(shader);
+		}
+	}
+	return 0;
+}
+
+//********************************** Helpers *********************************/
+
+/**
+ * Vertex position index.
+ */
+#define GL_VERT_POSXY_ID 0
+
+/**
+ * Vertex UV0 index.
+ */
+#define GL_VERT_TXUV0_ID 1
+
+ /**
+  * \c GL vec2 storage type.
+  */
+struct vec2 {
+	float x;
+	float y;
+};
+
+/**
+ * Combined 2D vertex and 2D texture coordinates.
+ */
+struct posTex2d {
+	struct vec2 pos;
+	struct vec2 uv0;
+};
+
+//****************************************************************************/
+
+/**
+ * Current quad rotation angle (in degrees, updated per frame).
+ */
+static float rotDeg = 0.0f;
+
+/**
+ * Emscripten (single) GL context.
+ */
+static EMSCRIPTEN_WEBGL_CONTEXT_HANDLE glCtx = 0;
+
+/**
+ * Emscripten resize handler.
+ */
+static EM_BOOL resize(int type, const EmscriptenUiEvent* e, void* data) {
+	double surfaceW;
+	double surfaceH;
+	if (emscripten_get_element_css_size   ("#canvas", &surfaceW, &surfaceH) == EMSCRIPTEN_RESULT_SUCCESS) {
+		emscripten_set_canvas_element_size("#canvas",  surfaceW,  surfaceH);
+		if (glCtx) {
+			glViewport(0, 0, (int) surfaceW, (int) surfaceH);
+		}
+	}
+	(void) type;
+	(void) data;
+	(void) e;
+	return EM_FALSE;
+}
+
+/**
+ * Boilerplate to create a WebGL context.
+ */
+static EM_BOOL initContext() {
+	// Default attributes
+	EmscriptenWebGLContextAttributes attr;
+	emscripten_webgl_init_context_attributes(&attr);
+	if ((glCtx = emscripten_webgl_create_context("#canvas", &attr))) {
+		// Bind the context and fire a resize to get the initial size
+		emscripten_webgl_make_context_current(glCtx);
+		emscripten_set_resize_callback(EMSCRIPTEN_EVENT_TARGET_DOCUMENT, NULL, EM_FALSE, resize);
+		resize(0, NULL, NULL);
+		return EM_TRUE;
+	}
+	return EM_FALSE;
+}
+
+/**
+ * Called once per frame (clears the screen and draws the rotating quad).
+ */
+static void tick() {
+	glClearColor(1.0f, 0.0f, 1.0f, 1.0f);
+	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
+
+	if (uRotId >= 0) {
+		glUniform1f(uRotId, rotDeg);
+		rotDeg += 0.1f;
+		if (rotDeg >= 360.0f) {
+			rotDeg -= 360.0f;
+		}
+	}
+
+	glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, 0);
+	glFlush();
+}
+
+/**
+ * Creates the GL context, shaders and quad data, decompresses the Zstd data
+ * and 'uploads' the resulting texture.
+ * 
+ * As a (naive) comparison, removing Zstd and building with "-Os -g0 s WASM=1
+ * -lGL emscripten.c" results in a 15kB WebAssembly file; re-adding Zstd
+ * increases the Wasm by 26kB.
+ */
+int main() {
+	if (initContext()) {
+		// Compile shaders and set the initial GL state
+		if ((progId = glCreateProgram())) {
+			 vertId = compileShader(GL_VERTEX_SHADER,   vertShader2D);
+			 fragId = compileShader(GL_FRAGMENT_SHADER, fragShader2D);
+			 
+			 glBindAttribLocation(progId, GL_VERT_POSXY_ID, "aPos");
+			 glBindAttribLocation(progId, GL_VERT_TXUV0_ID, "aUV0");
+			 
+			 glAttachShader(progId, vertId);
+			 glAttachShader(progId, fragId);
+			 glLinkProgram (progId);
+			 glUseProgram  (progId);
+			 uRotId = glGetUniformLocation(progId, "uRot");
+			 uTx0Id = glGetUniformLocation(progId, "uTx0");
+			 if (uTx0Id >= 0) {
+				 glUniform1i(uTx0Id, 0);
+			 }
+			
+			 glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
+			 glEnable(GL_BLEND);
+			 glDisable(GL_DITHER);
+
+			 glCullFace(GL_BACK);
+			 glEnable(GL_CULL_FACE);
+		}
+		
+		GLuint vertsBuf = 0;
+		GLuint indexBuf = 0;
+		GLuint txName   = 0;
+		// Create the textured quad (vert positions then UVs)
+		struct posTex2d verts2d[] = {
+			{{-0.85f, -0.85f}, {0.0f, 0.0f}}, // BL
+			{{ 0.85f, -0.85f}, {1.0f, 0.0f}}, // BR
+			{{-0.85f,  0.85f}, {0.0f, 1.0f}}, // TL
+			{{ 0.85f,  0.85f}, {1.0f, 1.0f}}, // TR
+		};
+		uint16_t index2d[] = {
+			0, 1, 2,
+			2, 1, 3,
+		};
+		glGenBuffers(1, &vertsBuf);
+		glBindBuffer(GL_ARRAY_BUFFER, vertsBuf);
+		glBufferData(GL_ARRAY_BUFFER,
+			sizeof(verts2d), verts2d, GL_STATIC_DRAW);
+		glVertexAttribPointer(GL_VERT_POSXY_ID, 2,
+			GL_FLOAT, GL_FALSE, sizeof(struct posTex2d), 0);
+		glVertexAttribPointer(GL_VERT_TXUV0_ID, 2,
+			GL_FLOAT, GL_FALSE, sizeof(struct posTex2d),
+				(void*) offsetof(struct posTex2d, uv0));
+		glGenBuffers(1, &indexBuf);
+		glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuf);
+		glBufferData(GL_ELEMENT_ARRAY_BUFFER,
+			sizeof(index2d), index2d, GL_STATIC_DRAW);
+		glEnableVertexAttribArray(GL_VERT_POSXY_ID);
+		glEnableVertexAttribArray(GL_VERT_TXUV0_ID);
+		
+		// Decode the Zstd data and create the texture
+		if (ZSTD_decompress(dstDxt1, DXT1_256x256, srcZstd, sizeof srcZstd) == DXT1_256x256) {
+			glGenTextures(1, &txName);
+			glBindTexture(GL_TEXTURE_2D, txName);
+			glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
+			glCompressedTexImage2D(GL_TEXTURE_2D, 0,
+				GL_COMPRESSED_RGB_S3TC_DXT1_EXT,
+					256, 256, 0, DXT1_256x256, dstDxt1);
+		} else {
+			printf("Failed to decode Zstd data\n");
+		}
+		emscripten_set_main_loop(tick, 0, EM_FALSE);
+		emscripten_exit_with_live_runtime();
+	}
+	return EXIT_FAILURE;
+}
@@ -0,0 +1,83 @@
+/*
+ * \file roundtrip.c
+ * In this example we include \c zstd.h and compile separately the amalgamated
+ * \c zstd.c:
+ * \code
+ *	cc -Wall -Wextra -Werror -I. -Os -g0 zstd.c examples/roundtrip.c 
+ * \endcode
+ *
+ * \author Carl Woffenden, Numfum GmbH (released under a CC0 license)
+ */
+
+#include <stddef.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "zstd.h"
+
+//************************** Test Data (DXT texture) **************************/
+
+/**
+ * Raw test data (borrowed from the Emscripten example).
+ * \n
+ * See \c testcard.png for the original.
+ */
+static uint8_t const rawData[] = {
+#include "testcard-dxt1.inl"
+};
+
+#ifndef ZSTD_VERSION_MAJOR
+/*
+ * For the case where the decompression library hasn't been included we add
+ * dummy functions to fake the process and stop the buffers being optimised out.
+ */
+size_t ZSTD_compressBound(size_t maxSrc) {
+	return maxSrc + 32;
+}
+int ZSTD_maxCLevel(void) {
+	return 20;
+}
+size_t ZSTD_compress(void* dst, size_t dstLen, const void* src, size_t srcLen, int level) {
+	return (memcmp(dst, src, (srcLen < dstLen) ? srcLen : dstLen)) ? level : dstLen;
+}
+unsigned ZSTD_isError(size_t code) {
+	return ((int) code) < 0;
+}
+size_t ZSTD_decompress(void* dst, size_t dstLen, const void* src, size_t srcLen) {
+	return (memcmp(dst, src, (srcLen < dstLen) ? srcLen : dstLen)) ? 0 : dstLen;
+}
+#endif
+
+//*****************************************************************************/
+
+/**
+ * Simple single-file test to compress \c rawData, decompress the result, then
+ * compare the decompressed version with the original.
+ */
+int main() {
+	size_t bounds = ZSTD_compressBound(sizeof rawData);
+	void* compBuf = malloc(bounds);
+	void* testBuf = malloc(sizeof rawData);
+	int compare   = -1;
+	if (compBuf && testBuf) {
+		size_t compSize = ZSTD_compress(compBuf, bounds, rawData, sizeof rawData, ZSTD_maxCLevel());
+		if (!ZSTD_isError(compSize)) {
+			printf("Compression: PASSED (size: %lu, uncompressed: %lu)\n", (unsigned long) compSize, (unsigned long) (sizeof rawData));
+			size_t decSize = ZSTD_decompress(testBuf, sizeof rawData, compBuf, compSize);
+			if (!ZSTD_isError(decSize)) {
+				printf("Decompression: PASSED\n");
+				compare = memcmp(rawData, testBuf, decSize);
+				printf("Byte comparison: %s\n", (compare == 0) ? "PASSED" : "FAILED");
+			} else {
+				printf("Decompression: FAILED\n");
+			}
+		} else {
+			printf("Compression: FAILED\n");
+		}
+		free(compBuf);
+		free(testBuf);
+	}
+	return (compare == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
+}
@@ -0,0 +1,75 @@
+/**
+ * \file simple.c
+ * Simple standalone example of using the single-file \c zstddeclib.
+ * 
+ * \note In this simple example we include the amalgamated source and compile
+ * just this single file, but we could equally (and more conventionally)
+ * include \c zstd.h and compile both this file and \c zstddeclib.c (the
+ * resulting binaries differ slightly in size but perform the same).
+ * 
+ * \author Carl Woffenden, Numfum GmbH (released under a CC0 license)
+ */
+
+#include <stddef.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "../zstddeclib.c"
+
+//************************* Test Data (DXT texture) **************************/
+
+/**
+ * Raw 256x256 DXT1 data (used to compare the result).
+ * \n
+ * See \c testcard.png for the original.
+ */
+static uint8_t const rawDxt1[] = {
+#include "testcard-dxt1.inl"
+};
+
+/**
+ * Zstd compressed version of \c #rawDxt1.
+ * \n
+ * See \c testcard.png for the original.
+ */
+static uint8_t const srcZstd[] = {
+#include "testcard-zstd.inl"
+};
+
+/**
+ * Destination for decoding \c #srcZstd.
+ */
+static uint8_t dstDxt1[sizeof rawDxt1] = {};
+
+#ifndef ZSTD_VERSION_MAJOR
+/**
+ * For the case where the decompression library hasn't been included we add a
+ * dummy function to fake the process and stop the buffers being optimised out.
+ */
+size_t ZSTD_decompress(void* dst, size_t dstLen, const void* src, size_t srcLen) {
+	return (memcmp(dst, src, (srcLen < dstLen) ? srcLen : dstLen)) ? 0 : dstLen;
+}
+#endif
+
+//****************************************************************************/
+
+/**
+ * Simple single-file test to decompress \c #srcZstd into \c # dstDxt1 then
+ * compare the resulting bytes with \c #rawDxt1.
+ * \n
+ * As a (naive) comparison, removing Zstd and building with "-Os -g0 simple.c"
+ * results in a 44kB binary (macOS 10.14, Clang 10); re-adding Zstd increases
+ * the binary by 56kB (after calling \c strip).
+ */
+int main() {
+	size_t size = ZSTD_decompress(dstDxt1, sizeof dstDxt1, srcZstd, sizeof srcZstd);
+	int compare = memcmp(rawDxt1, dstDxt1, sizeof dstDxt1);
+	printf("Decompressed size: %s\n", (size == sizeof dstDxt1) ? "PASSED" : "FAILED");
+	printf("Byte comparison: %s\n", (compare == 0) ? "PASSED" : "FAILED");
+	if (size == sizeof dstDxt1 && compare == 0) {
+		return EXIT_SUCCESS;
+	}
+	return EXIT_FAILURE;
+}
@@ -0,0 +1,261 @@
+0x28, 0xb5, 0x2f, 0xfd, 0x60, 0x00, 0x7f, 0x6d, 0x61, 0x00, 0x0a, 0x66,
+0xec, 0x17, 0x48, 0x60, 0x1c, 0x5a, 0xc9, 0x5d, 0x1a, 0x38, 0x07, 0xe8,
+0xc5, 0x82, 0x99, 0x68, 0xe6, 0x95, 0x45, 0x58, 0x0d, 0x0c, 0xf3, 0x36,
+0xc8, 0xd9, 0x0f, 0x46, 0x2d, 0x68, 0x11, 0xf8, 0x31, 0x10, 0xa1, 0x1a,
+0x2f, 0x99, 0x5c, 0x84, 0xfd, 0x92, 0x02, 0xe6, 0x3b, 0x44, 0x9b, 0x01,
+0x5d, 0x92, 0xff, 0x38, 0x26, 0x00, 0x6a, 0x6b, 0xc3, 0x53, 0xb2, 0x0c,
+0x25, 0xf3, 0xd8, 0x59, 0x68, 0x9b, 0x14, 0x8a, 0x89, 0x75, 0x18, 0x03,
+0x1d, 0xc9, 0x0f, 0x63, 0x01, 0x73, 0x01, 0x72, 0x01, 0x4f, 0x66, 0x31,
+0x58, 0x0f, 0x97, 0x4b, 0x0c, 0x4c, 0x06, 0xac, 0x07, 0x0b, 0x68, 0xd4,
+0xad, 0x80, 0x64, 0x13, 0x74, 0xa1, 0x12, 0x16, 0x58, 0xcf, 0x1a, 0x95,
+0x5f, 0x0d, 0x26, 0x55, 0xd0, 0x9c, 0xf4, 0x52, 0x35, 0x2e, 0x20, 0xc1,
+0x06, 0x69, 0x03, 0x0a, 0x93, 0x83, 0x5e, 0x27, 0x9b, 0x4c, 0x6d, 0xee,
+0x87, 0x03, 0x30, 0x6c, 0x46, 0xd7, 0x50, 0x5c, 0xca, 0xe6, 0xa6, 0x4d,
+0xa8, 0xf6, 0xab, 0xd7, 0x0e, 0x27, 0x27, 0x90, 0xc4, 0xb2, 0xd1, 0x10,
+0xfa, 0x43, 0x82, 0xc8, 0xf2, 0xe5, 0xff, 0xff, 0xd5, 0x52, 0x62, 0x43,
+0x87, 0x26, 0x2a, 0x05, 0x70, 0x0e, 0xb0, 0x2f, 0xc4, 0x56, 0xef, 0xb5,
+0xca, 0xb8, 0x53, 0xb7, 0x96, 0x0e, 0xe7, 0x00, 0x2c, 0xa8, 0xda, 0x3b,
+0x07, 0x70, 0xa7, 0x78, 0x38, 0x60, 0x87, 0x7a, 0x01, 0x3b, 0x75, 0xec,
+0xfa, 0x77, 0xe2, 0x46, 0x94, 0x61, 0x8e, 0x0d, 0x0c, 0xfb, 0xe7, 0x8b,
+0x13, 0x50, 0x31, 0xa9, 0x27, 0xcd, 0x27, 0xef, 0x6b, 0xa6, 0xab, 0x9c,
+0x4d, 0x95, 0x6c, 0x3a, 0xbb, 0x8e, 0x96, 0x92, 0x18, 0x5a, 0x7c, 0x4f,
+0xff, 0x7b, 0x38, 0xf2, 0xdb, 0x86, 0xde, 0xff, 0x1f, 0x2f, 0x21, 0x86,
+0x7d, 0xbf, 0x45, 0xd0, 0x6e, 0x77, 0x0a, 0xee, 0x0a, 0xee, 0x14, 0x9a,
+0xb8, 0x84, 0xf3, 0xac, 0xbe, 0xc8, 0x7f, 0x8d, 0xff, 0xff, 0xcf, 0x2a,
+0xfb, 0x69, 0xfc, 0xfb, 0xfd, 0x7a, 0x10, 0x22, 0x36, 0xfc, 0xff, 0x3f,
+0xcf, 0xd0, 0xf1, 0x7f, 0xfe, 0xff, 0x3d, 0x24, 0xdf, 0x78, 0x4a, 0xff,
+0xda, 0x9c, 0x39, 0xcf, 0xef, 0xe7, 0xfd, 0x52, 0x98, 0xb5, 0x40, 0x92,
+0xee, 0xdd, 0x99, 0xf5, 0x53, 0x5b, 0x65, 0x6b, 0xb5, 0xd8, 0x7b, 0xae,
+0xfa, 0xc1, 0x0f, 0x0c, 0x7f, 0x4f, 0x55, 0xa3, 0xad, 0x2c, 0xa0, 0xbd,
+0xf7, 0x2a, 0x0e, 0xe8, 0xbd, 0xc7, 0x5e, 0xf5, 0xd8, 0x54, 0x9e, 0x56,
+0xa3, 0xd6, 0x59, 0xd5, 0xfe, 0x1f, 0xc0, 0x30, 0x8c, 0xfc, 0x46, 0x04,
+0xae, 0x60, 0xbc, 0xe8, 0xcf, 0xec, 0x3d, 0xde, 0xf9, 0xf0, 0xfe, 0xef,
+0x7d, 0xcc, 0xf7, 0x2b, 0xe5, 0x1b, 0x70, 0xff, 0xff, 0x7e, 0x3f, 0x6e,
+0xe4, 0x02, 0x07, 0xfc, 0x1b, 0x7a, 0xff, 0xe7, 0x58, 0xfc, 0x7e, 0x3a,
+0xdc, 0x97, 0xfd, 0x57, 0xef, 0xa3, 0xfc, 0x2a, 0xc7, 0x4d, 0xf3, 0xcb,
+0x9d, 0xce, 0xac, 0xfe, 0xeb, 0x2e, 0x86, 0xb9, 0x69, 0x54, 0xef, 0xf9,
+0x55, 0xcf, 0xff, 0x48, 0x24, 0x72, 0x3a, 0x9d, 0x72, 0x2f, 0x2f, 0x2f,
+0xff, 0x3f, 0xe7, 0x01, 0x6c, 0x4d, 0x6c, 0xcd, 0x2d, 0x5b, 0x53, 0xb7,
+0x59, 0x22, 0x08, 0x0b, 0xa7, 0x92, 0x15, 0x75, 0x93, 0xb0, 0x5d, 0xaf,
+0x2a, 0x63, 0x95, 0x1d, 0x25, 0xd2, 0xd2, 0xa8, 0x1c, 0x84, 0xc9, 0xdc,
+0x72, 0xba, 0xd7, 0xfc, 0x69, 0xf5, 0xc7, 0x19, 0xa9, 0xbe, 0xfa, 0x26,
+0x55, 0x25, 0x75, 0xb7, 0x60, 0xa3, 0xd8, 0x68, 0x54, 0xb7, 0x1b, 0xa5,
+0x54, 0x62, 0xb1, 0x49, 0xde, 0xe2, 0xac, 0xa2, 0xe8, 0x7b, 0xff, 0x5f,
+0x75, 0x4e, 0xb8, 0xa2, 0xdd, 0x6a, 0xb7, 0xda, 0x6e, 0x2e, 0x04, 0xcd,
+0x08, 0x2f, 0xec, 0x8e, 0x49, 0xaf, 0x49, 0x6f, 0x8b, 0x4f, 0x2e, 0x1a,
+0xc5, 0x62, 0x7b, 0x6b, 0x3e, 0x32, 0x3e, 0x32, 0xbe, 0x08, 0x35, 0x4d,
+0x63, 0x93, 0xa6, 0xc8, 0x42, 0xe6, 0x21, 0xcc, 0x59, 0xc8, 0x4c, 0xe5,
+0x86, 0xe1, 0x03, 0x06, 0xa4, 0xec, 0xff, 0xb7, 0x78, 0x7e, 0x62, 0x43,
+0xc7, 0x2c, 0x50, 0x30, 0x4a, 0xc8, 0x9b, 0xf3, 0xbf, 0xe6, 0x62, 0xa0,
+0x50, 0xa6, 0x9c, 0xe3, 0x6e, 0x5b, 0xaf, 0x77, 0x8b, 0xbb, 0xe1, 0x70,
+0xaa, 0xaa, 0xaa, 0x92, 0xb4, 0x52, 0xad, 0x14, 0x87, 0x93, 0x0b, 0xe6,
+0x82, 0x39, 0x11, 0xb9, 0x20, 0x9a, 0x16, 0x34, 0x22, 0x68, 0xb2, 0x68,
+0xb2, 0x76, 0xd3, 0xe8, 0x6e, 0xda, 0x6b, 0x62, 0x34, 0x2e, 0x8d, 0xbd,
+0x2d, 0x3d, 0x6b, 0x4c, 0x26, 0x33, 0xda, 0x33, 0xf3, 0x91, 0x51, 0x46,
+0x79, 0xbd, 0x3e, 0x39, 0x9f, 0xcf, 0xd2, 0xb8, 0x5c, 0xfa, 0x22, 0xf8,
+0x8e, 0xb6, 0xbe, 0x08, 0x40, 0x14, 0x49, 0x40, 0x14, 0xf2, 0x0c, 0x2d,
+0x30, 0x85, 0x3c, 0x63, 0x29, 0xd3, 0x98, 0x85, 0x6c, 0xb5, 0xdb, 0xad,
+0x5c, 0x63, 0x9e, 0x72, 0xcf, 0x43, 0xe6, 0xaf, 0x77, 0xa6, 0xe2, 0x21,
+0x0c, 0x4d, 0xd3, 0x49, 0x1e, 0xc2, 0x14, 0x6f, 0xee, 0xdb, 0x7b, 0x7b,
+0x08, 0xb3, 0xa4, 0x60, 0x3b, 0x9d, 0x6e, 0x8b, 0x37, 0x4b, 0x0a, 0x74,
+0x35, 0x33, 0xbc, 0xf9, 0x64, 0x85, 0x63, 0x32, 0x29, 0x20, 0x59, 0x0c,
+0x3c, 0x96, 0x67, 0x62, 0xb7, 0x8a, 0x92, 0x4d, 0xa0, 0xd3, 0xf3, 0xd1,
+0x85, 0x80, 0x38, 0xcb, 0x64, 0x60, 0xc9, 0xb5, 0xaf, 0x97, 0x8d, 0x20,
+0x45, 0x28, 0xb8, 0xab, 0xe8, 0xc9, 0x0a, 0x88, 0x1f, 0xd6, 0x47, 0x54,
+0xf1, 0xd3, 0xfb, 0x62, 0xa7, 0xfd, 0xf2, 0x8b, 0xfd, 0xb6, 0xe4, 0x2e,
+0xb6, 0x91, 0x73, 0x1c, 0xd0, 0x7b, 0xba, 0x83, 0xc9, 0xac, 0x51, 0x39,
+0x92, 0xc5, 0x4f, 0x30, 0x1e, 0x2e, 0xd5, 0xf1, 0xa8, 0xa6, 0xa5, 0x80,
+0x70, 0xb9, 0xbc, 0xb7, 0xc2, 0x52, 0x32, 0x6c, 0xe3, 0x3d, 0xed, 0x41,
+0xa4, 0x4b, 0x31, 0x2a, 0xe6, 0x62, 0x11, 0x19, 0x95, 0x73, 0x1d, 0xbf,
+0xe1, 0x6c, 0xfc, 0x47, 0x75, 0x6c, 0x37, 0x63, 0x02, 0xf8, 0x34, 0x40,
+0x9a, 0x00, 0x1d, 0xf7, 0x32, 0x56, 0x77, 0xda, 0x5b, 0x9f, 0x9f, 0x0f,
+0xbb, 0x91, 0x5b, 0xbd, 0xe7, 0x58, 0x82, 0x4a, 0x20, 0xcd, 0x4f, 0x47,
+0x15, 0xf3, 0x51, 0xf1, 0x43, 0x51, 0x10, 0x96, 0xae, 0xba, 0xf7, 0x21,
+0x50, 0xef, 0x55, 0x27, 0x0c, 0x1f, 0xe0, 0x54, 0xf8, 0xc9, 0x69, 0xef,
+0xb9, 0x53, 0xf7, 0x83, 0x73, 0x9d, 0xce, 0x86, 0x07, 0x83, 0x44, 0x61,
+0x37, 0x35, 0x35, 0x33, 0x4e, 0x33, 0x33, 0x3e, 0x9f, 0x50, 0x48, 0x24,
+0xe6, 0xd0, 0x79, 0x49, 0xc3, 0x2d, 0xa0, 0x46, 0x31, 0x9a, 0x72, 0xc3,
+0x84, 0xff, 0x7a, 0x95, 0xbb, 0x00, 0x22, 0xcc, 0x14, 0x00, 0x04, 0xac,
+0x60, 0x64, 0x86, 0xe4, 0x6f, 0xb1, 0x58, 0xf4, 0xdc, 0xb0, 0x05, 0x00,
+0x72, 0x38, 0xf8, 0xce, 0xce, 0x8e, 0xcf, 0x37, 0x33, 0x43, 0x24, 0x0a,
+0x85, 0x33, 0x35, 0x35, 0x37, 0xc2, 0xa8, 0x28, 0x27, 0x1b, 0x9d, 0xce,
+0x29, 0x18, 0xe4, 0xfc, 0x09, 0x53, 0xa5, 0x51, 0xad, 0x74, 0x79, 0x7b,
+0xb5, 0x4a, 0x65, 0x94, 0x36, 0x89, 0xf5, 0x62, 0x9b, 0xd8, 0x9a, 0xe8,
+0x2b, 0xff, 0xa1, 0x73, 0xfe, 0x2e, 0x2c, 0x41, 0x45, 0x37, 0xef, 0x6e,
+0x7b, 0x38, 0xca, 0xa5, 0xd2, 0xb8, 0x1c, 0x3b, 0x96, 0x21, 0xbb, 0x5d,
+0xad, 0xd1, 0xdb, 0x1c, 0xf6, 0x5e, 0x4b, 0xd9, 0x59, 0x1b, 0x67, 0xf5,
+0x7a, 0x7c, 0x9a, 0x91, 0x3e, 0x8e, 0xe3, 0xee, 0x7b, 0xb9, 0xa4, 0xb9,
+0xf5, 0x70, 0xee, 0x1d, 0x4e, 0x4f, 0xcc, 0xd6, 0x7b, 0x07, 0x71, 0x48,
+0xf2, 0x06, 0xd0, 0x10, 0x82, 0x21, 0xe4, 0x55, 0x2e, 0xa5, 0x1d, 0xbe,
+0x48, 0x8c, 0x69, 0xbb, 0x24, 0x40, 0x68, 0x9b, 0xba, 0x5a, 0x5a, 0xa7,
+0xe2, 0xd1, 0xac, 0xc2, 0xd8, 0x87, 0x9c, 0xe3, 0x78, 0xee, 0xa6, 0xcd,
+0xdd, 0x68, 0x6e, 0xdd, 0xdb, 0x2e, 0xc6, 0xbb, 0x8b, 0xe9, 0xc1, 0xd9,
+0xf6, 0x62, 0x7e, 0x72, 0x7e, 0x72, 0x34, 0xe4, 0x68, 0xc8, 0x11, 0x32,
+0x10, 0x32, 0x20, 0x32, 0xf0, 0x12, 0x19, 0x90, 0xe0, 0x45, 0x91, 0xe0,
+0x25, 0x83, 0xba, 0xc9, 0x20, 0x26, 0x87, 0xa5, 0x72, 0xa9, 0x64, 0x72,
+0x80, 0x21, 0x54, 0x13, 0xeb, 0x29, 0x18, 0x42, 0x34, 0x84, 0x94, 0x34,
+0x84, 0xa4, 0x1d, 0xa4, 0x1d, 0x82, 0x1c, 0xef, 0xaf, 0x0e, 0x63, 0x47,
+0x6d, 0x10, 0xb9, 0xec, 0xd8, 0x2d, 0x3b, 0x9e, 0x21, 0xb1, 0x67, 0x48,
+0x34, 0x24, 0x1a, 0x52, 0xdb, 0xa4, 0x6d, 0x62, 0xd3, 0xfa, 0xff, 0xfa,
+0x03, 0x34, 0xef, 0x9d, 0xc9, 0xe1, 0x5d, 0xec, 0xf5, 0xf1, 0x79, 0x8c,
+0x97, 0xd2, 0x24, 0xc1, 0x2d, 0xe0, 0x39, 0x16, 0x1e, 0xa9, 0x41, 0xc3,
+0xbf, 0x4a, 0xd9, 0x3c, 0xea, 0x77, 0x96, 0x55, 0xe6, 0x95, 0xc3, 0xf1,
+0x8e, 0x7b, 0x4f, 0xad, 0x61, 0xf8, 0xe7, 0x01, 0xad, 0x46, 0xf5, 0x2c,
+0xac, 0x55, 0x2c, 0x94, 0xaa, 0x46, 0xfb, 0x5e, 0xcd, 0xaa, 0x1f, 0x78,
+0x4f, 0x2f, 0xd1, 0xc9, 0x02, 0xd6, 0x2c, 0x67, 0xef, 0x3f, 0x54, 0xab,
+0xda, 0x03, 0x79, 0x1f, 0xab, 0xfd, 0x0c, 0x38, 0x3c, 0xbc, 0xe1, 0xd5,
+0x01, 0xf6, 0xfb, 0xfb, 0xf1, 0x70, 0xee, 0xfd, 0x90, 0x13, 0x97, 0xc4,
+0xbc, 0x08, 0xe7, 0x4b, 0x88, 0x34, 0xf7, 0x56, 0x1e, 0x0c, 0xdb, 0xe4,
+0x9c, 0x78, 0xf1, 0xf4, 0x62, 0x4c, 0xb5, 0xf7, 0xdd, 0xd9, 0x4c, 0x5a,
+0x69, 0xa6, 0x36, 0x27, 0x03, 0xbe, 0x86, 0xc2, 0x72, 0xa2, 0x60, 0x73,
+0xf1, 0xe0, 0x17, 0x50, 0xb5, 0x93, 0x81, 0xac, 0xf1, 0xc9, 0xd4, 0x66,
+0x73, 0x71, 0xce, 0x63, 0xa8, 0x60, 0x98, 0xda, 0x86, 0x46, 0x72, 0xec,
+0x54, 0xc1, 0xe6, 0x8a, 0x10, 0x23, 0x2d, 0x0c, 0xdd, 0x44, 0x0b, 0xa0,
+0x44, 0xa4, 0x9d, 0x0e, 0x64, 0x31, 0x30, 0x45, 0xb1, 0x97, 0x4c, 0x6d,
+0x9f, 0x43, 0x99, 0x71, 0xa8, 0x9a, 0xe6, 0xbd, 0x3a, 0xe1, 0xff, 0x7f,
+0x33, 0x61, 0xf1, 0x48, 0xda, 0x48, 0x28, 0x10, 0x23, 0x9b, 0x24, 0xeb,
+0xee, 0xbd, 0x35, 0x0e, 0x6e, 0x75, 0x23, 0x20, 0xd6, 0x95, 0x8c, 0xa5,
+0x24, 0xec, 0x44, 0x67, 0x52, 0x2e, 0x78, 0x2a, 0xba, 0x3e, 0x3a, 0x4e,
+0xc9, 0x5c, 0x23, 0xb0, 0xd5, 0xfb, 0x29, 0xaf, 0x9a, 0x0b, 0x90, 0x89,
+0xd8, 0xec, 0xa9, 0xa8, 0x13, 0xfc, 0x22, 0xfa, 0xf2, 0x74, 0x2f, 0x4e,
+0x35, 0xb0, 0x6d, 0x6c, 0xfd, 0xc4, 0xfe, 0xd0, 0x98, 0x3d, 0xe5, 0x43,
+0x0a, 0xd0, 0x33, 0x26, 0x3b, 0x12, 0x7d, 0x65, 0xa1, 0xff, 0xff, 0x6f,
+0x53, 0x0e, 0x28, 0x84, 0xa7, 0xa2, 0x2e, 0xf8, 0x4a, 0xb6, 0xa1, 0x47,
+0xf5, 0xd0, 0x13, 0x9d, 0xd9, 0x50, 0xef, 0x9f, 0x31, 0xb4, 0x13, 0x67,
+0xf9, 0x0a, 0x99, 0xb5, 0x80, 0xec, 0x4a, 0x1a, 0x59, 0x21, 0x3b, 0xce,
+0xc5, 0x7e, 0x96, 0x85, 0x92, 0xc5, 0xb0, 0x75, 0xaa, 0x41, 0x16, 0xa9,
+0xd3, 0xde, 0x13, 0x7d, 0xd9, 0x61, 0x30, 0x1c, 0x73, 0x61, 0x54, 0x90,
+0xcf, 0xd4, 0xe8, 0xfe, 0xbf, 0xbf, 0x36, 0x57, 0x26, 0x38, 0xab, 0x06,
+0xc4, 0x7e, 0x3c, 0x5b, 0x35, 0xd2, 0x7c, 0x2c, 0x0a, 0x82, 0xf2, 0xa8,
+0xf2, 0x8b, 0x48, 0xa9, 0x90, 0x00, 0x01, 0x10, 0x10, 0x14, 0x04, 0x00,
+0x32, 0xa2, 0x06, 0x82, 0x28, 0x90, 0xc2, 0xb4, 0x85, 0xda, 0x01, 0x52,
+0xe9, 0x18, 0x85, 0x60, 0x00, 0x00, 0x20, 0x0c, 0x00, 0x14, 0x41, 0x00,
+0x40, 0xa0, 0x04, 0x40, 0x00, 0x80, 0x50, 0x03, 0x01, 0x10, 0x18, 0x01,
+0x80, 0xd4, 0x14, 0x99, 0x01, 0xfd, 0x07, 0xf8, 0x16, 0x0e, 0xd9, 0x5d,
+0xa3, 0x70, 0xfe, 0xda, 0x17, 0xfa, 0xce, 0x46, 0x9a, 0x99, 0x81, 0x1a,
+0x39, 0xba, 0x63, 0xb1, 0x18, 0x11, 0x58, 0xd7, 0xc7, 0xba, 0x03, 0x3e,
+0x01, 0xf2, 0xf9, 0x4e, 0x12, 0xa3, 0x50, 0x7b, 0xaf, 0x7b, 0x60, 0x5c,
+0x83, 0x23, 0xd2, 0x60, 0x27, 0x84, 0xad, 0xb8, 0x02, 0xed, 0xfe, 0xb4,
+0x9c, 0x08, 0x9f, 0x49, 0xae, 0x55, 0x02, 0x94, 0xc0, 0x1b, 0x90, 0x75,
+0x0b, 0x90, 0xc4, 0xc7, 0x43, 0x9e, 0x67, 0x25, 0x70, 0x61, 0x0d, 0xb8,
+0x7a, 0x97, 0x43, 0xfc, 0xd1, 0x7e, 0x68, 0xed, 0x03, 0xb7, 0x1e, 0x75,
+0xe9, 0x4d, 0x7a, 0x23, 0x18, 0x37, 0x63, 0x6f, 0xab, 0x5f, 0x7c, 0x5b,
+0x1c, 0x05, 0xdf, 0x3f, 0x00, 0x86, 0x37, 0xa0, 0xfa, 0x0c, 0xe0, 0xed,
+0x35, 0x35, 0x2f, 0xd8, 0xd1, 0x75, 0xba, 0x37, 0x34, 0x7e, 0xb0, 0x84,
+0x2a, 0x01, 0x0c, 0x98, 0xed, 0x47, 0xf9, 0x86, 0x81, 0x74, 0x00, 0x5d,
+0x8b, 0x4c, 0x18, 0x8a, 0x31, 0xcd, 0xae, 0x07, 0x44, 0xb5, 0xd5, 0x07,
+0xa0, 0xdf, 0xf4, 0xfa, 0xa6, 0x42, 0xd0, 0x4f, 0x17, 0xd8, 0xdf, 0xb6,
+0x34, 0x44, 0xe3, 0x01, 0xc4, 0xb6, 0x2d, 0xb5, 0x56, 0xc6, 0x2a, 0x1f,
+0x05, 0x6c, 0x35, 0xe0, 0x09, 0x31, 0xef, 0x60, 0xfe, 0xaf, 0x07, 0x80,
+0x32, 0xa0, 0xe9, 0xd3, 0x96, 0x45, 0xa7, 0xaa, 0xb6, 0xfb, 0x03, 0x10,
+0xe3, 0x97, 0x96, 0x8d, 0x3a, 0x01, 0xdd, 0x58, 0x58, 0x78, 0x00, 0xab,
+0xff, 0x06, 0xa0, 0xd6, 0x01, 0x58, 0x08, 0xb7, 0xdc, 0x2d, 0xa7, 0xfb,
+0x22, 0xa8, 0x67, 0x00, 0xe3, 0xcf, 0x82, 0x43, 0xfc, 0x96, 0x1b, 0x40,
+0x63, 0xcf, 0x9d, 0x42, 0x5d, 0x66, 0x40, 0xaa, 0xaf, 0x28, 0x94, 0xd3,
+0x2a, 0xd4, 0x02, 0x13, 0xd2, 0xdf, 0x03, 0x9c, 0x60, 0x6b, 0x16, 0x94,
+0xb4, 0xbe, 0x62, 0xc2, 0x35, 0x60, 0x45, 0x09, 0x23, 0x5a, 0xe0, 0x85,
+0xb3, 0x03, 0x50, 0x68, 0x0c, 0x20, 0xa5, 0xf9, 0x94, 0xd2, 0x35, 0x80,
+0xad, 0x4c, 0x4e, 0x40, 0x41, 0x97, 0x92, 0x75, 0xbe, 0x0c, 0x03, 0x50,
+0x85, 0x08, 0xaf, 0x36, 0x00, 0x68, 0xaf, 0x09, 0xb3, 0x0c, 0x20, 0x4f,
+0x81, 0x6a, 0x6a, 0xf5, 0x0d, 0x70, 0x69, 0x00, 0x4c, 0xb4, 0x0f, 0x59,
+0xe7, 0x31, 0x0a, 0x45, 0x9f, 0xde, 0x90, 0xd6, 0x38, 0x80, 0x6b, 0x2c,
+0xb9, 0x2f, 0xd4, 0x01, 0x40, 0x14, 0xd5, 0xed, 0x8e, 0x01, 0x53, 0xbf,
+0x03, 0x18, 0x1e, 0xb0, 0xc1, 0x85, 0x32, 0xec, 0x78, 0x2b, 0xf0, 0xbb,
+0xbb, 0x6c, 0xf3, 0x4d, 0xdc, 0x73, 0x40, 0xfd, 0x10, 0x09, 0x9e, 0x20,
+0xe2, 0x12, 0x8c, 0xe0, 0xd2, 0xed, 0x80, 0x6b, 0xcc, 0x78, 0x20, 0x03,
+0xd0, 0x5e, 0x06, 0xf4, 0xb0, 0xc4, 0x0e, 0x15, 0x1d, 0x80, 0xb4, 0x76,
+0xdf, 0x49, 0x03, 0x50, 0x82, 0xad, 0xda, 0x8b, 0x5a, 0x61, 0xc2, 0x5e,
+0xb5, 0x1e, 0x46, 0xc0, 0xde, 0xaa, 0x0e, 0x15, 0x06, 0xd2, 0xf4, 0xb2,
+0xd1, 0xed, 0x38, 0x0a, 0x03, 0x18, 0x33, 0x1a, 0x80, 0x61, 0x3e, 0xec,
+0x7c, 0x74, 0xa8, 0x1d, 0x80, 0x1a, 0xce, 0x25, 0x1d, 0x41, 0xd1, 0xc1,
+0x03, 0x28, 0xb5, 0xaf, 0x72, 0x9c, 0x59, 0x7a, 0xe1, 0x7d, 0xc0, 0xa5,
+0x08, 0x1e, 0x18, 0x24, 0xfa, 0xbd, 0x99, 0x4a, 0x31, 0xa0, 0xea, 0xee,
+0xf8, 0x36, 0x60, 0x98, 0xc9, 0x10, 0xd1, 0xa7, 0x35, 0x00, 0x8d, 0x40,
+0x8e, 0x5a, 0x35, 0x0f, 0x80, 0xb1, 0xd4, 0x32, 0x79, 0x40, 0x34, 0x05,
+0x7e, 0x98, 0xc6, 0x80, 0x3e, 0x90, 0x01, 0x65, 0xf4, 0x80, 0x73, 0x08,
+0x64, 0xd7, 0x36, 0xc1, 0x7c, 0xc0, 0x5c, 0x75, 0x00, 0xc5, 0x09, 0x58,
+0x9c, 0x13, 0x01, 0x72, 0x37, 0x9b, 0x79, 0xe4, 0x05, 0xd1, 0x01, 0x04,
+0x98, 0x08, 0x74, 0xfd, 0xfc, 0x3f, 0x1c, 0x00, 0x73, 0x01, 0xfc, 0x1c,
+0xcc, 0x16, 0x43, 0x19, 0x1d, 0xac, 0x61, 0x4b, 0x11, 0xc2, 0xa0, 0xf2,
+0x01, 0x0b, 0x7b, 0x3b, 0xf4, 0xfc, 0x58, 0x5d, 0x2d, 0x5c, 0x01, 0x8c,
+0x62, 0x17, 0x78, 0xbe, 0x60, 0x8c, 0x01, 0x6f, 0x91, 0x49, 0x65, 0x54,
+0x92, 0xe9, 0x01, 0x1e, 0x10, 0x77, 0x35, 0x00, 0xa8, 0xd4, 0xc7, 0x71,
+0x07, 0xd8, 0xcd, 0xa3, 0x7d, 0x69, 0x20, 0xac, 0x07, 0x00, 0x35, 0xc7,
+0x62, 0xee, 0x8c, 0x7d, 0x0c, 0xb8, 0x43, 0x0e, 0x00, 0x08, 0xfb, 0xe7,
+0xec, 0x33, 0x37, 0x04, 0x80, 0x2d, 0x1d, 0xa6, 0x13, 0x34, 0x1b, 0x1d,
+0xc0, 0xca, 0x00, 0x92, 0xed, 0x2e, 0x56, 0xbe, 0x91, 0x80, 0x0c, 0x88,
+0xa6, 0x01, 0xdf, 0x7f, 0x90, 0x49, 0xed, 0x0c, 0xe0, 0x08, 0x73, 0x28,
+0x74, 0xc7, 0xe1, 0xb1, 0x03, 0x5d, 0xc5, 0xab, 0x61, 0x42, 0xdf, 0x03,
+0x43, 0xf3, 0x35, 0x04, 0xcf, 0xc6, 0x1d, 0x79, 0x07, 0x40, 0x22, 0xe4,
+0x68, 0x0d, 0x01, 0x95, 0xad, 0x72, 0x69, 0x00, 0x39, 0x9d, 0x53, 0x8f,
+0x13, 0x0d, 0xb0, 0x29, 0x79, 0x1a, 0x39, 0x20, 0x12, 0x28, 0x9b, 0x02,
+0x8f, 0x74, 0x90, 0x4c, 0xe8, 0xd1, 0x57, 0xf4, 0x01, 0x44, 0x04, 0xe0,
+0x0c, 0x82, 0x91, 0xc5, 0x4f, 0x8f, 0xc6, 0x00, 0x43, 0x85, 0x65, 0xc8,
+0xe6, 0x34, 0x1d, 0x80, 0xc0, 0xca, 0xdb, 0x57, 0x6c, 0x00, 0x72, 0x42,
+0x5f, 0xd0, 0x49, 0x57, 0x47, 0xd4, 0x97, 0x18, 0x18, 0x80, 0x68, 0x8e,
+0x0a, 0xf1, 0x6b, 0x34, 0xf1, 0x60, 0x2c, 0x41, 0x29, 0xd3, 0x3d, 0x55,
+0x95, 0xb1, 0x3c, 0xd4, 0x95, 0x42, 0xef, 0xe7, 0xca, 0x00, 0x2e, 0xce,
+0x25, 0xc2, 0xca, 0xf5, 0x00, 0x17, 0x3b, 0x8c, 0x42, 0x88, 0x03, 0xde,
+0x97, 0xe1, 0x3a, 0x74, 0xb0, 0x33, 0xe0, 0x8f, 0x47, 0xeb, 0x2a, 0x5f,
+0x36, 0x3e, 0x5a, 0xff, 0xc5, 0x80, 0xb9, 0x13, 0xa9, 0x1f, 0xf8, 0x86,
+0xc9, 0x51, 0xf8, 0x4c, 0xaa, 0xe1, 0x65, 0x80, 0xb0, 0x8b, 0x91, 0xec,
+0xcc, 0xbf, 0x70, 0x19, 0x98, 0x03, 0x10, 0xf0, 0x38, 0x40, 0xc4, 0x65,
+0xbe, 0x41, 0xb2, 0x58, 0x3f, 0xe0, 0xcc, 0x0e, 0x08, 0x2b, 0x73, 0xf4,
+0xdd, 0x86, 0x06, 0xa0, 0xc6, 0x8f, 0x1a, 0x32, 0x66, 0x50, 0x8e, 0xe1,
+0x59, 0x67, 0x00, 0xed, 0x66, 0x1d, 0xdd, 0xfa, 0x7b, 0xe2, 0x56, 0x89,
+0xd9, 0xa0, 0x4f, 0x41, 0x94, 0x28, 0xb8, 0xc6, 0xc7, 0x64, 0xde, 0x9b,
+0x64, 0x44, 0x33, 0x39, 0xb5, 0x6c, 0xb9, 0x42, 0xe7, 0x7e, 0x16, 0xd2,
+0x01, 0x12, 0x03, 0xb3, 0x48, 0x47, 0x6b, 0x75, 0x26, 0x19, 0x8c, 0xac,
+0x6f, 0xb1, 0x6f, 0xdc, 0x04, 0x27, 0x3a, 0x00, 0xd6, 0xae, 0xfa, 0xe1,
+0xf7, 0x30, 0xa4, 0xdb, 0xd5, 0x86, 0x5a, 0x07, 0x11, 0xde, 0xea, 0xf4,
+0xb0, 0x83, 0x16, 0xbb, 0xc6, 0x00, 0x6e, 0xf2, 0x6b, 0x40, 0x81, 0x01,
+0x67, 0x0e, 0xa9, 0x82, 0x23, 0x04, 0x34, 0xed, 0x02, 0xf5, 0xe4, 0x0e,
+0x58, 0xe8, 0x8a, 0x58, 0x57, 0xb0, 0x56, 0x65, 0x3d, 0x40, 0x64, 0x03,
+0x6e, 0x7b, 0x07, 0x20, 0x99, 0x90, 0x36, 0x95, 0x9f, 0xdf, 0x3d, 0xe8,
+0x00, 0xa0, 0x57, 0x8f, 0x6d, 0xa4, 0xb3, 0x1d, 0x7a, 0x06, 0xa8, 0x26,
+0x41, 0xb0, 0x8c, 0x9c, 0x10, 0x85, 0x6c, 0xb4, 0x31, 0xa6, 0x5b, 0x7a,
+0x10, 0x51, 0x15, 0x3c, 0xa2, 0x42, 0xd3, 0x23, 0x02, 0xc0, 0x17, 0x7e,
+0x03, 0x28, 0xba, 0xce, 0x9b, 0xae, 0xdd, 0x1a, 0x19, 0xd0, 0x15, 0xac,
+0xeb, 0x20, 0x3c, 0x3a, 0x00, 0xc6, 0xbb, 0x8a, 0xd5, 0x64, 0xc2, 0x21,
+0x1d, 0x6c, 0x15, 0x5a, 0xd3, 0x44, 0x98, 0x14, 0x95, 0xb3, 0xb7, 0xdd,
+0xa6, 0xea, 0x06, 0x54, 0x78, 0xc3, 0xe8, 0x79, 0x9b, 0x86, 0x29, 0x76,
+0x8b, 0x6b, 0xaa, 0x0d, 0xa8, 0x2f, 0x22, 0x2a, 0xeb, 0x68, 0x81, 0x6c,
+0x56, 0xfd, 0x79, 0xac, 0x79, 0x4b, 0xa0, 0x01, 0x3f, 0x17, 0x43, 0x82,
+0xb4, 0xd5, 0x00, 0x14, 0xb7, 0xf5, 0x00, 0xf4, 0x15, 0xa8, 0xd7, 0x4b,
+0xb1, 0xbc, 0xa8, 0x36, 0x98, 0xf0, 0x8c, 0xe7, 0xf4, 0x7b, 0x35, 0xd8,
+0xad, 0x0d, 0x5f, 0x9d, 0x96, 0xab, 0xed, 0x48, 0xe2, 0xdc, 0x1c, 0xbe,
+0x12, 0xfa, 0x41, 0x6f, 0xf5, 0x1e, 0xb6, 0x9f, 0xee, 0xac, 0x21, 0xf4,
+0xf6, 0x00, 0x38, 0xb1, 0x1f, 0xfd, 0xd0, 0x0e, 0xc7, 0xdd, 0xa0, 0x39,
+0x07, 0x8c, 0x35, 0x1f, 0x7e, 0xcc, 0xbf, 0xf6, 0xe0, 0x06, 0x66, 0x7d,
+0x10, 0x3f, 0xc5, 0x3e, 0xde, 0x42, 0xf9, 0x3d, 0x00, 0x54, 0x81, 0x67,
+0x8a, 0xe6, 0x63, 0x0d, 0x01, 0xd0, 0x31, 0xe0, 0x6e, 0xd0, 0xe1, 0x59,
+0xf6, 0x1b, 0xf7, 0x0d, 0x52, 0x06, 0x80, 0x61, 0x4f, 0xe8, 0x77, 0xdd,
+0x6f, 0x48, 0x20, 0x1d, 0xbb, 0x2a, 0x16, 0x8b, 0x54, 0x87, 0x92, 0x83,
+0xe6, 0x8f, 0x55, 0x59, 0x06, 0x00, 0xe9, 0xc5, 0xce, 0x21, 0x63, 0x87,
+0xaf, 0x86, 0xcc, 0xba, 0xd6, 0xe7, 0x00, 0xf6, 0x91, 0x92, 0x92, 0xea,
+0xe8, 0x42, 0x06, 0x69, 0x13, 0xf5, 0x00, 0xd0, 0xb0, 0xa7, 0xcb, 0x4c,
+0xb0, 0xd2, 0x2d, 0x28, 0x63, 0xf0, 0x6a, 0xc7, 0x80, 0x6a, 0x19, 0xb2,
+0x66, 0x51, 0xf3, 0xb1, 0x21, 0xa0, 0x48, 0xad, 0x1e, 0x80, 0x62, 0xaf,
+0x00, 0xf4, 0xa5, 0x4e, 0x83, 0x75, 0x1b, 0xfe, 0x00, 0xc4, 0xcf, 0x55,
+0xb2, 0x50, 0xa6, 0xeb, 0x38, 0xed, 0x8f, 0xd3, 0x1d, 0x00, 0xf6, 0xe3,
+0x90, 0x1c, 0x60, 0x9e, 0x8e, 0xeb, 0x0b, 0xba, 0x44, 0x06, 0x68, 0xbb,
+0xd3, 0x10, 0x63, 0x35, 0xe1, 0x86, 0x5c, 0x5c, 0x2b, 0x85, 0xa6, 0xe7,
+0x38, 0x2c, 0x18, 0x83, 0x1f, 0x8f, 0x9b, 0x8e, 0x4d, 0x26, 0xcd, 0x34,
+0x0c, 0x66, 0x1d, 0x70, 0xb7, 0x01, 0xe6, 0x02, 0xa8, 0x51, 0x63, 0xcf,
+0xbb, 0x03, 0xca, 0x85, 0xc6, 0x9c, 0xf6, 0xf1, 0x51, 0xe0, 0x60, 0x07,
+0x40, 0x86, 0xf0, 0x1e, 0x6e, 0xef, 0x61, 0x10, 0xd9, 0x36, 0xcc, 0xfc,
+0x58, 0xe2, 0x37, 0x0d, 0x58, 0xb7, 0xbe, 0xca, 0xc9, 0xd8, 0xcd, 0xaa,
+0xd5, 0x5b, 0x77, 0x83, 0xcb, 0x0e, 0x30, 0xce, 0xc8, 0xb8, 0xcd, 0xbf,
+0x1e, 0x63, 0x04, 0xad, 0xb7, 0xcd, 0x43, 0x62, 0x4c, 0xe0, 0x1a, 0xd4,
+0x21, 0xe2, 0xdd, 0x33, 0xdf, 0xb1, 0xdd, 0xdc, 0x01, 0x22, 0x18, 0xce,
+0xa1, 0xd8, 0xcb, 0x67, 0xd5, 0x38, 0x4a, 0xbc, 0xd5, 0x81, 0x3d, 0x03,
+0x98, 0x35, 0x60, 0x41, 0x85, 0x0c, 0x1d, 0xe7, 0x76, 0xf8, 0x11, 0x52,
+0x76, 0xf6, 0x06, 0x16, 0x02, 0x45, 0xc8, 0xd8, 0x2f, 0x5e, 0x57, 0xbc,
+0x3b, 0x89, 0x97, 0x09, 0x3e, 0x03, 0x34, 0x1a, 0x9d, 0x37, 0x87, 0x48,
+0x0a, 0xe0, 0xa7, 0x4f, 0x8c, 0x3a, 0xa2, 0xaf, 0xfd, 0x7b, 0x80, 0xcf,
+0xe5, 0x18, 0x61, 0x68, 0xba, 0x61, 0x8b, 0x09, 0xaa, 0xa3, 0x0c, 0x47,
+0x3c, 0x43, 0x03, 0xac, 0xa3, 0x2e, 0x5e, 0x72, 0x0c, 0x80, 0x19, 0x61,
+0xe6, 0x6e, 0x0e, 0xd9, 0xe8, 0xe8, 0xaf, 0x11, 0x9b, 0x4a, 0x73, 0x7a,
+0x61, 0x66, 0xf0, 0x54, 0x1d, 0x18, 0xc8, 0x23, 0x36, 0xbf, 0xb5, 0xf4,
+0x86, 0x54, 0xed, 0xb5, 0x91, 0xee, 0xb8, 0xbc, 0xde, 0xc3, 0x87, 0x9b,
+0x2f, 0x81, 0xf2, 0xee, 0xa3, 0xec, 0x02
@@ -0,0 +1,78 @@
+/**
+ * \file zstd.c
+ * Single-file Zstandard library.
+ *
+ * Generate using:
+ * \code
+ *	combine.sh -r ../../lib -o zstd.c zstd-in.c
+ * \endcode
+ */
+/*
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+/*
+ * Settings to bake for the single library file.
+ *
+ * Note: It's important that none of these affects 'zstd.h' (only the
+ * implementation files we're amalgamating).
+ *
+ * Note: MEM_MODULE stops xxhash redefining BYTE, U16, etc., which are also
+ * defined in mem.h (breaking C99 compatibility).
+ *
+ * Note: the undefs for xxHash allow Zstd's implementation to coinside with with
+ * standalone xxHash usage (with global defines).
+ *
+ * Note: multithreading is enabled for all platforms apart from Emscripten.
+ */
+#define DEBUGLEVEL 0
+#define MEM_MODULE
+#undef  XXH_NAMESPACE
+#define XXH_NAMESPACE ZSTD_
+#undef  XXH_PRIVATE_API
+#define XXH_PRIVATE_API
+#undef  XXH_INLINE_ALL
+#define XXH_INLINE_ALL
+#define ZSTD_LEGACY_SUPPORT 0
+#ifndef __EMSCRIPTEN__
+#define ZSTD_MULTITHREAD
+#endif
+
+#include "common/debug.c"
+#include "common/entropy_common.c"
+#include "common/error_private.c"
+#include "common/fse_decompress.c"
+#include "common/threading.c"
+#include "common/pool.c"
+#include "common/zstd_common.c"
+
+#include "compress/fse_compress.c"
+#include "compress/hist.c"
+#include "compress/huf_compress.c"
+#include "compress/zstd_compress_literals.c"
+#include "compress/zstd_compress_sequences.c"
+#include "compress/zstd_compress_superblock.c"
+#include "compress/zstd_compress.c"
+#include "compress/zstd_double_fast.c"
+#include "compress/zstd_fast.c"
+#include "compress/zstd_lazy.c"
+#include "compress/zstd_ldm.c"
+#include "compress/zstd_opt.c"
+#ifdef ZSTD_MULTITHREAD
+#include "compress/zstdmt_compress.c"
+#endif
+
+#include "decompress/huf_decompress.c"
+#include "decompress/zstd_ddict.c"
+#include "decompress/zstd_decompress.c"
+#include "decompress/zstd_decompress_block.c"
+
+#include "dictBuilder/cover.c"
+#include "dictBuilder/divsufsort.c"
+#include "dictBuilder/fastcover.c"
+#include "dictBuilder/zdict.c"
@@ -0,0 +1,51 @@
+/**
+ * \file zstddeclib.c
+ * Single-file Zstandard decompressor.
+ *
+ * Generate using:
+ * \code
+ *	combine.sh -r ../../lib -o zstddeclib.c zstddeclib-in.c
+ * \endcode
+ */
+/*
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
+ * All rights reserved.
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+ */
+/*
+ * Settings to bake for the standalone decompressor.
+ *
+ * Note: It's important that none of these affects 'zstd.h' (only the
+ * implementation files we're amalgamating).
+ *
+ * Note: MEM_MODULE stops xxhash redefining BYTE, U16, etc., which are also
+ * defined in mem.h (breaking C99 compatibility).
+ *
+ * Note: the undefs for xxHash allow Zstd's implementation to coinside with with
+ * standalone xxHash usage (with global defines).
+ */
+#define DEBUGLEVEL 0
+#define MEM_MODULE
+#undef  XXH_NAMESPACE
+#define XXH_NAMESPACE ZSTD_
+#undef  XXH_PRIVATE_API
+#define XXH_PRIVATE_API
+#undef  XXH_INLINE_ALL
+#define XXH_INLINE_ALL
+#define ZSTD_LEGACY_SUPPORT 0
+#define ZSTD_STRIP_ERROR_STRINGS
+
+#include "common/debug.c"
+#include "common/entropy_common.c"
+#include "common/error_private.c"
+#include "common/fse_decompress.c"
+#include "common/zstd_common.c"
+
+#include "decompress/huf_decompress.c"
+#include "decompress/zstd_ddict.c"
+#include "decompress/zstd_decompress.c"
+#include "decompress/zstd_decompress_block.c"
@@ -1,10 +1,11 @@
 # ################################################################
-# Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+# Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 # All rights reserved.
 #
 # This source code is licensed under both the BSD-style license (found in the
 # LICENSE file in the root directory of this source tree) and the GPLv2 (found
 # in the COPYING file in the root directory of this source tree).
+# You may select, at your option, one of the above-listed licenses.
 # ################################################################

 ZSTD ?= zstd   # note: requires zstd installation on local system
@@ -36,7 +37,7 @@ harness: $(HARNESS_FILES)
 	$(CC) $(FLAGS) $^ -o $@

 clean:
-	@$(RM) harness
+	@$(RM) harness *.o
 	@$(RM) -rf harness.dSYM  # MacOS specific

 test: harness
@@ -59,4 +60,3 @@ test: harness
 	@./harness tmp.zst tmp dictionary
 	@$(DIFF) -s tmp README.md
 	@$(RM) tmp* dictionary
-	@$(MAKE) clean
@@ -13,6 +13,13 @@ It also contains implementations of Huffman and FSE table decoding.
 [Zstandard format specification]: https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md
 [format specification]: https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md

+While the library's primary objective is code clarity,
+it also happens to compile into a small object file.
+The object file can be made even smaller by removing error messages,
+using the macro directive `ZDEC_NO_MESSAGE` at compilation time.
+This can be reduced even further by foregoing dictionary support,
+by defining `ZDEC_NO_DICTIONARY`.
+
 `harness.c` provides a simple test harness around the decoder:

    harness <input-file> <output-file> [dictionary]
@@ -1,10 +1,11 @@
 /*
- * Copyright (c) 2017-present, Facebook, Inc.
+ * Copyright (c) 2017-2020, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
 * LICENSE file in the root directory of this source tree) and the GPLv2 (found
 * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 */

 #include <stdio.h>
@@ -21,108 +22,98 @@ typedef unsigned char u8;
 // Protect against allocating too much memory for output
 #define MAX_OUTPUT_SIZE ((size_t)1024 * 1024 * 1024)

-static size_t read_file(const char *path, u8 **ptr)
+// Error message then exit
+#define ERR_OUT(...) { fprintf(stderr, __VA_ARGS__); exit(1); }
+
+
+typedef struct {
+    u8* address;
+    size_t size;
+} buffer_s;
+
+static void freeBuffer(buffer_s b) { free(b.address); }
+
+static buffer_s read_file(const char *path)
 {
    FILE* const f = fopen(path, "rb");
-    if (!f) {
-        fprintf(stderr, "failed to open file %s \n", path);
-        exit(1);
-    }
+    if (!f) ERR_OUT("failed to open file %s \n", path);

    fseek(f, 0L, SEEK_END);
    size_t const size = (size_t)ftell(f);
    rewind(f);

-    *ptr = malloc(size);
-    if (!ptr) {
-        fprintf(stderr, "failed to allocate memory to hold %s \n", path);
-        exit(1);
-    }
+    void* const ptr = malloc(size);
+    if (!ptr) ERR_OUT("failed to allocate memory to hold %s \n", path);

-    size_t const read = fread(*ptr, 1, size, f);
-    if (read != size) {  /* must read everything in one pass */
-        fprintf(stderr, "error while reading file %s \n", path);
-        exit(1);
-    }
+    size_t const read = fread(ptr, 1, size, f);
+    if (read != size) ERR_OUT("error while reading file %s \n", path);

    fclose(f);
-
-    return read;
+    buffer_s const b = { ptr, size };
+    return b;
 }

-static void write_file(const char *path, const u8 *ptr, size_t size)
+static void write_file(const char* path, const u8* ptr, size_t size)
 {
    FILE* const f = fopen(path, "wb");
-    if (!f) {
-        fprintf(stderr, "failed to open file %s \n", path);
-        exit(1);
-    }
+    if (!f) ERR_OUT("failed to open file %s \n", path);

    size_t written = 0;
    while (written < size) {
        written += fwrite(ptr+written, 1, size, f);
-        if (ferror(f)) {
-            fprintf(stderr, "error while writing file %s\n", path);
-            exit(1);
-    }   }
+        if (ferror(f)) ERR_OUT("error while writing file %s\n", path);
+    }

    fclose(f);
 }

 int main(int argc, char **argv)
 {
-    if (argc < 3) {
-        fprintf(stderr, "usage: %s <file.zst> <out_path> [dictionary] \n",
-                argv[0]);
+    if (argc < 3)
+        ERR_OUT("usage: %s <file.zst> <out_path> [dictionary] \n", argv[0]);

-        return 1;
-    }
+    buffer_s const input = read_file(argv[1]);

-    u8* input;
-    size_t const input_size = read_file(argv[1], &input);
-
-    u8* dict = NULL;
-    size_t dict_size = 0;
+    buffer_s dict = { NULL, 0 };
    if (argc >= 4) {
-        dict_size = read_file(argv[3], &dict);
+        dict = read_file(argv[3]);
    }

-    size_t out_capacity = ZSTD_get_decompressed_size(input, input_size);
+    size_t out_capacity = ZSTD_get_decompressed_size(input.address, input.size);
    if (out_capacity == (size_t)-1) {
-        out_capacity = MAX_COMPRESSION_RATIO * input_size;
+        out_capacity = MAX_COMPRESSION_RATIO * input.size;
        fprintf(stderr, "WARNING: Compressed data does not contain "
                        "decompressed size, going to assume the compression "
                        "ratio is at most %d (decompressed size of at most "
                        "%u) \n",
                MAX_COMPRESSION_RATIO, (unsigned)out_capacity);
    }
-    if (out_capacity > MAX_OUTPUT_SIZE) {
-        fprintf(stderr,
-                "Required output size too large for this implementation \n");
-        return 1;
-    }
+    if (out_capacity > MAX_OUTPUT_SIZE)
+        ERR_OUT("Required output size too large for this implementation \n");

    u8* const output = malloc(out_capacity);
-    if (!output) {
-        fprintf(stderr, "failed to allocate memory \n");
-        return 1;
-    }
+    if (!output) ERR_OUT("failed to allocate memory \n");

    dictionary_t* const parsed_dict = create_dictionary();
-    if (dict) {
-        parse_dictionary(parsed_dict, dict, dict_size);
+    if (dict.size) {
+#if defined (ZDEC_NO_DICTIONARY)
+        printf("dict.size = %zu \n", dict.size);
+        ERR_OUT("no dictionary support \n");
+#else
+        parse_dictionary(parsed_dict, dict.address, dict.size);
+#endif
    }
    size_t const decompressed_size =
        ZSTD_decompress_with_dict(output, out_capacity,
-                                  input, input_size,
+                                  input.address, input.size,
                                  parsed_dict);

    free_dictionary(parsed_dict);

    write_file(argv[2], output, decompressed_size);

-    free(input);
+    freeBuffer(input);
+    freeBuffer(dict);
    free(output);
-    free(dict);
    return 0;
 }
@@ -1,34 +1,52 @@
 /*
- * Copyright (c) 2017-present, Facebook, Inc.
+ * Copyright (c) 2017-2020, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
 * LICENSE file in the root directory of this source tree) and the GPLv2 (found
 * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 */

 /// Zstandard educational decoder implementation
 /// See https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md

-#include <stdint.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
+#include <stdint.h>   // uint8_t, etc.
+#include <stdlib.h>   // malloc, free, exit
+#include <stdio.h>    // fprintf
+#include <string.h>   // memset, memcpy
 #include "zstd_decompress.h"

-/******* UTILITY MACROS AND TYPES *********************************************/
-// Max block size decompressed size is 128 KB and literal blocks can't be
-// larger than their block
-#define MAX_LITERALS_SIZE ((size_t)128 * 1024)

+/******* IMPORTANT CONSTANTS *********************************************/
+
+// Zstandard frame
+// "Magic_Number
+// 4 Bytes, little-endian format. Value : 0xFD2FB528"
+#define ZSTD_MAGIC_NUMBER 0xFD2FB528U
+
+// The size of `Block_Content` is limited by `Block_Maximum_Size`,
+#define ZSTD_BLOCK_SIZE_MAX ((size_t)128 * 1024)
+
+// literal blocks can't be larger than their block
+#define MAX_LITERALS_SIZE ZSTD_BLOCK_SIZE_MAX
+
+
+/******* UTILITY MACROS AND TYPES *********************************************/
 #define MAX(a, b) ((a) > (b) ? (a) : (b))
 #define MIN(a, b) ((a) < (b) ? (a) : (b))

+#if defined(ZDEC_NO_MESSAGE)
+#define MESSAGE(...)
+#else
+#define MESSAGE(...)  fprintf(stderr, "" __VA_ARGS__)
+#endif
+
 /// This decoder calls exit(1) when it encounters an error, however a production
 /// library should propagate error codes
 #define ERROR(s)                                                               \
    do {                                                                       \
-        fprintf(stderr, "Error: %s\n", s);                                     \
+        MESSAGE("Error: %s\n", s);                                     \
        exit(1);                                                               \
    } while (0)
 #define INP_SIZE()                                                             \
@@ -39,12 +57,12 @@
 #define BAD_ALLOC() ERROR("Memory allocation error")
 #define IMPOSSIBLE() ERROR("An impossibility has occurred")

-typedef uint8_t u8;
+typedef uint8_t  u8;
 typedef uint16_t u16;
 typedef uint32_t u32;
 typedef uint64_t u64;

-typedef int8_t i8;
+typedef int8_t  i8;
 typedef int16_t i16;
 typedef int32_t i32;
 typedef int64_t i64;
@@ -176,10 +194,6 @@ static void HUF_init_dtable_usingweights(HUF_dtable *const table,

 /// Free the malloc'ed parts of a decoding table
 static void HUF_free_dtable(HUF_dtable *const dtable);
-
-/// Deep copy a decoding table, so that it can be used and free'd without
-/// impacting the source table.
-static void HUF_copy_dtable(HUF_dtable *const dst, const HUF_dtable *const src);
 /*** END HUFFMAN PRIMITIVES ***********/

 /*** FSE PRIMITIVES *******************/
@@ -241,10 +255,6 @@ static void FSE_init_dtable_rle(FSE_dtable *const dtable, const u8 symb);

 /// Free the malloc'ed parts of a decoding table
 static void FSE_free_dtable(FSE_dtable *const dtable);
-
-/// Deep copy a decoding table, so that it can be used and free'd without
-/// impacting the source table.
-static void FSE_copy_dtable(FSE_dtable *const dst, const FSE_dtable *const src);
 /*** END FSE PRIMITIVES ***************/

 /******* END IMPLEMENTATION PRIMITIVE PROTOTYPES ******************************/
@@ -373,7 +383,7 @@ static void execute_match_copy(frame_context_t *const ctx, size_t offset,

 size_t ZSTD_decompress(void *const dst, const size_t dst_len,
                       const void *const src, const size_t src_len) {
-    dictionary_t* uninit_dict = create_dictionary();
+    dictionary_t* const uninit_dict = create_dictionary();
    size_t const decomp_size = ZSTD_decompress_with_dict(dst, dst_len, src,
                                                         src_len, uninit_dict);
    free_dictionary(uninit_dict);
@@ -417,12 +427,7 @@ static void decompress_data(frame_context_t *const ctx, ostream_t *const out,
 static void decode_frame(ostream_t *const out, istream_t *const in,
                         const dictionary_t *const dict) {
    const u32 magic_number = (u32)IO_read_bits(in, 32);
-    // Zstandard frame
-    //
-    // "Magic_Number
-    //
-    // 4 Bytes, little-endian format. Value : 0xFD2FB528"
-    if (magic_number == 0xFD2FB528U) {
+    if (magic_number == ZSTD_MAGIC_NUMBER) {
        // ZSTD frame
        decode_data_frame(out, in, dict);

@@ -576,43 +581,6 @@ static void parse_frame_header(frame_header_t *const header,
    }
 }

-/// A dictionary acts as initializing values for the frame context before
-/// decompression, so we implement it by applying it's predetermined
-/// tables and content to the context before beginning decompression
-static void frame_context_apply_dict(frame_context_t *const ctx,
-                                     const dictionary_t *const dict) {
-    // If the content pointer is NULL then it must be an empty dict
-    if (!dict || !dict->content)
-        return;
-
-    // If the requested dictionary_id is non-zero, the correct dictionary must
-    // be present
-    if (ctx->header.dictionary_id != 0 &&
-        ctx->header.dictionary_id != dict->dictionary_id) {
-        ERROR("Wrong dictionary provided");
-    }
-
-    // Copy the dict content to the context for references during sequence
-    // execution
-    ctx->dict_content = dict->content;
-    ctx->dict_content_len = dict->content_size;
-
-    // If it's a formatted dict copy the precomputed tables in so they can
-    // be used in the table repeat modes
-    if (dict->dictionary_id != 0) {
-        // Deep copy the entropy tables so they can be freed independently of
-        // the dictionary struct
-        HUF_copy_dtable(&ctx->literals_dtable, &dict->literals_dtable);
-        FSE_copy_dtable(&ctx->ll_dtable, &dict->ll_dtable);
-        FSE_copy_dtable(&ctx->of_dtable, &dict->of_dtable);
-        FSE_copy_dtable(&ctx->ml_dtable, &dict->ml_dtable);
-
-        // Copy the repeated offsets
-        memcpy(ctx->previous_offsets, dict->previous_offsets,
-               sizeof(ctx->previous_offsets));
-    }
-}
-
 /// Decompress the data from a frame block by block
 static void decompress_data(frame_context_t *const ctx, ostream_t *const out,
                            istream_t *const in) {
@@ -1411,7 +1379,7 @@ size_t ZSTD_get_decompressed_size(const void *src, const size_t src_len) {
    {
        const u32 magic_number = (u32)IO_read_bits(&in, 32);

-        if (magic_number == 0xFD2FB528U) {
+        if (magic_number == ZSTD_MAGIC_NUMBER) {
            // ZSTD frame
            frame_header_t header;
            parse_frame_header(&header, &in);
@@ -1431,17 +1399,33 @@ size_t ZSTD_get_decompressed_size(const void *src, const size_t src_len) {
 /******* END OUTPUT SIZE COUNTING *********************************************/

 /******* DICTIONARY PARSING ***************************************************/
-#define DICT_SIZE_ERROR() ERROR("Dictionary size cannot be less than 8 bytes")
-#define NULL_SRC() ERROR("Tried to create dictionary with pointer to null src");
-
 dictionary_t* create_dictionary() {
-    dictionary_t* dict = calloc(1, sizeof(dictionary_t));
+    dictionary_t* const dict = calloc(1, sizeof(dictionary_t));
    if (!dict) {
        BAD_ALLOC();
    }
    return dict;
 }

+/// Free an allocated dictionary
+void free_dictionary(dictionary_t *const dict) {
+    HUF_free_dtable(&dict->literals_dtable);
+    FSE_free_dtable(&dict->ll_dtable);
+    FSE_free_dtable(&dict->of_dtable);
+    FSE_free_dtable(&dict->ml_dtable);
+
+    free(dict->content);
+
+    memset(dict, 0, sizeof(dictionary_t));
+
+    free(dict);
+}
+
+
+#if !defined(ZDEC_NO_DICTIONARY)
+#define DICT_SIZE_ERROR() ERROR("Dictionary size cannot be less than 8 bytes")
+#define NULL_SRC() ERROR("Tried to create dictionary with pointer to null src");
+
 static void init_dictionary_content(dictionary_t *const dict,
                                    istream_t *const in);

@@ -1513,19 +1497,93 @@ static void init_dictionary_content(dictionary_t *const dict,
    memcpy(dict->content, content, dict->content_size);
 }

-/// Free an allocated dictionary
-void free_dictionary(dictionary_t *const dict) {
-    HUF_free_dtable(&dict->literals_dtable);
-    FSE_free_dtable(&dict->ll_dtable);
-    FSE_free_dtable(&dict->of_dtable);
-    FSE_free_dtable(&dict->ml_dtable);
+static void HUF_copy_dtable(HUF_dtable *const dst,
+                            const HUF_dtable *const src) {
+    if (src->max_bits == 0) {
+        memset(dst, 0, sizeof(HUF_dtable));
+        return;
+    }

-    free(dict->content);
+    const size_t size = (size_t)1 << src->max_bits;
+    dst->max_bits = src->max_bits;

-    memset(dict, 0, sizeof(dictionary_t));
+    dst->symbols = malloc(size);
+    dst->num_bits = malloc(size);
+    if (!dst->symbols || !dst->num_bits) {
+        BAD_ALLOC();
+    }

-    free(dict);
+    memcpy(dst->symbols, src->symbols, size);
+    memcpy(dst->num_bits, src->num_bits, size);
 }
+
+static void FSE_copy_dtable(FSE_dtable *const dst, const FSE_dtable *const src) {
+    if (src->accuracy_log == 0) {
+        memset(dst, 0, sizeof(FSE_dtable));
+        return;
+    }
+
+    size_t size = (size_t)1 << src->accuracy_log;
+    dst->accuracy_log = src->accuracy_log;
+
+    dst->symbols = malloc(size);
+    dst->num_bits = malloc(size);
+    dst->new_state_base = malloc(size * sizeof(u16));
+    if (!dst->symbols || !dst->num_bits || !dst->new_state_base) {
+        BAD_ALLOC();
+    }
+
+    memcpy(dst->symbols, src->symbols, size);
+    memcpy(dst->num_bits, src->num_bits, size);
+    memcpy(dst->new_state_base, src->new_state_base, size * sizeof(u16));
+}
+
+/// A dictionary acts as initializing values for the frame context before
+/// decompression, so we implement it by applying it's predetermined
+/// tables and content to the context before beginning decompression
+static void frame_context_apply_dict(frame_context_t *const ctx,
+                                     const dictionary_t *const dict) {
+    // If the content pointer is NULL then it must be an empty dict
+    if (!dict || !dict->content)
+        return;
+
+    // If the requested dictionary_id is non-zero, the correct dictionary must
+    // be present
+    if (ctx->header.dictionary_id != 0 &&
+        ctx->header.dictionary_id != dict->dictionary_id) {
+        ERROR("Wrong dictionary provided");
+    }
+
+    // Copy the dict content to the context for references during sequence
+    // execution
+    ctx->dict_content = dict->content;
+    ctx->dict_content_len = dict->content_size;
+
+    // If it's a formatted dict copy the precomputed tables in so they can
+    // be used in the table repeat modes
+    if (dict->dictionary_id != 0) {
+        // Deep copy the entropy tables so they can be freed independently of
+        // the dictionary struct
+        HUF_copy_dtable(&ctx->literals_dtable, &dict->literals_dtable);
+        FSE_copy_dtable(&ctx->ll_dtable, &dict->ll_dtable);
+        FSE_copy_dtable(&ctx->of_dtable, &dict->of_dtable);
+        FSE_copy_dtable(&ctx->ml_dtable, &dict->ml_dtable);
+
+        // Copy the repeated offsets
+        memcpy(ctx->previous_offsets, dict->previous_offsets,
+               sizeof(ctx->previous_offsets));
+    }
+}
+
+#else  // ZDEC_NO_DICTIONARY is defined
+
+static void frame_context_apply_dict(frame_context_t *const ctx,
+                                     const dictionary_t *const dict) {
+    (void)ctx;
+    if (dict && dict->content) ERROR("dictionary not supported");
+}
+
+#endif
 /******* END DICTIONARY PARSING ***********************************************/

 /******* IO STREAM OPERATIONS *************************************************/
@@ -1945,26 +2003,6 @@ static void HUF_free_dtable(HUF_dtable *const dtable) {
    free(dtable->num_bits);
    memset(dtable, 0, sizeof(HUF_dtable));
 }
-
-static void HUF_copy_dtable(HUF_dtable *const dst,
-                            const HUF_dtable *const src) {
-    if (src->max_bits == 0) {
-        memset(dst, 0, sizeof(HUF_dtable));
-        return;
-    }
-
-    const size_t size = (size_t)1 << src->max_bits;
-    dst->max_bits = src->max_bits;
-
-    dst->symbols = malloc(size);
-    dst->num_bits = malloc(size);
-    if (!dst->symbols || !dst->num_bits) {
-        BAD_ALLOC();
-    }
-
-    memcpy(dst->symbols, src->symbols, size);
-    memcpy(dst->num_bits, src->num_bits, size);
-}
 /******* END HUFFMAN PRIMITIVES ***********************************************/

 /******* FSE PRIMITIVES *******************************************************/
@@ -2279,25 +2317,4 @@ static void FSE_free_dtable(FSE_dtable *const dtable) {
    free(dtable->new_state_base);
    memset(dtable, 0, sizeof(FSE_dtable));
 }
-
-static void FSE_copy_dtable(FSE_dtable *const dst, const FSE_dtable *const src) {
-    if (src->accuracy_log == 0) {
-        memset(dst, 0, sizeof(FSE_dtable));
-        return;
-    }
-
-    size_t size = (size_t)1 << src->accuracy_log;
-    dst->accuracy_log = src->accuracy_log;
-
-    dst->symbols = malloc(size);
-    dst->num_bits = malloc(size);
-    dst->new_state_base = malloc(size * sizeof(u16));
-    if (!dst->symbols || !dst->num_bits || !dst->new_state_base) {
-        BAD_ALLOC();
-    }
-
-    memcpy(dst->symbols, src->symbols, size);
-    memcpy(dst->num_bits, src->num_bits, size);
-    memcpy(dst->new_state_base, src->new_state_base, size * sizeof(u16));
-}
 /******* END FSE PRIMITIVES ***************************************************/
@@ -1,10 +1,11 @@
 /*
- * Copyright (c) 2016-present, Facebook, Inc.
+ * Copyright (c) 2016-2020, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
 * LICENSE file in the root directory of this source tree) and the GPLv2 (found
 * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 */

 #include <stddef.h>   /* size_t */
@@ -3,7 +3,7 @@ Zstandard Compression Format

 ### Notices

-Copyright (c) 2016-present Yann Collet, Facebook, Inc.
+Copyright (c) 2016-2020 Yann Collet, Facebook, Inc.

 Permission is granted to copy and distribute this document
 for any purpose and without charge,
@@ -16,7 +16,7 @@ Distribution of this document is unlimited.

 ### Version

-0.3.5 (13/11/19)
+0.3.6 (25/05/20)


 Introduction
@@ -291,21 +291,10 @@ Format is __little-endian__.
 It's allowed to represent a small ID (for example `13`)
 with a large 4-bytes dictionary ID, even if it is less efficient.

-_Reserved ranges :_
-Within private environments, any `Dictionary_ID` can be used.
-
-However, for frames and dictionaries distributed in public space,
-`Dictionary_ID` must be attributed carefully.
-Rules for public environment are not yet decided,
-but the following ranges are reserved for some future registrar :
- low range  : `<= 32767`
- high range : `>= (1 << 31)`
-
-Outside of these ranges, any value of `Dictionary_ID`
-which is both `>= 32768` and `< (1<<31)` can be used freely,
-even in public environment.
-
-
+A value of `0` has same meaning as no `Dictionary_ID`,
+in which case the frame may or may not need a dictionary to be decoded,
+and the ID of such a dictionary is not specified.
+The decoder must know this information by other means.

 #### `Frame_Content_Size`

@@ -389,7 +378,7 @@ __`Block_Size`__
 The upper 21 bits of `Block_Header` represent the `Block_Size`.

 When `Block_Type` is `Compressed_Block` or `Raw_Block`,
-`Block_Size` is the size of `Block_Content` (hence excluding `Block_Header`).  
+`Block_Size` is the size of `Block_Content` (hence excluding `Block_Header`).

 When `Block_Type` is `RLE_Block`, since `Block_Content`’s size is always 1,
 `Block_Size` represents the number of times this byte must be repeated.
@@ -1429,13 +1418,17 @@ __`Dictionary_ID`__ : 4 bytes, stored in __little-endian__ format.
              It's used by decoders to check if they use the correct dictionary.

 _Reserved ranges :_
-              If the frame is going to be distributed in a private environment,
-              any `Dictionary_ID` can be used.
-              However, for public distribution of compressed frames,
-              the following ranges are reserved and shall not be used :
+If the dictionary is going to be distributed in a public environment,
+the following ranges of `Dictionary_ID` are reserved for some future registrar
+and shall not be used :
+
+    - low range  : <= 32767
+    - high range : >= (2^31)
+
+Outside of these ranges, any value of `Dictionary_ID`
+which is both `>= 32768` and `< (1<<31)` can be used freely,
+even in public environment.

-              - low range  : <= 32767
-              - high range : >= (2^31)

 __`Entropy_Tables`__ : follow the same format as tables in [compressed blocks].
              See the relevant [FSE](#fse-table-description)
@@ -1455,7 +1448,7 @@ __`Content`__ : The rest of the dictionary is its content.
              As long as the amount of data decoded from this frame is less than or
              equal to `Window_Size`, sequence commands may specify offsets longer
              than the total length of decoded output so far to reference back to the
-              dictionary, even parts of the dictionary with offsets larger than `Window_Size`.  
+              dictionary, even parts of the dictionary with offsets larger than `Window_Size`.
              After the total output has surpassed `Window_Size` however,
              this is no longer allowed and the dictionary is no longer accessible.

@@ -1673,6 +1666,7 @@ or at least provide a meaningful error code explaining for which reason it canno

 Version changes
 ---------------
+- 0.3.6 : clarifications for Dictionary_ID
 - 0.3.5 : clarifications for Block_Maximum_Size
 - 0.3.4 : clarifications for FSE decoding table
 - 0.3.3 : clarifications for field Block_Size
@@ -1,10 +1,10 @@
 <html>
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
-<title>zstd 1.4.4 Manual</title>
+<title>zstd 1.4.5 Manual</title>
 </head>
 <body>
-<h1>zstd 1.4.4 Manual</h1>
+<h1>zstd 1.4.5 Manual</h1>
 <hr>
 <a name="Contents"></a><h2>Contents</h2>
 <ol>
@@ -217,7 +217,10 @@ size_t     ZSTD_freeDCtx(ZSTD_DCtx* dctx);
                              * Default level is ZSTD_CLEVEL_DEFAULT==3.
                              * Special: value 0 means default, which is controlled by ZSTD_CLEVEL_DEFAULT.
                              * Note 1 : it's possible to pass a negative compression level.
-                              * Note 2 : setting a level resets all other compression parameters to default */
+                              * Note 2 : setting a level does not automatically set all other compression parameters
+                              *   to default. Setting this will however eventually dynamically impact the compression
+                              *   parameters which have not been manually set. The manually set
+                              *   ones will 'stick'. */
    </b>/* Advanced compression parameters :<b>
     * It's possible to pin down compression parameters to some specific values.
     * In which case, these values are no longer dynamically selected by the compressor */
@@ -451,11 +454,13 @@ size_t     ZSTD_freeDCtx(ZSTD_DCtx* dctx);
    </b>/* note : additional experimental parameters are also available<b>
     * within the experimental section of the API.
     * At the time of this writing, they include :
-     * ZSTD_c_format
+     * ZSTD_d_format
+     * ZSTD_d_stableOutBuffer
     * Because they are not stable, it's necessary to define ZSTD_STATIC_LINKING_ONLY to access them.
     * note : never ever use experimentalParam? names directly
     */
-     ZSTD_d_experimentalParam1=1000
+     ZSTD_d_experimentalParam1=1000,
+     ZSTD_d_experimentalParam2=1001

 } ZSTD_dParameter;
 </b></pre><BR>
@@ -1055,23 +1060,28 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
 size_t ZSTD_estimateCCtxSize_usingCParams(ZSTD_compressionParameters cParams);
 size_t ZSTD_estimateCCtxSize_usingCCtxParams(const ZSTD_CCtx_params* params);
 size_t ZSTD_estimateDCtxSize(void);
-</b><p>  These functions make it possible to estimate memory usage of a future
-  {D,C}Ctx, before its creation.
+</b><p>  These functions make it possible to estimate memory usage
+  of a future {D,C}Ctx, before its creation.

-  ZSTD_estimateCCtxSize() will provide a budget large enough for any
-  compression level up to selected one. Unlike ZSTD_estimateCStreamSize*(),
-  this estimate does not include space for a window buffer, so this estimate
-  is guaranteed to be enough for single-shot compressions, but not streaming
-  compressions. It will however assume the input may be arbitrarily large,
-  which is the worst case. If srcSize is known to always be small,
-  ZSTD_estimateCCtxSize_usingCParams() can provide a tighter estimation.
-  ZSTD_estimateCCtxSize_usingCParams() can be used in tandem with
-  ZSTD_getCParams() to create cParams from compressionLevel.
-  ZSTD_estimateCCtxSize_usingCCtxParams() can be used in tandem with
-  ZSTD_CCtxParams_setParameter().
+  ZSTD_estimateCCtxSize() will provide a memory budget large enough
+  for any compression level up to selected one.
+  Note : Unlike ZSTD_estimateCStreamSize*(), this estimate
+         does not include space for a window buffer.
+         Therefore, the estimation is only guaranteed for single-shot compressions, not streaming.
+  The estimate will assume the input may be arbitrarily large,
+  which is the worst case.

-  Note: only single-threaded compression is supported. This function will
-  return an error code if ZSTD_c_nbWorkers is >= 1. 
+  When srcSize can be bound by a known and rather "small" value,
+  this fact can be used to provide a tighter estimation
+  because the CCtx compression context will need less memory.
+  This tighter estimation can be provided by more advanced functions
+  ZSTD_estimateCCtxSize_usingCParams(), which can be used in tandem with ZSTD_getCParams(),
+  and ZSTD_estimateCCtxSize_usingCCtxParams(), which can be used in tandem with ZSTD_CCtxParams_setParameter().
+  Both can be used to estimate memory using custom compression parameters and arbitrary srcSize limits.
+
+  Note 2 : only single-threaded compression is supported.
+  ZSTD_estimateCCtxSize_usingCCtxParams() will return an error code if ZSTD_c_nbWorkers is >= 1.
+ 
 </p></pre><BR>

 <pre><b>size_t ZSTD_estimateCStreamSize(int compressionLevel);
@@ -1,10 +1,11 @@
 # ################################################################
-# Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+# Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 # All rights reserved.
 #
 # This source code is licensed under both the BSD-style license (found in the
 # LICENSE file in the root directory of this source tree) and the GPLv2 (found
 # in the COPYING file in the root directory of this source tree).
+# You may select, at your option, one of the above-listed licenses.
 # ################################################################

 # This Makefile presumes libzstd is installed, using `sudo make install`
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020 Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -39,6 +39,7 @@ static void compressFile_orDie(const char* fname, const char* outName, int cLeve
     */
    CHECK_ZSTD( ZSTD_CCtx_setParameter(cctx, ZSTD_c_compressionLevel, cLevel) );
    CHECK_ZSTD( ZSTD_CCtx_setParameter(cctx, ZSTD_c_checksumFlag, 1) );
+    ZSTD_CCtx_setParameter(cctx, ZSTD_c_nbWorkers, 4);

    /* This loop read from the input file, compresses that entire chunk,
     * and writes all output produced to the output file.
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2017-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2017-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,12 +1,24 @@
 # ################################################################
-# Copyright (c) 2015-present, Yann Collet, Facebook, Inc.
+# Copyright (c) 2015-2020, Yann Collet, Facebook, Inc.
 # All rights reserved.
 #
 # This source code is licensed under both the BSD-style license (found in the
 # LICENSE file in the root directory of this source tree) and the GPLv2 (found
 # in the COPYING file in the root directory of this source tree).
+# You may select, at your option, one of the above-listed licenses.
 # ################################################################

+Q = $(if $(filter 1,$(V) $(VERBOSE)),,@)
+
+# When cross-compiling from linux to windows, you might
+# need to specify this as "Windows." Fedora build fails
+# without it.
+#
+# Note: mingw-w64 build from linux to windows does not
+# fail on other tested distros (ubuntu, debian) even
+# without manually specifying the TARGET_SYSTEM.
+TARGET_SYSTEM ?= $(OS)
+
 # Version numbers
 LIBVER_MAJOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MAJOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./zstd.h`
 LIBVER_MINOR_SCRIPT:=`sed -n '/define ZSTD_VERSION_MINOR/s/.*[[:blank:]]\([0-9][0-9]*\).*/\1/p' < ./zstd.h`
@@ -19,11 +31,10 @@ LIBVER := $(shell echo $(LIBVER_SCRIPT))
 VERSION?= $(LIBVER)
 CCVER := $(shell $(CC) --version)

-CPPFLAGS+= -I. -I./common -DXXH_NAMESPACE=ZSTD_
-ifeq ($(OS),Windows_NT)   # MinGW assumed
+CPPFLAGS+= -DXXH_NAMESPACE=ZSTD_
+ifeq ($(TARGET_SYSTEM),Windows_NT)   # MinGW assumed
 CPPFLAGS   += -D__USE_MINGW_ANSI_STDIO   # compatibility with %zu formatting
 endif
-CFLAGS  ?= -O3
 DEBUGFLAGS= -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow \
            -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement \
            -Wstrict-prototypes -Wundef -Wpointer-arith \
@@ -38,6 +49,7 @@ ifeq ($HAVE_COLORNEVER, 1)
 GREP_OPTIONS += --color=never
 endif
 GREP = grep $(GREP_OPTIONS)
+SED_ERE_OPT ?= -E

 ZSTDCOMMON_FILES := $(sort $(wildcard common/*.c))
 ZSTDCOMP_FILES := $(sort $(wildcard compress/*.c))
@@ -50,18 +62,46 @@ ifeq ($(findstring GCC,$(CCVER)),GCC)
 decompress/zstd_decompress_block.o :	CFLAGS+=-fno-tree-vectorize
 endif

-ZSTD_LEGACY_SUPPORT ?= 5
+# This is a helper variable that configures a bunch of other variables to new,
+# space-optimized defaults.
+ZSTD_LIB_MINIFY ?= 0
+ifneq ($(ZSTD_LIB_MINIFY), 0)
+	HAVE_CC_OZ ?= $(shell echo "" | $(CC) -Oz -x c -c - -o /dev/null 2> /dev/null && echo 1 || echo 0)
+	ZSTD_LEGACY_SUPPORT ?= 0
+	ZSTD_LIB_DEPRECATED ?= 0
+	HUF_FORCE_DECOMPRESS_X1 ?= 1
+	ZSTD_FORCE_DECOMPRESS_SHORT ?= 1
+	ZSTD_NO_INLINE ?= 1
+	ZSTD_STRIP_ERROR_STRINGS ?= 1
+	ifneq ($(HAVE_CC_OZ), 0)
+		# Some compilers (clang) support an even more space-optimized setting.
+		CFLAGS += -Oz
+	else
+		CFLAGS += -Os
+	endif
+	CFLAGS += -fno-stack-protector -fomit-frame-pointer -fno-ident \
+	          -DDYNAMIC_BMI2=0 -DNDEBUG
+else
+	CFLAGS += -O3
+endif
+
+# Modules
 ZSTD_LIB_COMPRESSION ?= 1
 ZSTD_LIB_DECOMPRESSION ?= 1
 ZSTD_LIB_DICTBUILDER ?= 1
 ZSTD_LIB_DEPRECATED ?= 1
+
+# Legacy support
+ZSTD_LEGACY_SUPPORT ?= 5
+ZSTD_LEGACY_MULTITHREADED_API ?= 0
+
+# Build size optimizations
 HUF_FORCE_DECOMPRESS_X1 ?= 0
 HUF_FORCE_DECOMPRESS_X2 ?= 0
 ZSTD_FORCE_DECOMPRESS_SHORT ?= 0
 ZSTD_FORCE_DECOMPRESS_LONG ?= 0
 ZSTD_NO_INLINE ?= 0
 ZSTD_STRIP_ERROR_STRINGS ?= 0
-ZSTD_LEGACY_MULTITHREADED_API ?= 0

 ifeq ($(ZSTD_LIB_COMPRESSION), 0)
 	ZSTD_LIB_DICTBUILDER = 0
@@ -121,7 +161,6 @@ ifneq ($(ZSTD_LEGACY_SUPPORT), 0)
 ifeq ($(shell test $(ZSTD_LEGACY_SUPPORT) -lt 8; echo $$?), 0)
 	ZSTD_FILES += $(shell ls legacy/*.c | $(GREP) 'v0[$(ZSTD_LEGACY_SUPPORT)-7]')
 endif
-	CPPFLAGS += -I./legacy
 endif
 CPPFLAGS  += -DZSTD_LEGACY_SUPPORT=$(ZSTD_LEGACY_SUPPORT)

@@ -142,21 +181,21 @@ else
 endif


-.PHONY: default all clean install uninstall
+.PHONY: default lib-all all clean install uninstall

 default: lib-release

+# alias
+lib-all: all
+
 all: lib

 libzstd.a: ARFLAGS = rcs
 libzstd.a: $(ZSTD_OBJ)
 	@echo compiling static library
-	@$(AR) $(ARFLAGS) $@ $^
+	$(Q)$(AR) $(ARFLAGS) $@ $^

-libzstd.a-mt: CPPFLAGS += -DZSTD_MULTITHREAD
-libzstd.a-mt: libzstd.a
-
-ifneq (,$(filter Windows%,$(OS)))
+ifneq (,$(filter Windows%,$(TARGET_SYSTEM)))

 LIBZSTD = dll\libzstd.dll
 $(LIBZSTD): $(ZSTD_FILES)
@@ -169,27 +208,31 @@ LIBZSTD = libzstd.$(SHARED_EXT_VER)
 $(LIBZSTD): LDFLAGS += -shared -fPIC -fvisibility=hidden
 $(LIBZSTD): $(ZSTD_FILES)
 	@echo compiling dynamic library $(LIBVER)
-	@$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@
+	$(Q)$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@
 	@echo creating versioned links
-	@ln -sf $@ libzstd.$(SHARED_EXT_MAJOR)
-	@ln -sf $@ libzstd.$(SHARED_EXT)
+	$(Q)ln -sf $@ libzstd.$(SHARED_EXT_MAJOR)
+	$(Q)ln -sf $@ libzstd.$(SHARED_EXT)

 endif

-
+.PHONY: libzstd
 libzstd : $(LIBZSTD)

-libzstd-mt : CPPFLAGS += -DZSTD_MULTITHREAD
-libzstd-mt : libzstd
+.PHONY: lib
+lib : libzstd.a libzstd

-lib: libzstd.a libzstd
+# note : do not define lib-mt or lib-release as .PHONY
+# make does not consider implicit pattern rule for .PHONY target

-lib-mt: CPPFLAGS += -DZSTD_MULTITHREAD
-lib-mt: lib
+%-mt : CPPFLAGS += -DZSTD_MULTITHREAD
+%-mt : LDFLAGS  += -pthread
+%-mt : %
+	@echo multi-threading build completed
+
+%-release : DEBUGFLAGS :=
+%-release : %
+	@echo release build completed

-lib-release lib-release-mt: DEBUGFLAGS :=
-lib-release: lib
-lib-release-mt: lib-mt

 # Special case : building library in single-thread mode _and_ without zstdmt_compress.c
 ZSTDMT_FILES = compress/zstdmt_compress.c
@@ -198,20 +241,22 @@ libzstd-nomt: LDFLAGS += -shared -fPIC -fvisibility=hidden
 libzstd-nomt: $(ZSTD_NOMT_FILES)
 	@echo compiling single-thread dynamic library $(LIBVER)
 	@echo files : $(ZSTD_NOMT_FILES)
-	@$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@
+	$(Q)$(CC) $(FLAGS) $^ $(LDFLAGS) $(SONAME_FLAGS) -o $@

 clean:
-	@$(RM) -r *.dSYM   # macOS-specific
-	@$(RM) core *.o *.a *.gcda *.$(SHARED_EXT) *.$(SHARED_EXT).* libzstd.pc
-	@$(RM) dll/libzstd.dll dll/libzstd.lib libzstd-nomt*
-	@$(RM) common/*.o compress/*.o decompress/*.o dictBuilder/*.o legacy/*.o deprecated/*.o
+	$(Q)$(RM) -r *.dSYM   # macOS-specific
+	$(Q)$(RM) core *.o *.a *.gcda *.$(SHARED_EXT) *.$(SHARED_EXT).* libzstd.pc
+	$(Q)$(RM) dll/libzstd.dll dll/libzstd.lib libzstd-nomt*
+	$(Q)$(RM) common/*.o compress/*.o decompress/*.o dictBuilder/*.o legacy/*.o deprecated/*.o
 	@echo Cleaning library completed

 #-----------------------------------------------------------------------------
-# make install is validated only for Linux, macOS, BSD, Hurd and Solaris targets
+# make install is validated only for below listed environments
 #-----------------------------------------------------------------------------
 ifneq (,$(filter $(shell uname),Linux Darwin GNU/kFreeBSD GNU OpenBSD FreeBSD NetBSD DragonFly SunOS Haiku))

+all: libzstd.pc
+
 DESTDIR     ?=
 # directory variables : GNU conventions prefer lowercase
 # see https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html
@@ -219,11 +264,31 @@ DESTDIR     ?=
 prefix      ?= /usr/local
 PREFIX      ?= $(prefix)
 exec_prefix ?= $(PREFIX)
-libdir      ?= $(exec_prefix)/lib
+EXEC_PREFIX ?= $(exec_prefix)
+libdir      ?= $(EXEC_PREFIX)/lib
 LIBDIR      ?= $(libdir)
 includedir  ?= $(PREFIX)/include
 INCLUDEDIR  ?= $(includedir)

+PCLIBDIR ?= $(shell echo "$(LIBDIR)" | sed -n $(SED_ERE_OPT) -e "s@^$(EXEC_PREFIX)(/|$$)@@p")
+PCINCDIR ?= $(shell echo "$(INCLUDEDIR)" | sed -n $(SED_ERE_OPT) -e "s@^$(PREFIX)(/|$$)@@p")
+
+ifeq (,$(PCLIBDIR))
+# Additional prefix check is required, since the empty string is technically a
+# valid PCLIBDIR
+ifeq (,$(shell echo "$(LIBDIR)" | sed -n $(SED_ERE_OPT) -e "\\@^$(EXEC_PREFIX)(/|$$)@ p"))
+$(error configured libdir ($(LIBDIR)) is outside of prefix ($(PREFIX)), can't generate pkg-config file)
+endif
+endif
+
+ifeq (,$(PCINCDIR))
+# Additional prefix check is required, since the empty string is technically a
+# valid PCINCDIR
+ifeq (,$(shell echo "$(INCLUDEDIR)" | sed -n $(SED_ERE_OPT) -e "\\@^$(PREFIX)(/|$$)@ p"))
+$(error configured includedir ($(INCLUDEDIR)) is outside of exec_prefix ($(EXEC_PREFIX)), can't generate pkg-config file)
+endif
+endif
+
 ifneq (,$(filter $(shell uname),FreeBSD NetBSD DragonFly))
 PKGCONFIGDIR ?= $(PREFIX)/libdata/pkgconfig
 else
@@ -243,47 +308,47 @@ INSTALL_DATA    ?= $(INSTALL) -m 644
 libzstd.pc:
 libzstd.pc: libzstd.pc.in
 	@echo creating pkgconfig
-	@sed -e 's|@PREFIX@|$(PREFIX)|' \
-             -e 's|@VERSION@|$(VERSION)|' \
-             $< >$@
+	$(Q)@sed $(SED_ERE_OPT) -e 's|@PREFIX@|$(PREFIX)|' \
+          -e 's|@LIBDIR@|$(PCLIBDIR)|' \
+          -e 's|@INCLUDEDIR@|$(PCINCDIR)|' \
+          -e 's|@VERSION@|$(VERSION)|' \
+          $< >$@

 install: install-pc install-static install-shared install-includes
 	@echo zstd static and shared library installed

 install-pc: libzstd.pc
-	@$(INSTALL) -d -m 755 $(DESTDIR)$(PKGCONFIGDIR)/
-	@$(INSTALL_DATA) libzstd.pc $(DESTDIR)$(PKGCONFIGDIR)/
+	$(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(PKGCONFIGDIR)/
+	$(Q)$(INSTALL_DATA) libzstd.pc $(DESTDIR)$(PKGCONFIGDIR)/

 install-static: libzstd.a
 	@echo Installing static library
-	@$(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/
-	@$(INSTALL_DATA) libzstd.a $(DESTDIR)$(LIBDIR)
+	$(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/
+	$(Q)$(INSTALL_DATA) libzstd.a $(DESTDIR)$(LIBDIR)

 install-shared: libzstd
 	@echo Installing shared library
-	@$(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/
-	@$(INSTALL_PROGRAM) $(LIBZSTD) $(DESTDIR)$(LIBDIR)
-	@ln -sf $(LIBZSTD) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR)
-	@ln -sf $(LIBZSTD) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT)
+	$(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(LIBDIR)/
+	$(Q)$(INSTALL_PROGRAM) $(LIBZSTD) $(DESTDIR)$(LIBDIR)
+	$(Q)ln -sf $(LIBZSTD) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR)
+	$(Q)ln -sf $(LIBZSTD) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT)

 install-includes:
 	@echo Installing includes
-	@$(INSTALL) -d -m 755 $(DESTDIR)$(INCLUDEDIR)/
-	@$(INSTALL_DATA) zstd.h $(DESTDIR)$(INCLUDEDIR)
-	@$(INSTALL_DATA) common/zstd_errors.h $(DESTDIR)$(INCLUDEDIR)
-	@$(INSTALL_DATA) deprecated/zbuff.h $(DESTDIR)$(INCLUDEDIR)     # prototypes generate deprecation warnings
-	@$(INSTALL_DATA) dictBuilder/zdict.h $(DESTDIR)$(INCLUDEDIR)
+	$(Q)$(INSTALL) -d -m 755 $(DESTDIR)$(INCLUDEDIR)/
+	$(Q)$(INSTALL_DATA) zstd.h $(DESTDIR)$(INCLUDEDIR)
+	$(Q)$(INSTALL_DATA) common/zstd_errors.h $(DESTDIR)$(INCLUDEDIR)
+	$(Q)$(INSTALL_DATA) dictBuilder/zdict.h $(DESTDIR)$(INCLUDEDIR)

 uninstall:
-	@$(RM) $(DESTDIR)$(LIBDIR)/libzstd.a
-	@$(RM) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT)
-	@$(RM) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR)
-	@$(RM) $(DESTDIR)$(LIBDIR)/$(LIBZSTD)
-	@$(RM) $(DESTDIR)$(PKGCONFIGDIR)/libzstd.pc
-	@$(RM) $(DESTDIR)$(INCLUDEDIR)/zstd.h
-	@$(RM) $(DESTDIR)$(INCLUDEDIR)/zstd_errors.h
-	@$(RM) $(DESTDIR)$(INCLUDEDIR)/zbuff.h   # Deprecated streaming functions
-	@$(RM) $(DESTDIR)$(INCLUDEDIR)/zdict.h
+	$(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.a
+	$(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT)
+	$(Q)$(RM) $(DESTDIR)$(LIBDIR)/libzstd.$(SHARED_EXT_MAJOR)
+	$(Q)$(RM) $(DESTDIR)$(LIBDIR)/$(LIBZSTD)
+	$(Q)$(RM) $(DESTDIR)$(PKGCONFIGDIR)/libzstd.pc
+	$(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zstd.h
+	$(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zstd_errors.h
+	$(Q)$(RM) $(DESTDIR)$(INCLUDEDIR)/zdict.h
 	@echo zstd libraries successfully uninstalled

 endif
@@ -85,28 +85,48 @@ The file structure is designed to make this selection manually achievable for an

 - While invoking `make libzstd`, it's possible to define build macros
        `ZSTD_LIB_COMPRESSION, ZSTD_LIB_DECOMPRESSION`, `ZSTD_LIB_DICTBUILDER`,
-        and `ZSTD_LIB_DEPRECATED` as `0` to forgo compilation of the corresponding features.
-        This will also disable compilation of all dependencies
-        (eg. `ZSTD_LIB_COMPRESSION=0` will also disable dictBuilder).
+        and `ZSTD_LIB_DEPRECATED` as `0` to forgo compilation of the
+        corresponding features. This will also disable compilation of all
+        dependencies (eg. `ZSTD_LIB_COMPRESSION=0` will also disable
+        dictBuilder).

- There are some additional build macros that can be used to minify the decoder.
+- There are a number of options that can help minimize the binary size of
+  `libzstd`.

-  Zstandard often has more than one implementation of a piece of functionality,
-  where each implementation optimizes for different scenarios. For example, the
-  Huffman decoder has complementary implementations that decode the stream one
-  symbol at a time or two symbols at a time. Zstd normally includes both (and
-  dispatches between them at runtime), but by defining `HUF_FORCE_DECOMPRESS_X1`
-  or `HUF_FORCE_DECOMPRESS_X2`, you can force the use of one or the other, avoiding
+  The first step is to select the components needed (using the above-described
+  `ZSTD_LIB_COMPRESSION` etc.).
+
+  The next step is to set `ZSTD_LIB_MINIFY` to `1` when invoking `make`. This
+  disables various optional components and changes the compilation flags to
+  prioritize space-saving.
+
+  Detailed options: Zstandard's code and build environment is set up by default
+  to optimize above all else for performance. In pursuit of this goal, Zstandard
+  makes significant trade-offs in code size. For example, Zstandard often has
+  more than one implementation of a particular component, with each
+  implementation optimized for different scenarios. For example, the Huffman
+  decoder has complementary implementations that decode the stream one symbol at
+  a time or two symbols at a time. Zstd normally includes both (and dispatches
+  between them at runtime), but by defining `HUF_FORCE_DECOMPRESS_X1` or
+  `HUF_FORCE_DECOMPRESS_X2`, you can force the use of one or the other, avoiding
  compilation of the other. Similarly, `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT`
  and `ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG` force the compilation and use of
  only one or the other of two decompression implementations. The smallest
  binary is achieved by using `HUF_FORCE_DECOMPRESS_X1` and
-  `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT`.
+  `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT` (implied by `ZSTD_LIB_MINIFY`).

  For squeezing the last ounce of size out, you can also define
  `ZSTD_NO_INLINE`, which disables inlining, and `ZSTD_STRIP_ERROR_STRINGS`,
  which removes the error messages that are otherwise returned by
-  `ZSTD_getErrorName`.
+  `ZSTD_getErrorName` (implied by `ZSTD_LIB_MINIFY`).
+
+  Finally, when integrating into your application, make sure you're doing link-
+  time optimation and unused symbol garbage collection (via some combination of,
+  e.g., `-flto`, `-ffat-lto-objects`, `-fuse-linker-plugin`,
+  `-ffunction-sections`, `-fdata-sections`, `-fmerge-all-constants`,
+  `-Wl,--gc-sections`, `-Wl,-z,norelro`, and an archiver that understands
+  the compiler's intermediate representation, e.g., `AR=gcc-ar`). Consult your
+  compiler's documentation.

 - While invoking `make libzstd`, the build macro `ZSTD_LEGACY_MULTITHREADED_API=1`
  will expose the deprecated `ZSTDMT` API exposed by `zstdmt_compress.h` in
@@ -1,35 +1,15 @@
 /* ******************************************************************
-   bitstream
-   Part of FSE library
-   Copyright (C) 2013-present, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-   You can contact the author at :
-   - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ * bitstream
+ * Part of FSE library
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ * You can contact the author at :
+ * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */
 #ifndef BITSTREAM_H_MODULE
 #define BITSTREAM_H_MODULE
@@ -162,8 +142,7 @@ MEM_STATIC unsigned BIT_highbit32 (U32 val)
    {
 #   if defined(_MSC_VER)   /* Visual */
        unsigned long r=0;
-        _BitScanReverse ( &r, val );
-        return (unsigned) r;
+        return _BitScanReverse ( &r, val ) ? (unsigned)r : 0;
 #   elif defined(__GNUC__) && (__GNUC__ >= 3)   /* Use GCC Intrinsic */
        return __builtin_clz (val) ^ 31;
 #   elif defined(__ICCARM__)    /* IAR Intrinsic */
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -17,7 +17,7 @@
 /* force inlining */

 #if !defined(ZSTD_NO_INLINE)
-#if defined (__GNUC__) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L   /* C99 */
+#if (defined(__GNUC__) && !defined(__STRICT_ANSI__)) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L   /* C99 */
 #  define INLINE_KEYWORD inline
 #else
 #  define INLINE_KEYWORD
@@ -38,6 +38,17 @@

 #endif

+/**
+  On MSVC qsort requires that functions passed into it use the __cdecl calling conversion(CC). 
+  This explictly marks such functions as __cdecl so that the code will still compile 
+  if a CC other than __cdecl has been made the default.
+*/
+#if  defined(_MSC_VER)
+#  define WIN_CDECL __cdecl
+#else
+#  define WIN_CDECL 
+#endif
+
 /**
 * FORCE_INLINE_TEMPLATE is used to define C "templates", which take constant
 * parameters. They must be inlined for the compiler to eliminate the constant
@@ -114,6 +125,9 @@
 #    include <mmintrin.h>   /* https://msdn.microsoft.com/fr-fr/library/84szxsww(v=vs.90).aspx */
 #    define PREFETCH_L1(ptr)  _mm_prefetch((const char*)(ptr), _MM_HINT_T0)
 #    define PREFETCH_L2(ptr)  _mm_prefetch((const char*)(ptr), _MM_HINT_T1)
+#    elif defined(__aarch64__)
+#     define PREFETCH_L1(ptr)  __asm__ __volatile__("prfm pldl1keep, %0" ::"Q"(*(ptr)))
+#     define PREFETCH_L2(ptr)  __asm__ __volatile__("prfm pldl2keep, %0" ::"Q"(*(ptr)))
 #  elif defined(__GNUC__) && ( (__GNUC__ >= 4) || ( (__GNUC__ == 3) && (__GNUC_MINOR__ >= 1) ) )
 #    define PREFETCH_L1(ptr)  __builtin_prefetch((ptr), 0 /* rw==read */, 3 /* locality */)
 #    define PREFETCH_L2(ptr)  __builtin_prefetch((ptr), 0 /* rw==read */, 2 /* locality */)
@@ -136,7 +150,7 @@

 /* vectorization
 * older GCC (pre gcc-4.3 picked as the cutoff) uses a different syntax */
-#if !defined(__clang__) && defined(__GNUC__)
+#if !defined(__INTEL_COMPILER) && !defined(__clang__) && defined(__GNUC__)
 #  if (__GNUC__ == 4 && __GNUC_MINOR__ > 3) || (__GNUC__ >= 5)
 #    define DONT_VECTORIZE __attribute__((optimize("no-tree-vectorize")))
 #  else
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2018-present, Facebook, Inc.
+ * Copyright (c) 2018-2020, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,35 +1,15 @@
 /* ******************************************************************
-   debug
-   Part of FSE library
-   Copyright (C) 2013-present, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-   You can contact the author at :
-   - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ * debug
+ * Part of FSE library
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ * You can contact the author at :
+ * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */


@@ -1,35 +1,15 @@
 /* ******************************************************************
-   debug
-   Part of FSE library
-   Copyright (C) 2013-present, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-   You can contact the author at :
-   - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ * debug
+ * Part of FSE library
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ * You can contact the author at :
+ * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */


@@ -99,7 +79,7 @@ extern "C" {
 #  include <assert.h>
 #else
 #  ifndef assert   /* assert may be already defined, due to prior #include <assert.h> */
-#    define assert(condition) ((void)0)   /* disable assert (default) */
+#    define assert(condition) ((void) (condition))   /* disable assert (default) */
 #  endif
 #endif

@@ -1,36 +1,16 @@
-/*
-   Common functions of New Generation Entropy library
-   Copyright (C) 2016, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-    You can contact the author at :
-    - FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy
-    - Public forum : https://groups.google.com/forum/#!forum/lz4c
-*************************************************************************** */
+/* ******************************************************************
+ * Common functions of New Generation Entropy library
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
+ *
+ *  You can contact the author at :
+ *  - FSE+HUF source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *  - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
+****************************************************************** */

 /* *************************************
 *  Dependencies
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -47,6 +47,7 @@ const char* ERR_getErrorString(ERR_enum code)
        /* following error codes are not stable and may be removed or changed in a future version */
    case PREFIX(frameIndex_tooLarge): return "Frame index is too large";
    case PREFIX(seekableIO): return "An I/O error occurred when reading/seeking";
+    case PREFIX(dstBuffer_wrong): return "Destination buffer is wrong";
    case PREFIX(maxCode):
    default: return notErrorCode;
    }
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -49,7 +49,7 @@ typedef ZSTD_ErrorCode ERR_enum;
 /*-****************************************
 *  Error codes handling
 ******************************************/
-#undef ERROR   /* reported already defined on VS 2015 (Rich Geldreich) */
+#undef ERROR   /* already defined on Visual Studio */
 #define ERROR(name) ZSTD_ERROR(name)
 #define ZSTD_ERROR(name) ((size_t)-PREFIX(name))

@@ -57,6 +57,10 @@ ERR_STATIC unsigned ERR_isError(size_t code) { return (code > ERROR(maxCode)); }

 ERR_STATIC ERR_enum ERR_getErrorCode(size_t code) { if (!ERR_isError(code)) return (ERR_enum)0; return (ERR_enum) (0-code); }

+/* check and forward error code */
+#define CHECK_V_F(e, f) size_t const e = f; if (ERR_isError(e)) return e
+#define CHECK_F(f)   { CHECK_V_F(_var_err__, f); }
+

 /*-****************************************
 *  Error Strings
@@ -1,35 +1,15 @@
 /* ******************************************************************
-   FSE : Finite State Entropy codec
-   Public Prototypes declaration
-   Copyright (C) 2013-2016, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-   You can contact the author at :
-   - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ * FSE : Finite State Entropy codec
+ * Public Prototypes declaration
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ * You can contact the author at :
+ * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */

 #if defined (__cplusplus)
@@ -1,35 +1,15 @@
 /* ******************************************************************
-   FSE : Finite State Entropy decoder
-   Copyright (C) 2013-2015, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-    You can contact the author at :
-    - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
-    - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ * FSE : Finite State Entropy decoder
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ *  You can contact the author at :
+ *  - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *  - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */


@@ -51,11 +31,6 @@
 #define FSE_isError ERR_isError
 #define FSE_STATIC_ASSERT(c) DEBUG_STATIC_ASSERT(c)   /* use only *after* variable declarations */

-/* check and forward error code */
-#ifndef CHECK_F
-#define CHECK_F(f) { size_t const e = f; if (FSE_isError(e)) return e; }
-#endif
-

 /* **************************************************************
 *  Templates
@@ -287,7 +262,7 @@ size_t FSE_decompress_wksp(void* dst, size_t dstCapacity, const void* cSrc, size
    /* normal FSE decoding mode */
    size_t const NCountLength = FSE_readNCount (counting, &maxSymbolValue, &tableLog, istart, cSrcSize);
    if (FSE_isError(NCountLength)) return NCountLength;
-    //if (NCountLength >= cSrcSize) return ERROR(srcSize_wrong);   /* too small input size; supposed to be already checked in NCountLength, only remaining case : NCountLength==cSrcSize */
+    /* if (NCountLength >= cSrcSize) return ERROR(srcSize_wrong); */  /* too small input size; supposed to be already checked in NCountLength, only remaining case : NCountLength==cSrcSize */
    if (tableLog > maxLog) return ERROR(tableLog_tooLarge);
    ip += NCountLength;
    cSrcSize -= NCountLength;
@@ -1,35 +1,15 @@
 /* ******************************************************************
-   huff0 huffman codec,
-   part of Finite State Entropy library
-   Copyright (C) 2013-present, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-   You can contact the author at :
-   - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ * huff0 huffman codec,
+ * part of Finite State Entropy library
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ * You can contact the author at :
+ * - Source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */

 #if defined (__cplusplus)
@@ -110,7 +90,7 @@ HUF_PUBLIC_API size_t HUF_compress2 (void* dst, size_t dstCapacity,
 /** HUF_compress4X_wksp() :
 *  Same as HUF_compress2(), but uses externally allocated `workSpace`.
 * `workspace` must have minimum alignment of 4, and be at least as large as HUF_WORKSPACE_SIZE */
-#define HUF_WORKSPACE_SIZE (6 << 10)
+#define HUF_WORKSPACE_SIZE ((6 << 10) + 256)
 #define HUF_WORKSPACE_SIZE_U32 (HUF_WORKSPACE_SIZE / sizeof(U32))
 HUF_PUBLIC_API size_t HUF_compress4X_wksp (void* dst, size_t dstCapacity,
                                     const void* src, size_t srcSize,
@@ -209,6 +189,7 @@ size_t HUF_buildCTable (HUF_CElt* CTable, const unsigned* count, unsigned maxSym
 size_t HUF_writeCTable (void* dst, size_t maxDstSize, const HUF_CElt* CTable, unsigned maxSymbolValue, unsigned huffLog);
 size_t HUF_compress4X_usingCTable(void* dst, size_t dstSize, const void* src, size_t srcSize, const HUF_CElt* CTable);
 size_t HUF_estimateCompressedSize(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue);
+int HUF_validateCTable(const HUF_CElt* CTable, const unsigned* count, unsigned maxSymbolValue);

 typedef enum {
   HUF_repeat_none,  /**< Cannot use the previous table */
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -301,7 +301,7 @@ int POOL_tryAdd(POOL_ctx* ctx, POOL_function function, void* opaque)
 struct POOL_ctx_s {
    int dummy;
 };
-static POOL_ctx g_ctx;
+static POOL_ctx g_poolCtx;

 POOL_ctx* POOL_create(size_t numThreads, size_t queueSize) {
    return POOL_create_advanced(numThreads, queueSize, ZSTD_defaultCMem);
@@ -311,11 +311,11 @@ POOL_ctx* POOL_create_advanced(size_t numThreads, size_t queueSize, ZSTD_customM
    (void)numThreads;
    (void)queueSize;
    (void)customMem;
-    return &g_ctx;
+    return &g_poolCtx;
 }

 void POOL_free(POOL_ctx* ctx) {
-    assert(!ctx || ctx == &g_ctx);
+    assert(!ctx || ctx == &g_poolCtx);
    (void)ctx;
 }

@@ -337,7 +337,7 @@ int POOL_tryAdd(POOL_ctx* ctx, POOL_function function, void* opaque) {

 size_t POOL_sizeof(POOL_ctx* ctx) {
    if (ctx==NULL) return 0;  /* supports sizeof NULL */
-    assert(ctx == &g_ctx);
+    assert(ctx == &g_poolCtx);
    return sizeof(*ctx);
 }

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -18,7 +18,7 @@ extern "C" {

 #include <stddef.h>   /* size_t */
 #define ZSTD_STATIC_LINKING_ONLY   /* ZSTD_customMem */
-#include "zstd.h"
+#include "../zstd.h"

 typedef struct POOL_ctx_s POOL_ctx;

@@ -2,12 +2,13 @@
 * Copyright (c) 2016 Tino Reichardt
 * All rights reserved.
 *
+ * You can contact the author at:
+ * - zstdmt source repository: https://github.com/mcmilk/zstdmt
+ *
 * This source code is licensed under both the BSD-style license (found in the
 * LICENSE file in the root directory of this source tree) and the GPLv2 (found
 * in the COPYING file in the root directory of this source tree).
- *
- * You can contact the author at:
- * - zstdmt source repository: https://github.com/mcmilk/zstdmt
+ * You may select, at your option, one of the above-listed licenses.
 */

 /**
@@ -2,12 +2,13 @@
 * Copyright (c) 2016 Tino Reichardt
 * All rights reserved.
 *
+ * You can contact the author at:
+ * - zstdmt source repository: https://github.com/mcmilk/zstdmt
+ *
 * This source code is licensed under both the BSD-style license (found in the
 * LICENSE file in the root directory of this source tree) and the GPLv2 (found
 * in the COPYING file in the root directory of this source tree).
- *
- * You can contact the author at:
- * - zstdmt source repository: https://github.com/mcmilk/zstdmt
+ * You may select, at your option, one of the above-listed licenses.
 */

 #ifndef THREADING_H_938743
@@ -1,35 +1,15 @@
 /*
-*  xxHash - Fast Hash algorithm
-*  Copyright (C) 2012-2016, Yann Collet
-*
-*  BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-*
-*  Redistribution and use in source and binary forms, with or without
-*  modification, are permitted provided that the following conditions are
-*  met:
-*
-*  * Redistributions of source code must retain the above copyright
-*  notice, this list of conditions and the following disclaimer.
-*  * Redistributions in binary form must reproduce the above
-*  copyright notice, this list of conditions and the following disclaimer
-*  in the documentation and/or other materials provided with the
-*  distribution.
-*
-*  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-*  "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-*  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-*  A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-*  OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-*  SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-*  LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-*  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-*  THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-*  (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-*  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-*
-*  You can contact the author at :
-*  - xxHash homepage: http://www.xxhash.com
-*  - xxHash source repository : https://github.com/Cyan4973/xxHash
+ *  xxHash - Fast Hash algorithm
+ *  Copyright (c) 2012-2020, Yann Collet, Facebook, Inc.
+ *
+ *  You can contact the author at :
+ *  - xxHash homepage: http://www.xxhash.com
+ *  - xxHash source repository : https://github.com/Cyan4973/xxHash
+ * 
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 */


@@ -115,7 +95,7 @@ static void* XXH_memcpy(void* dest, const void* src, size_t size) { return memcp
 /* *************************************
 *  Compiler Specific Options
 ***************************************/
-#if defined (__GNUC__) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L   /* C99 */
+#if (defined(__GNUC__) && !defined(__STRICT_ANSI__)) || defined(__cplusplus) || defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L   /* C99 */
 #  define INLINE_KEYWORD inline
 #else
 #  define INLINE_KEYWORD
@@ -729,7 +709,9 @@ FORCE_INLINE_TEMPLATE XXH_errorcode XXH64_update_endian (XXH64_state_t* state, c
    state->total_len += len;

    if (state->memsize + len < 32) {  /* fill in tmp buffer */
-        XXH_memcpy(((BYTE*)state->mem64) + state->memsize, input, len);
+        if (input != NULL) {
+            XXH_memcpy(((BYTE*)state->mem64) + state->memsize, input, len);
+        }
        state->memsize += (U32)len;
        return XXH_OK;
    }
@@ -1,35 +1,15 @@
 /*
-   xxHash - Extremely Fast Hash algorithm
-   Header File
-   Copyright (C) 2012-2016, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-   You can contact the author at :
-   - xxHash source repository : https://github.com/Cyan4973/xxHash
+ * xxHash - Extremely Fast Hash algorithm
+ * Header File
+ * Copyright (c) 2012-2020, Yann Collet, Facebook, Inc.
+ *
+ * You can contact the author at :
+ * - xxHash source repository : https://github.com/Cyan4973/xxHash
+ * 
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 */

 /* Notice extracted from xxHash homepage :
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -76,6 +76,7 @@ typedef enum {
  /* following error codes are __NOT STABLE__, they can be removed or changed in future versions */
  ZSTD_error_frameIndex_tooLarge = 100,
  ZSTD_error_seekableIO          = 102,
+  ZSTD_error_dstBuffer_wrong     = 104,
  ZSTD_error_maxCode = 120  /* never EVER use this value directly, it can change in future versions! Use ZSTD_isError() instead */
 } ZSTD_ErrorCode;

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-present, Yann Collet, Facebook, Inc.
+ * Copyright (c) 2016-2020, Yann Collet, Facebook, Inc.
 * All rights reserved.
 *
 * This source code is licensed under both the BSD-style license (found in the
@@ -19,12 +19,15 @@
 /*-*************************************
 *  Dependencies
 ***************************************/
+#ifdef __aarch64__
+#include <arm_neon.h>
+#endif
 #include "compiler.h"
 #include "mem.h"
 #include "debug.h"                 /* assert, DEBUGLOG, RAWLOG, g_debuglevel */
 #include "error_private.h"
 #define ZSTD_STATIC_LINKING_ONLY
-#include "zstd.h"
+#include "../zstd.h"
 #define FSE_STATIC_LINKING_ONLY
 #include "fse.h"
 #define HUF_STATIC_LINKING_ONLY
@@ -53,6 +56,31 @@ extern "C" {
 #define MIN(a,b) ((a)<(b) ? (a) : (b))
 #define MAX(a,b) ((a)>(b) ? (a) : (b))

+/**
+ * Ignore: this is an internal helper.
+ *
+ * This is a helper function to help force C99-correctness during compilation.
+ * Under strict compilation modes, variadic macro arguments can't be empty.
+ * However, variadic function arguments can be. Using a function therefore lets
+ * us statically check that at least one (string) argument was passed,
+ * independent of the compilation flags.
+ */
+static INLINE_KEYWORD UNUSED_ATTR
+void _force_has_format_string(const char *format, ...) {
+  (void)format;
+}
+
+/**
+ * Ignore: this is an internal helper.
+ *
+ * We want to force this function invocation to be syntactically correct, but
+ * we don't want to force runtime evaluation of its arguments.
+ */
+#define _FORCE_HAS_FORMAT_STRING(...) \
+  if (0) { \
+    _force_has_format_string(__VA_ARGS__); \
+  }
+
 /**
 * Return the specified error if the condition evaluates to true.
 *
@@ -62,7 +90,9 @@ extern "C" {
 */
 #define RETURN_ERROR_IF(cond, err, ...) \
  if (cond) { \
-    RAWLOG(3, "%s:%d: ERROR!: check %s failed, returning %s", __FILE__, __LINE__, ZSTD_QUOTE(cond), ZSTD_QUOTE(ERROR(err))); \
+    RAWLOG(3, "%s:%d: ERROR!: check %s failed, returning %s", \
+           __FILE__, __LINE__, ZSTD_QUOTE(cond), ZSTD_QUOTE(ERROR(err))); \
+    _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
    RAWLOG(3, ": " __VA_ARGS__); \
    RAWLOG(3, "\n"); \
    return ERROR(err); \
@@ -75,7 +105,9 @@ extern "C" {
 */
 #define RETURN_ERROR(err, ...) \
  do { \
-    RAWLOG(3, "%s:%d: ERROR!: unconditional check failed, returning %s", __FILE__, __LINE__, ZSTD_QUOTE(ERROR(err))); \
+    RAWLOG(3, "%s:%d: ERROR!: unconditional check failed, returning %s", \
+           __FILE__, __LINE__, ZSTD_QUOTE(ERROR(err))); \
+    _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
    RAWLOG(3, ": " __VA_ARGS__); \
    RAWLOG(3, "\n"); \
    return ERROR(err); \
@@ -90,7 +122,9 @@ extern "C" {
  do { \
    size_t const err_code = (err); \
    if (ERR_isError(err_code)) { \
-      RAWLOG(3, "%s:%d: ERROR!: forwarding error in %s: %s", __FILE__, __LINE__, ZSTD_QUOTE(err), ERR_getErrorName(err_code)); \
+      RAWLOG(3, "%s:%d: ERROR!: forwarding error in %s: %s", \
+             __FILE__, __LINE__, ZSTD_QUOTE(err), ERR_getErrorName(err_code)); \
+      _FORCE_HAS_FORMAT_STRING(__VA_ARGS__); \
      RAWLOG(3, ": " __VA_ARGS__); \
      RAWLOG(3, "\n"); \
      return err_code; \
@@ -193,10 +227,22 @@ static const U32 OF_defaultNormLog = OF_DEFAULTNORMLOG;
 /*-*******************************************
 *  Shared functions to include for inlining
 *********************************************/
-static void ZSTD_copy8(void* dst, const void* src) { memcpy(dst, src, 8); }
+static void ZSTD_copy8(void* dst, const void* src) {
+#ifdef __aarch64__
+    vst1_u8((uint8_t*)dst, vld1_u8((const uint8_t*)src));
+#else
+    memcpy(dst, src, 8);
+#endif
+}

 #define COPY8(d,s) { ZSTD_copy8(d,s); d+=8; s+=8; }
-static void ZSTD_copy16(void* dst, const void* src) { memcpy(dst, src, 16); }
+static void ZSTD_copy16(void* dst, const void* src) {
+#ifdef __aarch64__
+    vst1q_u8((uint8_t*)dst, vld1q_u8((const uint8_t*)src));
+#else
+    memcpy(dst, src, 16);
+#endif
+}
 #define COPY16(d,s) { ZSTD_copy16(d,s); d+=16; s+=16; }

 #define WILDCOPY_OVERLENGTH 32
@@ -215,7 +261,7 @@ typedef enum {
 *         - ZSTD_overlap_src_before_dst: The src and dst may overlap, but they MUST be at least 8 bytes apart.
 *           The src buffer must be before the dst buffer.
 */
-MEM_STATIC FORCE_INLINE_ATTR DONT_VECTORIZE
+MEM_STATIC FORCE_INLINE_ATTR 
 void ZSTD_wildcopy(void* dst, const void* src, ptrdiff_t length, ZSTD_overlap_e const ovtype)
 {
    ptrdiff_t diff = (BYTE*)dst - (const BYTE*)src;
@@ -232,13 +278,18 @@ void ZSTD_wildcopy(void* dst, const void* src, ptrdiff_t length, ZSTD_overlap_e
        } while (op < oend);
    } else {
        assert(diff >= WILDCOPY_VECLEN || diff <= -WILDCOPY_VECLEN);
-        /* Separate out the first two COPY16() calls because the copy length is
+        /* Separate out the first COPY16() call because the copy length is
         * almost certain to be short, so the branches have different
-         * probabilities.
-         * On gcc-9 unrolling once is +1.6%, twice is +2%, thrice is +1.8%.
-         * On clang-8 unrolling once is +1.4%, twice is +3.3%, thrice is +3%.
+         * probabilities. Since it is almost certain to be short, only do
+         * one COPY16() in the first call. Then, do two calls per loop since
+         * at that point it is more likely to have a high trip count.
         */
-        COPY16(op, ip);
+#ifndef __aarch64__
+        do {
+            COPY16(op, ip);
+        }
+        while (op < oend);
+#else
        COPY16(op, ip);
        if (op >= oend) return;
        do {
@@ -246,9 +297,29 @@ void ZSTD_wildcopy(void* dst, const void* src, ptrdiff_t length, ZSTD_overlap_e
            COPY16(op, ip);
        }
        while (op < oend);
+#endif
    }
 }

+MEM_STATIC size_t ZSTD_limitCopy(void* dst, size_t dstCapacity, const void* src, size_t srcSize)
+{
+    size_t const length = MIN(dstCapacity, srcSize);
+    if (length > 0) {
+        memcpy(dst, src, length);
+    }
+    return length;
+}
+
+/* define "workspace is too large" as this number of times larger than needed */
+#define ZSTD_WORKSPACETOOLARGE_FACTOR 3
+
+/* when workspace is continuously too large
+ * during at least this number of times,
+ * context's memory usage is considered wasteful,
+ * because it's sized to handle a worst case scenario which rarely happens.
+ * In which case, resize it down to free some memory */
+#define ZSTD_WORKSPACETOOLARGE_MAXDURATION 128
+

 /*-*******************************************
 *  Private declarations
@@ -273,6 +344,31 @@ typedef struct {
    U32   longLengthPos;
 } seqStore_t;

+typedef struct {
+    U32 litLength;
+    U32 matchLength;
+} ZSTD_sequenceLength;
+
+/**
+ * Returns the ZSTD_sequenceLength for the given sequences. It handles the decoding of long sequences
+ * indicated by longLengthPos and longLengthID, and adds MINMATCH back to matchLength.
+ */
+MEM_STATIC ZSTD_sequenceLength ZSTD_getSequenceLength(seqStore_t const* seqStore, seqDef const* seq)
+{
+    ZSTD_sequenceLength seqLen;
+    seqLen.litLength = seq->litLength;
+    seqLen.matchLength = seq->matchLength + MINMATCH;
+    if (seqStore->longLengthPos == (U32)(seq - seqStore->sequencesStart)) {
+        if (seqStore->longLengthID == 1) {
+            seqLen.litLength += 0xFFFF;
+        }
+        if (seqStore->longLengthID == 2) {
+            seqLen.matchLength += 0xFFFF;
+        }
+    }
+    return seqLen;
+}
+
 /**
 * Contains the compressed frame size and an upper-bound for the decompressed frame size.
 * Note: before using `compressedSize`, check for errors using ZSTD_isError().
@@ -299,8 +395,7 @@ MEM_STATIC U32 ZSTD_highbit32(U32 val)   /* compress, dictBuilder, decodeCorpus
    {
 #   if defined(_MSC_VER)   /* Visual */
        unsigned long r=0;
-        _BitScanReverse(&r, val);
-        return (unsigned)r;
+        return _BitScanReverse(&r, val) ? (unsigned)r : 0;
 #   elif defined(__GNUC__) && (__GNUC__ >= 3)   /* GCC Intrinsic */
        return __builtin_clz (val) ^ 31;
 #   elif defined(__ICCARM__)    /* IAR Intrinsic */
@@ -1,35 +1,15 @@
 /* ******************************************************************
-   FSE : Finite State Entropy encoder
-   Copyright (C) 2013-present, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-    You can contact the author at :
-    - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
-    - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ * FSE : Finite State Entropy encoder
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ *  You can contact the author at :
+ *  - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *  - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */

 /* **************************************************************
@@ -37,14 +17,14 @@
 ****************************************************************/
 #include <stdlib.h>     /* malloc, free, qsort */
 #include <string.h>     /* memcpy, memset */
-#include "compiler.h"
-#include "mem.h"        /* U32, U16, etc. */
-#include "debug.h"      /* assert, DEBUGLOG */
+#include "../common/compiler.h"
+#include "../common/mem.h"        /* U32, U16, etc. */
+#include "../common/debug.h"      /* assert, DEBUGLOG */
 #include "hist.h"       /* HIST_count_wksp */
-#include "bitstream.h"
+#include "../common/bitstream.h"
 #define FSE_STATIC_LINKING_ONLY
-#include "fse.h"
-#include "error_private.h"
+#include "../common/fse.h"
+#include "../common/error_private.h"


 /* **************************************************************
@@ -645,9 +625,6 @@ size_t FSE_compress_usingCTable (void* dst, size_t dstSize,

 size_t FSE_compressBound(size_t size) { return FSE_COMPRESSBOUND(size); }

-#define CHECK_V_F(e, f) size_t const e = f; if (ERR_isError(e)) return e
-#define CHECK_F(f)   { CHECK_V_F(_var_err__, f); }
-
 /* FSE_compress_wksp() :
 * Same as FSE_compress2(), but using an externally allocated scratch buffer (`workSpace`).
 * `wkspSize` size must be `(1<<tableLog)`.
@@ -1,42 +1,22 @@
 /* ******************************************************************
-   hist : Histogram functions
-   part of Finite State Entropy project
-   Copyright (C) 2013-present, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-    You can contact the author at :
-    - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
-    - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ * hist : Histogram functions
+ * part of Finite State Entropy project
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ *  You can contact the author at :
+ *  - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *  - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */

 /* --- dependencies --- */
-#include "mem.h"             /* U32, BYTE, etc. */
-#include "debug.h"           /* assert, DEBUGLOG */
-#include "error_private.h"   /* ERROR */
+#include "../common/mem.h"             /* U32, BYTE, etc. */
+#include "../common/debug.h"           /* assert, DEBUGLOG */
+#include "../common/error_private.h"   /* ERROR */
 #include "hist.h"


@@ -1,36 +1,16 @@
 /* ******************************************************************
-   hist : Histogram functions
-   part of Finite State Entropy project
-   Copyright (C) 2013-present, Yann Collet.
-
-   BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
-   Redistribution and use in source and binary forms, with or without
-   modification, are permitted provided that the following conditions are
-   met:
-
-       * Redistributions of source code must retain the above copyright
-   notice, this list of conditions and the following disclaimer.
-       * Redistributions in binary form must reproduce the above
-   copyright notice, this list of conditions and the following disclaimer
-   in the documentation and/or other materials provided with the
-   distribution.
-
-   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-    You can contact the author at :
-    - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
-    - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ * hist : Histogram functions
+ * part of Finite State Entropy project
+ * Copyright (c) 2013-2020, Yann Collet, Facebook, Inc.
+ *
+ *  You can contact the author at :
+ *  - FSE source repository : https://github.com/Cyan4973/FiniteStateEntropy
+ *  - Public forum : https://groups.google.com/forum/#!forum/lz4c
+ *
+ * This source code is licensed under both the BSD-style license (found in the
+ * LICENSE file in the root directory of this source tree) and the GPLv2 (found
+ * in the COPYING file in the root directory of this source tree).
+ * You may select, at your option, one of the above-listed licenses.
 ****************************************************************** */

 /* --- dependencies --- */
--- a/Show More
+++ b/Show More
				`@@ -0,0 +1 @@`
				`include("${CMAKE_CURRENT_LIST_DIR}/zstdTargets.cmake")`