Skip to content

Latest commit

 

History

History
125 lines (92 loc) · 7.47 KB

README.md

File metadata and controls

125 lines (92 loc) · 7.47 KB
Raw

LZW streaming compressor/decompressor

Github Actions Codecov
Github Actions test status Codecov

LZW streaming compressor/decompressor based on LZW AB compatible with UNIX compress.

Compressor dictionaries

  • Linked list (idea from LZW AB). It has low memory usage <= 327 KB (16 bit codes). It is slow in general, recommended for small amount of data.
  • Sparse array (enabled by default). It has high memory usage <= 33.5 MB (16 bit codes). It is fast, recommended for large amount of data.

You can add your own implementation, see api for dictionary.

Why?

  • You can compress and decompress .Z (tar.Z) archives.
  • You can resurrect legacy apis (like Content-Encoding: compress in HTTP 1.1).
  • You can support ancient software and make your application looks like ancient software.

New features

  • Accurate ratio calculation without overhead, compressor provides smaller archive (compatible with UNIX compress).
  • Unaligned bit groups switch (only disabled mode is compatible with UNIX compress).
  • LSB/MSB switch (only LSB mode is compatible with UNIX compress).
  • Magic header switch (only enabled magic header is compatible with UNIX compress).

Dependencies

Operating systems: GNU/Linux, OSX, FreeBSD, Windows (Visual Studio, MinGW, Cygwin).

Popular OS Runtime dependencies Compilation dependencies
Ubuntu libgmp10/libtommath1 libgmp-dev/libtommath-dev
cmake, gcc/clang, asciidoc, docbook-xml
CentOS gmp/libtommath gmp-devel/libtommath-devel
cmake3, gcc/clang, asciidoc, docbook-dtds
ArchLinux gmp/libtommath gmp/libtommath
cmake, gcc/clang, asciidoc, docbook-xml
OSX gmp/libtommath gmp/libtommath
cmake, gcc/llvm, asciidoc
Windows libtommath msvc, windows sdk, ucrt, libtommath
cmake, curl, git, python, asciidoc, docbook-xml
xsltproc, docbook-bundle, zip, nsis

Installation

You can download and install prebuilt packages on releases page.

You can use scripts/release.sh, it will create package for your system from source. You can also use overlay for gentoo.

Installation for Windows (Visual Studio) from source is not so trivial:

  1. Install Visual Studio with components: msvc, windows sdk, ucrt.
  2. Install chocolatey with components: cmake, curl, git, python, xsltproc, docbook-bundle, zip, nsis.
  3. Install asciidoc.
  4. Download docbook-xml-4.5.zip and unpack it to C:\ProgramData\chocolatey\lib\docbook-bundle\docbook-4.5.
  5. Edit C:\ProgramData\chocolatey\lib\docbook-bundle\catalog.xml and append <nextCatalog catalog="./docbook-4.5/catalog.xml"/>.
  6. Launch x64 Native Tools Command Promp as admin.
git clone "https://github.com/libtom/libtommath.git" --depth 1 && cd "libtommath"
nmake -f "makefile.msvc" install && cd ".."
set "XML_CATALOG_FILES=C:\ProgramData\chocolatey\lib\docbook-bundle\catalog.xml"
set "INCLUDE=C:\devel\include;%INCLUDE%" && set "LIB=C:\devel\lib;%LIB%"
set "PATH=C:\devel\bin;C:\Program Files\Git\bin;C:\Program Files\CMake\bin;%PATH%"
cd build && bash "..\scripts\release.sh"

Please use the following command for OSX.

export XML_CATALOG_FILES=/usr/local/etc/xml/catalog

Quick start

cd build
cmake -DLZWS_COMPRESSOR_DICTIONARY="sparse-array" ".." && cmake --build "."
echo -n "TOBEORNOTTOBEORTOBEORNOT" | ./src/cli/lzws | ./src/cli/lzws -d
Option Values Default Description
LZWS_COMPRESSOR_DICTIONARY sparse-array, linked-list sparse-array dictionary implementation
LZWS_BIGNUM_LIBRARY gmp, tommath, "" (auto) "" (auto) bignum library
LZWS_SHARED ON, OFF ON build shared binaries
LZWS_STATIC ON, OFF OFF build static binaries
LZWS_CLI ON, OFF ON build cli
LZWS_TESTS ON, OFF ON build tests
LZWS_EXAMPLES ON, OFF OFF build examples
LZWS_MAN ON, OFF OFF build man
LZWS_ARGTABLE3_FALLBACK github, github-archive github-archive argtable3 fallback
bitbucket, bitbucket-archive
gitlab, gitlab-archive
LZWS_COVERAGE ON, OFF OFF enable coverage

Examples and docs

See examples, man and doc folders.

CI

Please visit scripts/test-images. See test script scripts/toolchains.sh and test script scripts/coverage_toolchains.sh for CI.

Real world testing

See lzws archive collector.

License

Copyright (c) 2016 David Bryant, 2018+ other authors, all rights reserved (see AUTHORS). Distributed under the BSD Software License (see LICENSE).

Notice

Releases with GMP provided without static linked binaries (GMP LGPL license limitation).