Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lzip support #401

Merged
merged 2 commits into from Apr 8, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/macos-latest.yml
Expand Up @@ -8,7 +8,7 @@ jobs:

strategy:
matrix:
go-version: [1.21]
go-version: [1.22]
runs-on: macos-latest
steps:
- name: Install Go
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ubuntu-latest.yml
Expand Up @@ -8,7 +8,7 @@ jobs:

strategy:
matrix:
go-version: [1.21]
go-version: [1.22]
runs-on: ubuntu-latest
steps:
- name: Install Go
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/windows-latest.yml
Expand Up @@ -8,7 +8,7 @@ jobs:

strategy:
matrix:
go-version: [1.21]
go-version: [1.22]
runs-on: windows-latest
steps:
- name: Install Go
Expand Down
1 change: 1 addition & 0 deletions README.md
Expand Up @@ -36,6 +36,7 @@ Introducing **Archiver 4.0** - a cross-platform, multi-format archive utility an
- flate (.zip)
- gzip (.gz)
- lz4 (.lz4)
- lzip (.lz)
- snappy (.sz)
- xz (.xz)
- zlib (.zz)
Expand Down
7 changes: 7 additions & 0 deletions formats_test.go
Expand Up @@ -315,6 +315,13 @@ func TestIdentifyFindFormatByStreamContent(t *testing.T) {
compressorName: ".lz4",
wantFormatName: ".lz4",
},
{
name: "should recognize lz",
openCompressionWriter: Lzip{}.OpenWriter,
content: []byte("this is text"),
compressorName: ".lz",
wantFormatName: ".lz",
},
{
name: "should recognize sz",
openCompressionWriter: Sz{}.OpenWriter,
Expand Down
7 changes: 4 additions & 3 deletions go.mod
@@ -1,8 +1,8 @@
module github.com/mholt/archiver/v4

go 1.21.3
go 1.22

toolchain go1.22.0
toolchain go1.22.2

require (
github.com/andybalholm/brotli v1.0.5
Expand All @@ -11,14 +11,15 @@ require (
github.com/klauspost/pgzip v1.2.6
github.com/nwaples/rardecode/v2 v2.0.0-beta.2
github.com/therootcompany/xz v1.0.1
github.com/ulikunitz/xz v0.5.11
github.com/ulikunitz/xz v0.5.12
)

require (
github.com/STARRY-S/zip v0.1.0
github.com/bodgit/sevenzip v1.4.3
github.com/golang/snappy v0.0.4
github.com/pierrec/lz4/v4 v4.1.18
github.com/sorairolake/lzip-go v0.3.1
golang.org/x/text v0.13.0
)

Expand Down
6 changes: 4 additions & 2 deletions go.sum
Expand Up @@ -98,6 +98,8 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd/go.mod h1:hPqNNc0+uJM6H+SuU8sEs5K5IQeKccPqeSjfgcKGgPk=
github.com/sorairolake/lzip-go v0.3.1 h1:/v5NxPwhyEV/0NxdniSOPt0zTTZweQfIr2d/f9cE0Uk=
github.com/sorairolake/lzip-go v0.3.1/go.mod h1:sGvZv/ZFQzR0DSbXsjCyA6Nmb84TIPT8QbmhFBfVRlI=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
Expand All @@ -110,8 +112,8 @@ github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXl
github.com/therootcompany/xz v1.0.1 h1:CmOtsn1CbtmyYiusbfmhmkpAAETj0wBIH6kCYaX+xzw=
github.com/therootcompany/xz v1.0.1/go.mod h1:3K3UH1yCKgBneZYhuQUvJ9HPD19UEXEI0BWbMn8qNMY=
github.com/ulikunitz/xz v0.5.6/go.mod h1:2bypXElzHzzJZwzH67Y6wb67pO62Rzfn7BSiF4ABRW8=
github.com/ulikunitz/xz v0.5.11 h1:kpFauv27b6ynzBNT/Xy+1k+fK4WswhN/6PN5WhFAGw8=
github.com/ulikunitz/xz v0.5.11/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/ulikunitz/xz v0.5.12 h1:37Nm15o69RwBkXM0J6A5OlE67RZTfzUxTj8fB3dfcsc=
github.com/ulikunitz/xz v0.5.12/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
Expand Down
52 changes: 52 additions & 0 deletions lzip.go
@@ -0,0 +1,52 @@
package archiver

import (
"bytes"
"io"
"strings"

"github.com/sorairolake/lzip-go"
)

func init() {
RegisterFormat(Lzip{})
}

// Lzip facilitates lzip compression.
type Lzip struct{}

func (Lzip) Name() string { return ".lz" }

func (lz Lzip) Match(filename string, stream io.Reader) (MatchResult, error) {
var mr MatchResult

// match filename
if strings.Contains(strings.ToLower(filename), lz.Name()) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, indeed -- make this comparison more strict and that should hopefully fix the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be fixed with the following changes:

Suggested change
if strings.Contains(strings.ToLower(filename), lz.Name()) {
if filepath.Ext(strings.ToLower(filename)) == lz.Name() {

However, this requires that the extension be the last element of the path.

Copy link
Owner

@mholt mholt Apr 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would fail for those cases as you say, but maybe a check for both a suffix of .lz or the string contains .lz.?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is almost no possibility that .lz and .lz4 exists anywhere other than at the end of the path.

I think /path/to/foo.tar.lz or bar.7z.lz4 are possible, but /path/to/foo.lz.tar or bar.lz4.7z are almost impossible.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point.

mr.ByName = true
}

// match file header
buf, err := readAtMost(stream, len(lzipHeader))
if err != nil {
return mr, err
}
mr.ByStream = bytes.Equal(buf, lzipHeader)

return mr, nil
}

func (Lzip) OpenWriter(w io.Writer) (io.WriteCloser, error) {
return lzip.NewWriter(w), nil
}

func (Lzip) OpenReader(r io.Reader) (io.ReadCloser, error) {
lzr, err := lzip.NewReader(r)
if err != nil {
return nil, err
}
return io.NopCloser(lzr), err
}

// magic number at the beginning of lzip files
// https://datatracker.ietf.org/doc/html/draft-diaz-lzip-09#section-2
var lzipHeader = []byte("LZIP")