Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid magic number when decompressing. Headers not added when compressing #208

Open
anand-dotworld opened this issue Jul 7, 2023 · 0 comments

Comments

@anand-dotworld
Copy link

anand-dotworld commented Jul 7, 2023

I have an gin api in which i compress a string using lz4 frame compression and send the result in the response to a frontend. In the javascript I fetch that api and decompress the data using lz4 package by pierrec, at that time I am getting Invalid magic number error. Can anyone help me with this?

Backend
`package main

import (
"bytes"
"fmt"
"io"

"github.com/pierrec/lz4"

"github.com/gin-gonic/gin"

)

var fileContent = CompressBlock compresses the source buffer starting at soffet into the destination one. This is the fast version of LZ4 compression and also the default one. The size of the compressed data is returned. If it is 0 and no error, then the data is incompressible. An error is returned if the destination buffer is too small.

func main() {
router := gin.Default()

// Define the API endpoint
router.POST("/api/endpoint", func(c *gin.Context) {
	toCompress := []byte(fileContent)
	// compressed := make([]byte, len(toCompress))

	result, err := compress(toCompress)
	if err != nil {
		fmt.Println(err)
	}

	c.JSON(200, gin.H{"result": result})
})

// Start the server
router.Run(":8000")

}

func compress(in []byte) ([]byte, error) {
r := bytes.NewReader(in)
w := &bytes.Buffer{}
zw := lz4.NewWriter(w)
_, err := io.Copy(zw, r)
if err != nil {
return nil, err
}
// Closing is very important
if err := zw.Close(); err != nil {
return nil, err
}
return w.Bytes(), nil
}
`

Frontend
`
import fetch from "node-fetch";
import LZ4 from "lz4";

const apiUrl = "http://localhost:8000/api/endpoint";

const postData = async () => {
try {
const response = await fetch(apiUrl, {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({}),
});

if (!response.ok) {
  throw new Error("Request failed with status " + response.status);
}
const result = await response.json();
// const decoded = base64ToArrayBuffer(result.result);
const buffer = Buffer.from(result.result);
const decompressed = LZ4.decode(buffer);
const decodedBuffer = Buffer.from(decompressed);
console.log(decodedBuffer.toString());

} catch (error) {
console.error(error);
}
};

postData();
`

Error
Error: Invalid magic number: 4D466941 @0 at Decoder.emit_Error (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder_stream.js:64:22) at Decoder.read_MagicNumber (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder_stream.js:93:8) at Decoder._main (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder_stream.js:289:25) at Decoder._transform (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder_stream.js:60:7) at Decoder.Transform._write (node:internal/streams/transform:184:23) at writeOrBuffer (node:internal/streams/writable:389:12) at _write (node:internal/streams/writable:330:10) at Decoder.Writable.end (node:internal/streams/writable:609:17) at Object.LZ4_uncompress [as decode] (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder.js:14:10) at postData (file:///home/anand/go/src/github.com/anand-dotworld/test/client/index.js:24:30)

Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant