what's the raw data format?

[Pomax]

the example suggests the raw data has the same format as the HTML canvas, namely one [r,b,g,a] cluster per pixel. Is that true? (if so, can the README be amended with that information?)

[ReDreamport]

Could the encode support [RGB] format?

[Pomax]

that would mostly confuse matters, given that canvas encodes pixels as [RGBA]. Also to answer my own question, it does indeed follow the same ordering.

[ReDreamport]

My raw image data come from USB camera directly(via v4l2). The output is RGB format.

There's a similar module node-png.

var png = new Png(buffer, width, height, buffer_type, bits_per_pixel);

The fourth argument is 'rgb', 'bgr', 'rgba', 'bgra', or 'gray'. Defaults to 'rgb'.

Could you kindly provide with a similar API, that supports flexible format? Thanks!

[Pomax]

Ah, raw data, rather than an actual image.

[benwiley4000]

Sorry for the quite late response, but thanks for flagging this @Pomax - that's a great question whose answer should be specified more clearly in the readme.

You're correct about the pixel data format ([r,g,b,a]). Further, it's worth noting that the full data output is row-first, left-to-right. So if we have a 2x2 jpeg which looks like this:

r = 255,0,0,255
g = 0,255,0,255
b = 0,0,255,255
B = 255,255,255,255

r	g
b	B

Then the decoded data output will look like:

[255, 0, 0, 255, 0, 255, 0, 255, 0, 0, 255, 255, 255, 255, 255, 255]

I'm not sure what the best format for explaining this is. I'd welcome a pull request.

[benwiley4000]

@ReDreamport that's not a bad idea - I'd also take a pull request for specifying an optional bufferType property in the rawImageData hash sent to jpeg.encode. 🙂

e.g.

jpeg.encode({
  data: frameData,
  width: width,
  height: height,
  bufferType: 'rgb' // default: 'rgba'
});

[patrickhulce]

It'd also be nice to support decoding to the original colorspace as well instead of forcing to RGBA.

[benwiley4000]

@patrickhulce do you have an API in mind? I would assume that by default we should have a predictable colorspace, though I agree the option would be nice. ETA: Saw your comment in #30 about the changes to the pdf.js version, hm...

[s1155030742]

I am about to implement croping to the Decoded data.
Therefore, do any know the raw data order?
For example, If I have a width = 2, Height = 3 jpeg File.Then I used jpeg.decode(file).
So, then result is x1y1,x2y1,x1y2,x2y2,x1y3,x2y3 or x1y1,x1y2,x1y3,x2y1,x2y2,x2y3?

[Pomax]

I would point out that a request to support colorspaces is definitely out of scope for this issue, and super deserves its own issue instead.

[Pomax]

@benwiley4000 in terms of docs, I'd add a section before the "License" section, after the example usage, reserved for explaining implementation details:

---start---

The finer points

Pixel encoding

Image pixel ordering follows the Canvas2D standard, with each pixel is encoded as the four channels [r,g,b,a], using one byte for each channel, with the pixels ordered as a flat Uint8ClampedArray, with rows of pixel data placed end to end, starting at the top row in the image.

---end---

And then probably with some linkouts to the canvas2d api and the imagedata data structure.

[HRK44]

Do we have any update on this?

I don't see why I should input a RGBA image to encode it in JPEG ?

[patrickhulce]

@HRK44 I'm confused, what did you think you would input? It's obviously got to be in some colorspace 😆

Is it the presence of the alpha channel that's confusing?

[HRK44]

@patrickhulce I have a RGB raw data image that I want to encode into JPEG, but since it looks like it's waiting for a RGBA input, the jpeg created is not correct.

[Pomax]

So just remap the data? You know the alpha channel is 255 for every pixel, so if you have a typed array for your data that uses [R,G,B] ordering, derive a new array that's 4*arr.length/3 long (and really: make whatever generates that raw data do that for you) and set [R,G,B,255] for every [R.G.B] triplet in the original array.

[HRK44]

@Pomax I wanted to avoid going through the whole raw data to modify it since I have millions of images - think about scaling. The point of taking multiple kind of inputs is to speed up the process by knowing what kind of data is coming, and since we know (especially in this case with JPEG) that it doesn't care about the Alpha channel, I find it strange to have it in the pre-requisite of the input data.

[patrickhulce]

@HRK44 if that's your only concern then I can certainly tell you that you won't be getting anything more out of us. The only change we would likely make is converting different inputs into the same colorspace and since it's already written respecting the Canvas2D spec, then we're not going to be re-writing the whole library to focus on RGB.

(also if you're dealing with millions of images I'd strongly advise against using this library. It is very slow and only ideal for cases where native encoding/decoding is not an option. I'd suggest using sharp instead)

[Pomax]

Reading through https://github.com/eugeneware/jpeg-js/blob/master/lib/encoder.js#L626-L674 it looks like the alpha channel gets ignored during the encode step, so you might be able to adapt that code in a way that computes pos based on 24 rather than 32 bit color tuples, given an encoding option as suggested by Ben in #3 (comment)

[HRK44]

@Pomax @patrickhulce Thanks for the advice, I'm not an expert in image processing that's why it seemed weird as a first approach.

I was also comparing with other libraries that takes those params like fast-png.

I'll check out sharp :)

Edit: @Pomax yes that's why I was asking if there is any update on this (based on Ben's comment), since I'm not familiar with image processing I'm not able to provide a good PR for this.