Policy on formats with paywalled specifications

[fintelia]

Most existing image formats have extensive publicly available technical documentation or specifications to guide the implementation of encoders and decoders. PNG, TIFF, WebP, GIF, QOI, among others have complete specifications available for anyone to access. AVIF's spec references a few non-free bits but the bulk is accessible. A few other common formats lack public (or perhaps any) specs but have been sufficiently reverse engineered that they can be independently re-implemented. Without this culture of openness the image-rs project would not have been possible.

In contrast, the specification for JPEG XL is locked behind a paywall. There is an open source reference implementation, but it weighs in at 140,000+ lines of multithreaded C++ making it of questionable value for anyone hoping to learn the details of the format. Especially considering that sparse public documentation that does exist indicates that JPEG XL is an incredibly complex format, meaning that any reverse engineering effort would surely be a slow and painful process.

PR #1945 seeks to add JPEG XL decoding to image-rs using the jxl-oxide crate, which means at some point we're going to have to make a decision on whether to support it. My personal view is that I'd rather not spend any of my time helping spread non-free image file formats when there's already so many open formats in widespread use. But I'd imagine some users may feel differently (especially JXL's "enthusiastic" supporters who hope it will one day replace all other image formats...)

[tirr-c]

JPEG 1 (the original one) specification is also paywalled. But writing encoder and decoder, and producing/consuming JPEG images are still totally free, it's only the official specification document that's not free (ISO claims copyright of their spec documents for some reason). I used the community draft version of JPEG XL specification when I wrote jxl-oxide.

Also I think it's totally okay to write and publish detailed technical documentation from the spec, and I feel writing explanation of JPEG XL is doable (it's much simpler than AV1 decoder specification).

[Quackdoc]

I believe this should be determined by the implementations and not the spec cost. and here is why;

if we are judging by documentation alone, I would consider libjxl's code base to be a far better then avif in terms of making an open decoder, this is because to create a sufficient avif decoder, you need an av1 decoder at least capable of decoding keyframes if you want stills, and pretty much a full av1 decoder if you want animations. the av1 spec is a 600 page behemoth that is incredibly obtuse, rav1d the most promising decoder after dav1d, simply elected to transpile dav1d instead of starting from scratch due to the sheer scale of the av1 spec. there is no separating AVIF from AV1, as AVIF is more or less a modified mp4 container around an AV1 bitstream.

If you wanted to judge effort, I would argue and this is evidenced in reality due to the current state of things, creating a jpegxl decoder based on the documentation available, being a community draft as well as libjxl, is significantly easier then creating an an av1 decoder which is necessary for av1 all three active decoders have significant resources being google behind libgav1, and dav1d being the go-to for everything and rav1d which is worked on by promisso sponsored by AWS. Libgav1 is more or less on lifesupport as far as work being done. Dav1d is the industry supported encoder, and as said rav1d was simply transpiled/forked from dav1d's code base. so basically 2 decoders, or 1.5 if you consider that rav1d was forked project.

In contrast, JXL has 3 open decoders aside from libjxl which are functional to varying degrees to which you can see here. I won't pretend to know the resources or backing to these three jxl decoders. however I would consider the fact that there are three open decoders that are relatively independent on libjxl a great testament to JXL. and proof that spec cost should not be an indication of openness. Especially when implementing JXL is both free for the user, and free for the implementer.

I suppose we would need to define the "culture of openess". IMO JXL has that culture, with 4 open source decoders capable of decoding JXL images to at least some degree I would say JXL meets that bar.

[fintelia]

I'm sorry, but the cost of the spec is the entire point. I got involved in image-rs by noticing that one of my tiff files didn't decode properly, tracking down what was wrong with the relevant crate (part of which involved a very careful reading of the spec!), and creating a patch. Users of jxl-oxide have no similar opportunity, unless they decide up front to spend $263 for a copy of the spec or get it through backchannels.

To give another example, I was able to write a very, very simple lossless webp encoder just by reading the webp specification and looking for the minimal subset needed to produce a valid file. I'm not sure what would be needed to do the same for JPEG XL, but I strongly suspect it would involve spelunking through the code of at least one existing implementation.

I want to be excited about JPEG XL. I like image formats and from a technical perspective JXL is a very cool format. I'm really impressed by its lossless compression ratio and appreciate that its higher decompression cost is offset by designing in parallel decoding from the start. But for a format that (among other things) seeks to replace PNG -- perhaps the poster child for an open format -- having the spec behind a paywall feels like a step backwards.

Perhaps the best outcome is someone coming along and writing comprehensive human readable public documentation of the format. For all I know, it might just be a matter of writing an errata for the August 2019 committee draft. But I'm not holding by breath. The original authors obviously didn't think it was important enough to refuse signing over their copyright to ISO, nor has anyone else stepped up over the years.

[lonjil]

Refusing to sign over the copyright would've done nothing, JXL would've simply not existed instead. I have in fact seen them talk about creating a freely available description of the format. Having read the spec myself, it would be a bit of work, but most of the important bits are purely factual, in the form of tables and such, which are not protected by copyright.

Keep in mind, regarding "over the years", that JXL is still very young, 3rd party implementations have only existed for a bit more than a year, and the original authors are still not done with the 1.0 release of libjxl. No one involved with JXL likes closed specs, so this will certainly be dealt with sooner or later.

[Quackdoc]

@fintelia can we get an update on this? some of the stuff is becoming available in jxl-oxide now that is needed for inclusion (IE. limits) so I would like to try and budget some time to work on the PR again, but if it's not going to be accepted, It's not something I am wanting to invest too much time and effort into.

[fintelia]

Please hold off on the PR. It sounds like there's going to be a freely available spec for JPEG XL so we can wait for that. In the mean time, anyone who needs it can directly use the jxl-oxide crate

[Fraetor]

One thing to note is that because of the ISOs unique relationship with national standards body they can usually be obtained without cost through University libraries and other institutions.

[damian101]

JPEG XL is an incredibly complex format

It is quite complex, but doesn't come close to the also mentioned AVIF in complexity...

[kornelski]

This is an annoying problem. They're technically free to use and free to implement, except getting hands on the actual spec.

ISO-paywalled formats like JXL and HEIF/MIAF (guts of AVIF) have some information available publicly, either as pre-ISO drafts, or substantial quotes, or guides written by other implementors, or just googlable leaked copies with questionable copyright status.

However, when implementing AVIF muxing, I've learned the hard way that such information can't be trusted, because ISO standards make backwards-incompatible changes! I wrote code based on an older version of the standard, and ended up making files that up-to-date implementations rejected.

[Fraetor]

Before they are standardised there are certainly no promises, but for second editions and further drafts should be pretty good, as they are mostly clarifications. I'll try and find out what the status of the committee draft of the JPEG XL second edition is.

Edit: It seems that second edition is a no-go from a copyright point of view. Maybe because it is mostly the same as the first edition.

[lonjil]

Edit: It seems that second edition is a no-go from a copyright point of view. Maybe because it is mostly the same as the first edition.

No, it's because ISO hates your guts. They're also trying to clamp down on things that have traditionally had close to final public drafts, like C.