fix(gatsby-plugin-sharp): Pass failOn to all pipelines

[LekoArts]

Description

#37165 tried to fix an error that really should be fixed by correctly passing failOn to all our sharp pipelines.

Related Issues

#37151

[ch59059]

[LekoArts]

we missed this 🙈

[Jeff-Tian]

Hey, you closed my PR #37165 and copied most code changes into this PR, but you lost the test case, which is terrible.

[LekoArts]

@Jeff-Tian can you please generate a smaller file? I don't want to add 4MB to this repository just for this one test case. Ideal would be below 500KB

[Jeff-Tian]

Yes, there are rooms to improve for #37165, but you can comment and I can change it there, instead of closing it directly.

[LekoArts]

If it's about co-authorship for you I'm happy to add you to the commit, I just closed it because I wanted to do you a favor and fix it in a timely manner as your PR was incorrect. Going back and forth in a PR takes way more time than talking about it in advance (e.g. in a discussion) and then have a clear plan for a PR from the get go. If you give me a smaller test image I'll add it as a test case.

Sorry if that made you feel unappreciated, that wasn't my intention.

[Jeff-Tian]

@LekoArts Thanks, feel much better now. However, I can only trigger the meta data error with the 4MB picture. If I compressed it then the meta data error won't occur.

[LekoArts]

After talking to the author of sharp in lovell/sharp#3480 I also updated the wording on the failOn settting and the correct expectations of it. The test image you had in your PR will still fail because the image headers are corrupted. We still want to fail on this. I don't have a test file at hand that has correct headers but wrong pixel values, so we'll just assume that this is covered by sharp itself as a test case (which it is).

[Jeff-Tian]

Hi @LekoArts, I managed to create a very small png file that can trigger the metadata error. See #37203.

You mentioned that my PR #37165 was not correctly fixed the issue, but in contrast, this PR can NOT fix the issue unless includes the #37203 in.

Although the failOn option passes to sharp, it doesn't pass to reportError.

We might think that setting failOn to none will make sharp not throwing. However, by adding a test case, I found it still throws.

So I still insist passing the pluginOptions down to reportError. Does it make sense to you?

[Jeff-Tian]

Great, passing failOn down to sharp makes sense.

How about also passing the pluginOptions down to reportErrors so we can control the behavior of reportError (not to exit the whole process)?

[LekoArts]

If sharp thinks an image is invalid and an error should be thrown, gatsby-plugin-sharp should behave the same. Otherwise we have different behaviors between the two and it won't be clear in the future why an error is thrown or not. That's why I don't want to add the change to the reportError you added. So yes, it won't fix your issue, you'll need to use non-corrupt images.

Allowing the option you made in the other PR is a footgun for people and would allow them to ship missing images to the production site which we don't want.

[Jeff-Tian]

It's not the case. See #37165's test image, it's NOT a corrupt image. It's a normal image and there are many similar images widely used.

There are cases that a gatsby site contains many images, and only a few normal images happen to trigger the sharp throw an exception. In this scenario, we the site builders should have a way to control the whole building process.

And the pluginOptions is the perfect way to control, isn't it?

If you want the whole process to exit when sharp throws you can still set failOn to other values.

[LekoArts]

Please open an issue on https://github.com/lovell/sharp/issues about that specific image then, sharp thinks it's corrupt so we'll handle it that way.

[Jeff-Tian]

If it's about co-authorship for you I'm happy to add you to the commit, I just closed it because I wanted to do you a favor and fix it in a timely manner as your PR was incorrect. Going back and forth in a PR takes way more time than talking about it in advance (e.g. in a discussion) and then have a clear plan for a PR from the get go. If you give me a smaller test image I'll add it as a test case.

Sorry if that made you feel unappreciated, that wasn't my intention.

Could you please add me to the commit? @LekoArts

[LekoArts]

@Jeff-Tian Done 👍

[Jeff-Tian]

Please open an issue on https://github.com/lovell/sharp/issues about that specific image then, sharp thinks it's corrupt so we'll handle it that way.

Hi @LekoArts, sorry for the late response to this issue.

With the help of the author of sharp (lovell/sharp#3509), it turns out that the root cause of the building process halt is that some images with png extension actually are BMP images.

And that makes sharp(...).metadata failed as seems sharp doesn't support BMP image formats natively.

I found ways to detect these images (lovell/sharp#806 (comment)).

I am still trying to make my gatsby site building process continue even with those fake png files. So I am thinking to insert some code to detect the images and if BMP is found, then using bmp-js to decode them.

Is it a correct way to insert these kind of logic to gatsby-plugin-sharp? If not, what is the best way to handle BMP images during gatsby building?

Thanks in advance!

[jcpmcc5]

Please open an issue on https://github.com/lovell/sharp/issues about that specific image then, sharp thinks it's corrupt so we'll handle it that way.

Hi @LekoArts, sorry for the late response to this issue.

With the help of the author of sharp (lovell/sharp#3509), it turns out that the root cause of the building process halt is that some images with png extension actually are BMP images.

And that makes sharp(...).metadata failed as seems sharp doesn't support BMP image formats natively.

I found ways to detect these images (lovell/sharp#806 (comment)).

I am still trying to make my gatsby site building process continue even with those fake png files. So I am thinking to insert some code to detect the images and if BMP is found, then using bmp-js to decode them.

Is it a correct way to insert these kind of logic to gatsby-plugin-sharp? If not, what is the best way to handle BMP images during gatsby building?

Thanks in advance!

Any updates on handling bmp images during gatsby building? Thank you