Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added rounding when converting P and PA #5824

Merged
merged 1 commit into from Dec 28, 2021
Merged

Conversation

radarhere
Copy link
Member

Resolves #5728

The issue discusses a pixel in an image with the palette value (177, 175, 175), and asks why converting from P to L doesn't convert it to 176. That conversion uses the L macro -

p2l(UINT8 *out, const UINT8 *in, int xsize, const UINT8 *palette) {
int x;
/* FIXME: precalculate greyscale palette? */
for (x = 0; x < xsize; x++) {
*out++ = L(&palette[in[x] * 4]) / 1000;

#define L(rgb) ((INT32)(rgb)[0] * 299 + (INT32)(rgb)[1] * 587 + (INT32)(rgb)[2] * 114)

Applying the L macro,

(177*299 + 175*587 + 175*114) / 1000 = 175.598

Except the p2l code above rounds it down. This PR applies proper rounding to bring it to 176.

@@ -93,7 +93,7 @@ def test_trns_p(tmp_path):
f = str(tmp_path / "temp.png")

im_l = im.convert("L")
assert im_l.info["transparency"] == 0 # undone
assert im_l.info["transparency"] == 1 # undone
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first palette entry is (0, 1, 2). (0*299 + 1*587 + 2*114) / 1000 = 0.815, so this is now rounded to 1.

@radarhere radarhere changed the title Round output from L macro Round output from L macro in palette conversion Nov 9, 2021
@@ -1013,7 +1013,7 @@ p2l(UINT8 *out, const UINT8 *in, int xsize, const UINT8 *palette) {
int x;
/* FIXME: precalculate greyscale palette? */
for (x = 0; x < xsize; x++) {
*out++ = L(&palette[in[x] * 4]) / 1000;
*out++ = round(L(&palette[in[x] * 4]) / 1000.0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Floating point arithmetic slower than integer. It's better to use L24 macro here, it already contains rounding.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I've updated the commit.

@radarhere radarhere changed the title Round output from L macro in palette conversion Added rounding to P mode conversion Nov 9, 2021
@radarhere radarhere changed the title Added rounding to P mode conversion Added rounding when converting P and PA Nov 9, 2021
@hugovk hugovk merged commit e7b5325 into python-pillow:main Dec 28, 2021
@radarhere radarhere deleted the l_macro branch December 28, 2021 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Conversion P to L wrong?
3 participants