Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gallery block: image media API call throws 500 error #41825

Closed
glendaviesnz opened this issue Jun 20, 2022 · 35 comments
Closed

Gallery block: image media API call throws 500 error #41825

glendaviesnz opened this issue Jun 20, 2022 · 35 comments
Labels
[Block] Gallery Affects the Gallery Block - used to display groups of images [Type] Bug An existing feature does not function as intended

Comments

@glendaviesnz
Copy link
Contributor

glendaviesnz commented Jun 20, 2022

Description

As noted here there are some instances where the call by the Gallery block to the API to get image data for all gallery images results in a 500 error.

Since #34389 the request to get the image data was combined into a single request, so this might be related to a server resource issue as the issue report here relates to a gallery with over 100 images.

Step-by-step reproduction instructions

I was not able to replicate this issue on a local install, but @dynamek gets it consistently on their install, but only with a large gallery with over 100 images.

@glendaviesnz glendaviesnz added [Type] Bug An existing feature does not function as intended [Block] Gallery Affects the Gallery Block - used to display groups of images labels Jun 20, 2022
@glendaviesnz
Copy link
Contributor Author

glendaviesnz commented Jun 20, 2022

@Mamaduka it seems like there might be some instances where the combined media api call in the Gallery is causing a 500 server error. The user reporting it says there isn't anything obvious in the debug log, and I can't replicate it locally. Do you have any thoughts on the best way to try and track down the cause of the server error with this API call?

@dynamek
Copy link

dynamek commented Jun 21, 2022

I've tried everything I can think of server side including:
Increasing memory limit to 2048M
Max execution 2400
Max input 2400
Post max size 32M

Additional directives for HTTP & HTTPS:
FcgidBusyTimeout 600
FcgidIOTimeout 550

Additional nginx directives:
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
proxy_buffers 8 32k;
proxy_buffer_size 64k;
fastcgi_buffers 8 32k;
fastcgi_buffer_size 64k;

@Mamaduka
Copy link
Member

Thanks for the ping, @glendaviesnz. I will try to look into this today/tomorrow.

@getsource
Copy link
Member

I asked about this / forwarded the ticket to the hosting community team to see if anyone else has encountered the issue there.

@dynamek:

  • Is there any particular host that you are using, or are you the host? Wondering because we could potentially reach out to the host directly.
  • Are you using any sort of Web Application Firewall (WAF) that might be blocking / intercepting requests?

@dynamek
Copy link

dynamek commented Jun 22, 2022

I am the host, but with UKFast so I can ask for their assistance if need be. I don't believe any firewall would be affecting it.

@BernardZijlstra
Copy link

Do you have Nginx error log entries available from these 500 errors?

@dynamek
Copy link

dynamek commented Jun 22, 2022

Yes - latest few lines are

2022-06-19 22:58:43 Error 147.147.192.4   11065#0: *1102068 upstream sent too big header while reading response header from upstream       nginx error
2022-06-19 22:59:32 Error 147.147.192.4   11065#0: *1102795 upstream sent too big header while reading response header from upstream       nginx error
2022-06-20 11:55:59 Error 147.147.192.4   3929#0: *7 connect() failed (111: Connection refused) while connecting to upstream       nginx error

I don't remember there being nginx error when I first found this issue so not sure if they have occurred from adding 'Additional nginx directives' as above.

@BernardZijlstra
Copy link

Looks like the HTTP headers won't fit into the buffer. Try tweaking the large_client_header_buffers setting from Nginx which default it is "4 8k" enlarge it to "4 16k".

@dynamek
Copy link

dynamek commented Jun 23, 2022

Thank you - I have now done this but the image options are still buffering for one gallery block.

This is what it is now set to
large_client_header_buffers 4 16k;

@dynamek
Copy link

dynamek commented Jun 23, 2022

Also on the one that I can now set the image options for hasn't fixed the problem that now over half the images aren't openable with lightbox on the front end. If you scroll to about half of the way down and hover some images you will see that the image does not have a link with fancy box class assigned. I believe this to be because the image option wasn't set correctly but it hasn't fixed the problem there.
https://dev.lalachoirs.co.uk/photo-gallery/bicton-summer-social-2019-and-head-shave-2/

@dynamek
Copy link

dynamek commented Jun 23, 2022

I'm getting apache error

Premature end of script headers: index.php, referer: https://dev.lalachoirs.co.uk/wp-admin/post.php?post=62085&action=edit
for this gallery https://dev.lalachoirs.co.uk/photo-gallery/la-las-10th-birthday-party/

@BernardZijlstra
Copy link

If I look at the pictures loading from both gallery url's, the load okay according to the network tab from the dev tools. I assume that the error from the Nginx log is not showing up now. With Plesk Nginx functions as a proxy in front of Apache httpd so it is possible that your problem moved from Nginx to httpd. Observe the error logs when interacting with the photo gallery.

@dynamek
Copy link

dynamek commented Jun 23, 2022

Looks to have reverted back to just apache errors. There are no errors from loading the front end.

2022-06-23 23:00:48 Error 147.147.192.4 500 GET /wp-json/wp/v2/media?context=edit&include%5B0%5D=55608&include%5B1%5D=55609&include%5B2%5D=55610&include%5B3%5D=55611&include%5B4%5D=55612&include%5B5%5D=55613&include%5B6%5D=55614&include%5B7%5D=55615&include%5B8%5D=55616&include%5B9%5D=55617&include%5B10%5D=55618&include%5B11%5D=55619&include%5B12%5D=55620&include%5B13%5D=55621&include%5B14%5D=55622&include%5B15%5D=55623&include%5B16%5D=55624&include%5B17%5D=55625&include%5B18%5D=55626&include%5B19%5D=55627&include%5B20%5D=55629&include%5B21%5D=55630&include%5B22%5D=55631&include%5B23%5D=55632&include%5B24%5D=55633&include%5B25%5D=55634&include%5B26%5D=55635&include%5B27%5D=55636&include%5B28%5D=55637&include%5B29%5D=55638&include%5B30%5D=55639&include%5B31%5D=55640&include%5B32%5D=55641&include%5B33%5D=55642&include%5B34%5D=55643&include%5B35%5D=55644&include%5B36%5D=55645&include%5B37%5D=55646&include%5B38%5D=55647&include%5B39%5D=55648&include%5B40%5D=55649&include%5B41%5D=55662&include%5B42%5D=55663&include%5B43%5D=55664&include%5B44%5D=55665&include%5B45%5D=55666&include%5B46%5D=55667&include%5B47%5D=55668&include%5B48%5D=55669&include%5B49%5D=55670&include%5B50%5D=55671&include%5B51%5D=55672&include%5B52%5D=55673&include%5B53%5D=55674&include%5B54%5D=55675&include%5B55%5D=55676&include%5B56%5D=55677&include%5B57%5D=55678&include%5B58%5D=55679&include%5B59%5D=55680&include%5B60%5D=55681&include%5B61%5D=55682&include%5B62%5D=55683&include%5B63%5D=55685&include%5B64%5D=55695&include%5B65%5D=55696&include%5B66%5D=55697&include%5B67%5D=55698&include%5B68%5D=55689&include%5B69%5D=55701&include%5B70%5D=55702&include%5B71%5D=55703&include%5B72%5D=55720&include%5B73%5D=55722&include%5B74%5D=55737&include%5B75%5D=55738&include%5B76%5D=55739&include%5B77%5D=55740&include%5B78%5D=55741&include%5B79%5D=55742&include%5B80%5D=55743&include%5B81%5D=55744&include%5B82%5D=55745&include%5B83%5D=55746&include%5B84%5D=55747&include%5B85%5D=55748&include%5B86%5D=55749&include%5B87%5D=55750&include%5B88%5D=55751&include%5B89%5D=55752&include%5B90%5D=55753&include%5B91%5D=55754&include%5B92%5D=55755&include%5B93%5D=55756&include%5B94%5D=55757&include%5B95%5D=55758&include%5B96%5D=55759&include%5B97%5D=55783&include%5B98%5D=55782&include%5B99%5D=55781&include%5B100%5D=55780&include%5B101%5D=55779&include%5B102%5D=55778&include%5B103%5D=55777&include%5B104%5D=55776&include%5B105%5D=55775&include%5B106%5D=55774&include%5B107%5D=55773&include%5B108%5D=55772&include%5B109%5D=55771&include%5B110%5D=55770&include%5B111%5D=55769&include%5B112%5D=55768&include%5B113%5D=55767&include%5B114%5D=55766&include%5B115%5D=55765&include%5B116%5D=55764&include%5B117%5D=55763&include%5B118%5D=55762&include%5B119%5D=55761&include%5B120%5D=55760&include%5B121%5D=55784&include%5B122%5D=55797&include%5B123%5D=55796&include%5B124%5D=55795&include%5B125%5D=55794&include%5B126%5D=55793&include%5B127%5D=55792&include%5B128%5D=55791&include%5B129%5D=55790&include%5B130%5D=55789&include%5B131%5D=55788&include%5B132%5D=55787&include%5B133%5D=55786&include%5B134%5D=55785&include%5B135%5D=55798&include%5B136%5D=55812&include%5B137%5D=55811&include%5B138%5D=55810&include%5B139%5D=55809&include%5B140%5D=55808&include%5B141%5D=55807&include%5B142%5D=55806&include%5B143%5D=55805&include%5B144%5D=55804&include%5B145%5D=55803&include%5B146%5D=55802&include%5B147%5D=55801&include%5B148%5D=55800&include%5B149%5D=55799&include%5B150%5D=55813&include%5B151%5D=55829&include%5B152%5D=55828&include%5B153%5D=55827&include%5B154%5D=55826&include%5B155%5D=55825&include%5B156%5D=55824&include%5B157%5D=55823&include%5B158%5D=55822&include%5B159%5D=55821&include%5B160%5D=55820&include%5B161%5D=55819&include%5B162%5D=55818&include%5B163%5D=55817&include%5B164%5D=55816&include%5B165%5D=55815&include%5B166%5D=55814&include%5B167%5D=55830&include%5B168%5D=55847&include%5B169%5D=55846&include%5B170%5D=55845&include%5B171%5D=55844&include%5B172%5D=55843&include%5B173%5D=55842&include%5B174%5D=55841&include%5B175%5D=55840&include%5B176%5D=55839&include%5B177%5D=55838&include%5B178%5D=55837&include%5B179%5D=55836&include%5B180%5D=55835&include%5B181%5D=55834&include%5B182%5D=55833&include%5B183%5D=55832&include%5B184%5D=55831&include%5B185%5D=55848&include%5B186%5D=55878&include%5B187%5D=55877&include%5B188%5D=55876&include%5B189%5D=55875&include%5B190%5D=55874&include%5B191%5D=55873&include%5B192%5D=55872&include%5B193%5D=55871&include%5B194%5D=55870&include%5B195%5D=55869&include%5B196%5D=55868&include%5B197%5D=55867&include%5B198%5D=55866&include%5B199%5D=55865&include%5B200%5D=55864&include%5B201%5D=55863&include%5B202%5D=55862&include%5B203%5D=55861&include%5B204%5D=55860&include%5B205%5D=55859&include%5B206%5D=55858&include%5B207%5D=55857&include%5B208%5D=55856&include%5B209%5D=55855&include%5B210%5D=55854&include%5B211%5D=55853&include%5B212%5D=55852&include%5B213%5D=55851&include%5B214%5D=55850&include%5B215%5D=55849&include%5B216%5D=55879&include%5B217%5D=55909&include%5B218%5D=55908&include%5B219%5D=55907&include%5B220%5D=55906&include%5B221%5D=55905&include%5B222%5D=55904&include%5B223%5D=55903&include%5B224%5D=55902&include%5B225%5D=55901&include%5B226%5D=55900&include%5B227%5D=55899&include%5B228%5D=55898&include%5B229%5D=55897&include%5B230%5D=55896&include%5B231%5D=55895&include%5B232%5D=55894&include%5B233%5D=55893&include%5B234%5D=55892&include%5B235%5D=55891&include%5B236%5D=55890&include%5B237%5D=55889&include%5B238%5D=55888&include%5B239%5D=55887&include%5B240%5D=55886&include%5B241%5D=55885&include%5B242%5D=55884&include%5B243%5D=55883&include%5B244%5D=55882&include%5B245%5D=55881&include%5B246%5D=55880&per_page=100&_locale=user&page=2 HTTP/1.0     898 Apache SSL/TLS access
2022-06-23 23:00:50 Error 147.147.192.4   Premature end of script headers: index.php, referer: https://dev.lalachoirs.co.uk/wp-admin/post.php?post=62085&action=edit       Apache error

@dynamek
Copy link

dynamek commented Jun 24, 2022

Ok so we have managed to fix the 'loading options' issue by following the suggestions here
https://wordpress.org/support/topic/rest-api-error-when-adding-too-much-images-on-wordpress-gallery-image-size-load/
See final comment link.

Now we are back to having nginx errors - example:

2022-06-24 17:21:42 Error 147.147.192.4   13216#0: *1881908 open() "/var/www/vhosts/lalachoirs.co.uk/dev.lalachoirs.co.uk/wp-json/wp/v2/media" failed (2: No such file or directory)       nginx error

And the problem of no link and therefore the lightbox not working on some of the images on front end.

@ramonjd
Copy link
Member

ramonjd commented Aug 30, 2022

Are there any changes in Gutenberg required here? Can folks still replicate the issue?

@bezbee
Copy link

bezbee commented Aug 31, 2022

I experience the same issue with a gallery block of as fewer as 3 images. See here - https://japanball.com/articles-features/japanball-tours/guest-memoirs/2022-netherlands-tour-recap-and-photo-log/

A handful of galleries on the page that do not link images to media files have image size in the constantly loading state.
What details can I provide for the issue to be looked at and resolved?

@bezbee
Copy link

bezbee commented Sep 7, 2022

Any update on the issue? Is anyone stiil working on it?

@glendaviesnz
Copy link
Contributor Author

What details can I provide for the issue to be looked at and resolved?

Are you able to provide details about any error messages that show in the browser dev tools console, or network tab when you load the page in the editor and the image size select hangs on loading?

@bezbee
Copy link

bezbee commented Sep 14, 2022

@glendaviesnz,
Thanks for helping me!
I provided a link for the post where the gallery block that is set to "link media to their media files" does not behave as expected.
https://japanball.com/articles-features/japanball-tours/guest-memoirs/2022-netherlands-tour-recap-and-photo-log/

The console tab shows no errors and no relevant blocked resources on the network tab could be noticed.

Can I do anything else? Is there a fix?

@bezbee
Copy link

bezbee commented Oct 4, 2022

Any update?

@glendaviesnz
Copy link
Contributor Author

Sorry about the delay on replying @bezbee. Are you able to add the following filter to the network tab rest_route=%2Fwp%2Fv2%2Fmedia and then reload a page with the problem Gallery block in the editor, and let me know if that API call returns any error messages?

@bezbee
Copy link

bezbee commented Oct 5, 2022

Hey @glendaviesnz!
Thanks for being so responsive!
Below are some details.

When the page is opened in the editor:
The console tab shows errors
Screen Shot 2022-10-04 at 20 39 09

The network tab does not show any errors (statuses: 101, 200, 204, 206)
Screen Shot 2022-10-04 at 20 38 00

Did I do correctly what you asked for?

@glendaviesnz
Copy link
Contributor Author

504 indicates that the server is timing out before the request is returned, so I don't think the problem is related to the Gallery block. @Mamaduka have you seen anything like the above 504 error on media retrieval before? It is unrelated to the 500 header length issue I think.

@Mamaduka
Copy link
Member

Mamaduka commented Oct 6, 2022

I can't recall.

I see that 504 happens even when include has a single ID. I guess that some plugin is performing a heavy operation when images are requested.

Re: 500 errors

I came across this issue #11541 last week, and there's an example of how core solved a similar issue for the script loader. So maybe we could try doing the same with the new apiFetch middleware?

P.S. I think we can close this and all 500 error-related issues as duplicates of #11541.

@glendaviesnz
Copy link
Contributor Author

Thanks @Mamaduka. @bezbee looking at those errors, I don't think this a Gutenberg related issue, as @Mamaduka notes there might be another plugin in your system that is causing timeouts when processing images. You could try disabling any image related plugins one at a time and see if that resolves the issue, if it does then contact the plugin authors of the one that seems to cause the issue.

Closing this one now in favour of #11541

@dynamek
Copy link

dynamek commented Dec 19, 2022

I am having this problem again with a new gallery block today. A difference though with this one is that none of the images are able to open in lightbox. The gallery has 160 photos in it.
I have recently updated PHP to 8.1.13 but the additional directives I was advised to add are still in place.
FcgidBusyTimeout 2400
FcgidIOTimeout 2400
LimitRequestLine 16380
LimitRequestFieldSize 16380

I've tried deactivating image related plugins and that doesn't help.
Is someone working to fix this in the core Wordpress files?

@kbauman
Copy link

kbauman commented Dec 21, 2022

I just tried to edit a gallery with over 100 images, and it broke the gallery. Now I'm here with the same problem.

@Mamaduka
Copy link
Member

@dynamek, @kbauman, do you see any errors in the DevTools console?

I recently fixed a regression that was only affection galleries with over 100 images - #46143.

Any details that can help narrow down the problem are much appreciated.

@kbauman
Copy link

kbauman commented Dec 21, 2022

@Mamaduka

This is the error: per_page must be between 1 (inclusive) and 100 (inclusive)

If I edit in FF dev tools, and change the per_page to 100, and resend, I get a 502 bad gateway. The gallery was working fine, until I tried to edit it today.

The images load, and I can add and delete images, but none of the images will link to the media file anymore.

@Mamaduka
Copy link
Member

Thanks for more details, @kbauman!

This is the error: per_page must be between 1 (inclusive) and 100 (inclusive)

This error is fixed via #46143, and it's already available in Gutenberg plugin 14.7.

I understand that not every project can install the plugin, so a small snippet below will increase the maximum supported per_page for the media endpoint.

/**
 * Increase the default WP core maximum for the media endpoint.
 */
add_filter( 'rest_attachment_collection_params', function( $query_params ) {
	$query_params['per_page']['maximum'] = 200;

	return $query_params;
} );

@kbauman
Copy link

kbauman commented Dec 21, 2022

Great! Thank you. That fixed it.

@dynamek
Copy link

dynamek commented Dec 22, 2022

Ok so that has fixed the 'loading options...' bug - thank you.
However it hasn't fixed the link to enable lightbox on the front end. Do we have to re-add the gallery block for this to work?

@Mamaduka
Copy link
Member

Thanks for the feedback, @dynamek. The core doesn't ship with lightbox, so it's hard to debug a thing without knowing what you are using. Do you mind sharing the name of the plugin?

@dynamek
Copy link

dynamek commented Dec 23, 2022

I don't know if that is relevant as it is the whole link that isn't there. If there was no lightbox installed it should still have the link that would open the image but this is not there. The gallery is sent to Link to Media File,
Example https://www.lalachoirs.co.uk/photo-gallery/exeter-cathedral-2022/
If you want to know though I am using Easy Fancybox

@dynamek
Copy link

dynamek commented Jan 8, 2023

Hi - has anyone had a chance to get back to this one after the holidays? I'd be grateful for a solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Block] Gallery Affects the Gallery Block - used to display groups of images [Type] Bug An existing feature does not function as intended
Development

No branches or pull requests

8 participants