Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[framework] load products iteratively while generating image sitemaps #3144

Merged
merged 2 commits into from
May 15, 2024
Merged

[framework] load products iteratively while generating image sitemaps #3144

merged 2 commits into from
May 15, 2024

Conversation

AndrejBlaho
Copy link
Contributor

@AndrejBlaho AndrejBlaho commented Apr 30, 2024 •
edited by github-actions bot

Q A
Description, reason for the PR Image sitemap generation was failing on memory if there was large number of products loaded at once. Now products are loaded iteratively which should help with the issue. Rewriting cron into iterative one would solve the problem completely regardless of data size, however Presta bundle does not support iterative data load, so we decided to go with this compromise for now.
New feature No
BC breaks Yes
Fixes issues closes #2618
Have you read and signed our License Agreement for contributions? Yes

馃寪 Live Preview:

@AndrejBlaho AndrejBlaho added the Enhancement New feature or request for change from user point of view label Apr 30, 2024
@AndrejBlaho AndrejBlaho self-assigned this Apr 30, 2024
@AndrejBlaho AndrejBlaho force-pushed the ab-generate-image-sitemap-iteratively branch 6 times, most recently from 71d4290 to 55580ac Compare May 6, 2024 19:04
vitek-rostislav
vitek-rostislav approved these changes May 7, 2024
View reviewed changes
UPGRADE-15.0.md Outdated Show resolved Hide resolved
@AndrejBlaho AndrejBlaho force-pushed the ab-generate-image-sitemap-iteratively branch 3 times, most recently from 52d943a to 916fb1e Compare May 7, 2024 13:39
@vitek-rostislav
Copy link
Contributor

鉁旓笍 Tested locally with ~40k products in the DB, the problem is solved 馃憣 In the production mode it works like a charm. The truth is, in dev mode, the generating fails on the memory exhaustion but the reason is the doctrine backtrace collecting that was enabled in #3055.

So after this tweak, to be able to generate the sitemap locally, you need either switch to the production mode or set profiling_collect_backtrace to false in packages/dev/doctrine.yaml

@AndrejBlaho AndrejBlaho force-pushed the ab-generate-image-sitemap-iteratively branch from 916fb1e to 25ec55c Compare May 15, 2024 09:45
@AndrejBlaho AndrejBlaho merged commit 3f80fc6 into 15.0 May 15, 2024
17 checks passed
@AndrejBlaho AndrejBlaho deleted the ab-generate-image-sitemap-iteratively branch May 15, 2024 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vitek-rostislav vitek-rostislav vitek-rostislav approved these changes

Assignees

@AndrejBlaho AndrejBlaho

Labels
Enhancement New feature or request for change from user point of view
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Image sitemap generating fail on memory
2 participants
@AndrejBlaho @vitek-rostislav