Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(gatsby-source-contentful): Increase Contentful sync by up to 10x #30422

Merged
merged 5 commits into from Mar 30, 2021
Merged

feat(gatsby-source-contentful): Increase Contentful sync by up to 10x #30422

merged 5 commits into from Mar 30, 2021

Conversation

axe312ger
Copy link
Collaborator

  • increase default pageLimit to 1000
  • add backoff logic that retries with a lower pageLimit till we are able to sync data from Contentful

The first commit in here should be replaced by #30257

@axe312ger axe312ger added the status: needs core review Currently awaiting review from Core team member label Mar 23, 2021
@axe312ger axe312ger added this to In progress in Contentful source plugin improvements via automation Mar 23, 2021
@gatsbot gatsbot bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Mar 23, 2021
@LekoArts LekoArts added topic: source-contentful Related to Gatsby's integration with Contentful and removed status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer labels Mar 25, 2021
@axe312ger axe312ger force-pushed the feat/contentful-page-limit-backoff branch from 69789c9 to b6a7dd5 Compare March 29, 2021 08:50
wardpeet
wardpeet reviewed Mar 29, 2021
View reviewed changes
Copy link
Contributor

@wardpeet wardpeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small comments

packages/gatsby-source-contentful/src/fetch.js Outdated Show resolved Hide resolved
packages/gatsby-source-contentful/src/fetch.js Show resolved Hide resolved
@axe312ger
Copy link
Collaborator Author

@wardpeet done

@axe312ger axe312ger force-pushed the feat/contentful-page-limit-backoff branch from a49c874 to 4fc1237 Compare March 29, 2021 16:25
wardpeet
wardpeet approved these changes Mar 30, 2021
View reviewed changes
Copy link
Contributor

@wardpeet wardpeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@wardpeet wardpeet changed the title Perf: Increase Contentful sync by up to 10x feat(gatsby-source-contentful): Increase Contentful sync by up to 10x Mar 30, 2021
@wardpeet wardpeet merged commit b9791fe into master Mar 30, 2021
Contentful source plugin improvements automation moved this from In progress to Done Mar 30, 2021
@wardpeet wardpeet deleted the feat/contentful-page-limit-backoff branch March 30, 2021 08:50
@axe312ger axe312ger added this to To cherry-pick in Release candidate via automation Mar 30, 2021
@axe312ger axe312ger added this to To cherry-pick in V2 Release hotfixes via automation Mar 30, 2021
vladar pushed a commit that referenced this pull request Mar 30, 2021
@axe312ger @vladar
feat(gatsby-source-contentful): Increase Contentful sync by up to 10x (…
bfe289f 
…#30422)

Co-authored-by: Ward Peeters <ward@coding-tech.com>
(cherry picked from commit b9791fe)
@gatsbybot gatsbybot mentioned this pull request Mar 30, 2021
Merged
@vladar vladar moved this from To cherry-pick to Backport PR opened in Release candidate Mar 30, 2021
vladar pushed a commit that referenced this pull request Mar 30, 2021
@axe312ger
feat(gatsby-source-contentful): Increase Contentful sync by up to 10x (…
b3861fd 
…#30422) (#30571)

Co-authored-by: Ward Peeters <ward@coding-tech.com>
(cherry picked from commit b9791fe)

Co-authored-by: Benedikt Rötsch <axe312ger@users.noreply.github.com>
@vladar vladar moved this from Backport PR opened to Backported in Release candidate Mar 30, 2021
@gatsbybot gatsbybot mentioned this pull request Apr 2, 2021
Merged
pieh pushed a commit that referenced this pull request Apr 2, 2021
@axe312ger @pieh
feat(gatsby-source-contentful): Increase Contentful sync by up to 10x (…
d44ea1f 
…#30422)

Co-authored-by: Ward Peeters <ward@coding-tech.com>
(cherry picked from commit b9791fe)
@pieh pieh moved this from To cherry-pick to Backport PR opened in V2 Release hotfixes Apr 2, 2021
@pieh pieh moved this from Backport PR opened to Backported in V2 Release hotfixes Apr 2, 2021
pieh pushed a commit that referenced this pull request Apr 2, 2021
@axe312ger
feat(gatsby-source-contentful): Increase Contentful sync by up to 10x (…
fc61c88 
…#30422) (#30643)

Co-authored-by: Ward Peeters <ward@coding-tech.com>
(cherry picked from commit b9791fe)

Co-authored-by: Benedikt Rötsch <axe312ger@users.noreply.github.com>
This was referenced Jul 16, 2021
Open
Open
Open
Open
Open
Open
Open
pragmaticpat pushed a commit to pragmaticpat/gatsby that referenced this pull request Apr 28, 2022
@ViCo0TeCH @wardpeet
feat(gatsby-source-contentful): Increase Contentful sync by up to 10x (…
cad651b 
…gatsbyjs#30422)

Co-authored-by: Ward Peeters <ward@coding-tech.com>
@tonyg101 tonyg101 mentioned this pull request Jul 29, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wardpeet wardpeet wardpeet approved these changes

Assignees
No one assigned
Labels
status: needs core review Currently awaiting review from Core team member topic: source-contentful Related to Gatsby's integration with Contentful
Projects
Contentful source plugin improvements
  
Done
Release candidate
Backported
V2 Release hotfixes
Backported
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@axe312ger @wardpeet @LekoArts