[gatsby-source-wordpress] url rewriting for files no longer works when updating the wordpress page while dev server runs

[rburgst]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

It looks as if audio files within custom fields are no longer being downloaded and rewritten to local links in the case where you have a dev server running and update the page in wordpress. This used to work with gatsby <= 4.9.x.

Reproduction Link

https://github.com/gatsbyjs/gatsby-starter-wordpress-blog

Steps to Reproduce

1. create a new wordpress instance, e.g. via the following docker file:

 version: "3.9"
   
services:
 db:
   image: mariadb
   volumes:
     - db_data:/var/lib/mysql
   restart: always
   environment:
     MYSQL_ROOT_PASSWORD: somewordpress
     MYSQL_DATABASE: wordpress
     MYSQL_USER: wordpress
     MYSQL_PASSWORD: wordpress
   
 wordpress:
   depends_on:
     - db
   image: wordpress:latest
   volumes:
     - wordpress_data:/var/www/html
   ports:
     - "8000:80"
   restart: always
   environment:
     WORDPRESS_DB_HOST: db
     WORDPRESS_DB_USER: wordpress
     WORDPRESS_DB_PASSWORD: wordpress
     WORDPRESS_DB_NAME: wordpress
volumes:
 db_data: {}
 wordpress_data: {}

2. add the following plugins
   ** wp-gatsby
   ** wpgraphql
   ** advanced custom fields
   ** wpgraphql advanced custom fields
3. create a customfield group e.g. foofield which is linked to posts
4. add a field contentField of type wysiwyg
5. go to the sample post that comes with the standard wordpress installation
6. in the custom field editor add some text as well as an mp3 audio
7. now start the gatsby dev server and query the custom field with the following query

 query MyQuery {
 allWpPost {
   nodes {
     content
     foofield {
       contentfield
     }
   }
 }
}

Expected Result

the HTML content in the custom field should contain an <audio> tag that contains the rewritten audio mp3.

Actual Result

{
  "data": {
    "allWpPost": {
      "nodes": [
        {
          "content": "\n<p>Welcome to WordPress. This is your first post. Edit or delete it, then start writing!</p>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"http://localhost:8000/wp-content/uploads/2022/05/Aufloesung_stimmen1_2_3_4-klavier_SW.mp3\"></audio></figure>\n",
          "foofield": {
            "contentfield": "<p>foo content</p>\n<!--[if lt IE 9]><script>document.createElement('audio');</script><![endif]-->\n<audio class=\"wp-audio-shortcode\" id=\"audio-1-1\" preload=\"none\" style=\"width: 100%;\" controls=\"controls\"><source type=\"audio/mpeg\" src=\"http://localhost:8000/wp-content/uploads/2022/05/Aufloesung_stimmen1_2_3_4-klavier_SW.mp3?_=1\" /><a href=\"http://localhost:8000/wp-content/uploads/2022/05/Aufloesung_stimmen1_2_3_4-klavier_SW.mp3\">http://localhost:8000/wp-content/uploads/2022/05/Aufloesung_stimmen1_2_3_4-klavier_SW.mp3</a></audio>\n"
          }
        }
      ]
    }
  },
  "extensions": {}
}

Note that the src attribute points to the original wordpress url of the mp3 file.

Environment

System:
    OS: macOS 12.3.1
    CPU: (10) arm64 Apple M1 Pro
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.13.1 - /var/folders/9h/6y70jr3x7cb4cykkjldnppgc0000gn/T/yarn--1652892929785-0.9133048353254971/node
    Yarn: 1.22.18 - /var/folders/9h/6y70jr3x7cb4cykkjldnppgc0000gn/T/yarn--1652892929785-0.9133048353254971/yarn
    npm: 8.1.2 - ~/.volta/tools/image/node/16.13.1/bin/npm
  Browsers:
    Chrome: 101.0.4951.64
    Firefox: 100.0.1
    Safari: 15.4
  npmPackages:
    gatsby: ^4.0.0 => 4.14.0 
    gatsby-plugin-image: ^2.0.0 => 2.14.0 
    gatsby-plugin-manifest: ^4.0.0 => 4.14.0 
    gatsby-plugin-offline: ^5.0.0 => 5.14.0 
    gatsby-plugin-react-helmet: ^5.0.0 => 5.14.0 
    gatsby-plugin-sharp: ^4.0.0 => 4.14.0 
    gatsby-source-filesystem: ^4.0.0 => 4.14.0 
    gatsby-source-wordpress: ^6.0.0 => 6.14.0 
    gatsby-transformer-sharp: ^4.0.0 => 4.14.0

Config Flags

N/A, none have been configured in the sample wordpress repo.

[rburgst]

this is related to #35626

[rburgst]

it appears that fetchReferencedMediaItems no longer returns previously cached node ids
https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-wordpress/src/steps/source-nodes/create-nodes/process-node.js#L754-L757

So the consequence is that when you have the dev server running and you update the page with the custom field, then the mp3 file link will not be replaced and the graphql query will return the original url.

so at least when updating a page the following result is ignored:

gatsby/packages/gatsby-source-wordpress/src/steps/source-nodes/fetch-nodes/fetch-referenced-media-items.js

Line 335 in 7156882

const previouslyCachedMediaItemNodes = await Promise.all(

During debugging I found that it works correctly if I clear the cache before (due to the fact that in that case we dont have any pre-cached media files and therefore, the file will actually be fetched from wordpress).

Unfortunately I still have a similar problem in our production app where clearing the cache does not seem to work either. So it must be yet another issue.

[rburgst]

I have debugged a bit further and in https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-wordpress/src/steps/source-nodes/create-nodes/process-node.js#L769 my MediaItem node does not have a localFile element.
Is it possible that this is a race condition?

Turns out I had mp3 files ignored from downloading via mime type filters. So also in my production app the rewriting works.
So for now the only problem I can really replicate (even with the trivial example wordpress installation) is refreshing the dev server when you edit a page in wordpress while the dev server is running. In that case previously cached media items will not be rewritten (as per my previous comment).

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hey again!

It’s been 60 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to comment on this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜