Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scripts: Change webpack configuration to include render files in the build folder #43917

Merged
merged 7 commits into from Sep 9, 2022
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/scripts/CHANGELOG.md
Expand Up @@ -2,6 +2,10 @@

## Unreleased

### New Feature

- Update webpack configuration for the `build` and `start` commands to automatically copy PHP files listed in the `render` field of `block.json` files from the source to the build folder ([#43917](https://github.com/WordPress/gutenberg/pull/43917)).

## 24.0.0 (2022-08-24)

### Breaking Change
Expand Down
32 changes: 20 additions & 12 deletions packages/scripts/README.md
Expand Up @@ -62,19 +62,23 @@ We commit to keeping the breaking changes minimal so you can upgrade `@wordpress

### `build`

Transforms your code according the configuration provided so it’s ready for production and optimized for the best performance. The entry points for your project get detected by scanning all script fields in `block.json` files located in the `src` directory. The script fields in `block.json` should pass relative paths to `block.json` in the same folder.
Transforms your code according the configuration provided so it’s ready for production and optimized for the best performance.

_This script exits after producing a single build. For incremental builds, better suited for development, see the [start](#start) script._

The entry points for your project get detected by scanning all script fields in `block.json` files located in the `src` directory. The script fields in `block.json` should pass relative paths to `block.json` in the same folder.

_Example:_

```json
{
"editorScript": "file:index.js",
"editorStyle": "file:editor.css",
"style": "file:style.css"
"script": "file:script.js",
"viewScript": "file:view.js"
}
```

The fallback entry point is `src/index.js` (other supported extensions: `.jsx`, `.ts`, and `.tsx`) in case there is no `block.json` file found. The output generated will be written to `build/index.js`. This script exits after producing a single build. For incremental builds, better suited for development, see the [start](#start) script.
The fallback entry point is `src/index.js` (other supported extensions: `.jsx`, `.ts`, and `.tsx`) in case there is no `block.json` file found. In that scenario, the output generated will be written to `build/index.js`.

_Example:_

Expand All @@ -93,13 +97,13 @@ This is how you execute the script with presented setup:

- `npm run build` - builds the code for production.
- `npm run build:custom` - builds the code for production with two entry points and a custom output directory. Paths for custom entry points are relative to the project root.
- `npm run build:copy-php` - builds the code for production and opts into copying PHP files from the `src` directory and its subfolders to the output directory.
- `npm run build:copy-php` - builds the code for production and opts into copying all PHP files from the `src` directory and its subfolders to the output directory. By default, only PHP files listed in the `render` field in the detected `block.json` files get copied.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, only PHP files listed in the render field in the detected block.json files get copied.

As I understand from #42430 the new render field in block.json only accepts a path for just one file.

Shouldn't this say something like...

By default, only the PHP file defined for the render field (if any) in the detected block.json get copied.

Copy link
Member

@gziolo gziolo Sep 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There might be multiple block.json files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, it's perfect in the way it's written 👍

- `build:custom-directory` - builds the code for production using the `custom-directory` as the source code directory.

This script automatically use the optimized config but sometimes you may want to specify some custom options:

- `--webpack-bundle-analyzer` – enables visualization for the size of webpack output files with an interactive zoomable treemap.
- `--webpack-copy-php` – enables copying PHP files from the source directory ( default is `src` ) and its subfolders to the output directory.
- `--webpack-copy-php` – enables copying all PHP files from the source directory ( default is `src` ) and its subfolders to the output directory.
- `--webpack-no-externals` – disables scripts' assets generation, and omits the list of default externals.
- `--webpack-src-dir` – Allows customization of the source code directory. Default is `src`.

Expand Down Expand Up @@ -335,19 +339,23 @@ It reuses the same logic as `npm pack` command to create an npm package tarball.

### `start`

Transforms your code according the configuration provided so it’s ready for development. The script will automatically rebuild if you make changes to the code, and you will see the build errors in the console. The entry points for your project get detected by scanning all script fields in `block.json` files located in the `src` directory. The script fields in `block.json` should pass relative paths to `block.json` in the same folder.
Transforms your code according the configuration provided so it’s ready for development. The script will automatically rebuild if you make changes to the code, and you will see the build errors in the console.

_For single builds, better suited for production, see the [build](#build) script._

The entry points for your project get detected by scanning all script fields in `block.json` files located in the `src` directory. The script fields in `block.json` should pass relative paths to `block.json` in the same folder.

_Example:_

```json
{
"editorScript": "file:index.js",
"editorStyle": "file:editor.css",
"style": "file:style.css"
"script": "file:script.js",
"viewScript": "file:view.js"
}
```

The fallback entry point is `src/index.js` (other supported extensions: `.jsx`, `.ts`, and `.tsx`) in case there is no `block.json` file found. The output generated will be written to `build/index.js`. For single builds, better suited for production, see the [build](#build) script.
The fallback entry point is `src/index.js` (other supported extensions: `.jsx`, `.ts`, and `.tsx`) in case there is no `block.json` file found. In that scenario, the output generated will be written to `build/index.js`.

_Example:_

Expand All @@ -368,14 +376,14 @@ This is how you execute the script with presented setup:
- `npm start` - starts the build for development.
- `npm run start:hot` - starts the build for development with "Fast Refresh". The page will automatically reload if you make changes to the files.
- `npm run start:custom` - starts the build for development which contains two entry points and a custom output directory. Paths for custom entry points are relative to the project root.
- `npm run start:copy-php` - starts the build for development and opts into copying PHP files from the `src` directory and its subfolders to the output directory.
- `npm run start:copy-php` - starts the build for development and opts into copying all PHP files from the `src` directory and its subfolders to the output directory. By default, only PHP files listed in the `render` field in the detected `block.json` files get copied.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, only PHP files listed in the render field in the detected block.json files get copied.

As I understand from #42430 the new render field in block.json only accepts a path for just one file.

Shouldn't this say something like...

By default, only the PHP file defined for the render field (if any) in the detected block.json get copied.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I see what you mean. It's hard to explain as it's one file per render, but it might be in multiple block.json files processed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

- `npm run start:custom-directory` - builds the code for production using the `custom-directory` as the source code directory.

This script automatically use the optimized config but sometimes you may want to specify some custom options:

- `--hot` – enables "Fast Refresh". The page will automatically reload if you make changes to the code. _For now, it requires that WordPress has the [`SCRIPT_DEBUG`](https://wordpress.org/support/article/debugging-in-wordpress/#script_debug) flag enabled and the [Gutenberg](https://wordpress.org/plugins/gutenberg/) plugin installed._
- `--webpack-bundle-analyzer` – enables visualization for the size of webpack output files with an interactive zoomable treemap.
- `--webpack-copy-php` – enables copying PHP files from the source directory ( default is `src` ) and its subfolders to the output directory.
- `--webpack-copy-php` – enables copying all PHP files from the source directory ( default is `src` ) and its subfolders to the output directory.
- `--webpack-devtool` – controls how source maps are generated. See options at https://webpack.js.org/configuration/devtool/#devtool.
- `--webpack-no-externals` – disables scripts' assets generation, and omits the list of default externals.
- `--webpack-src-dir` – Allows customization of the source code directory. Default is `src`.
Expand Down
19 changes: 15 additions & 4 deletions packages/scripts/config/webpack.config.js
Expand Up @@ -26,6 +26,7 @@ const {
hasCssnanoConfig,
hasPostCSSConfig,
getWebpackEntryPoints,
getRenderPropPaths,
} = require( '../utils' );

const isProduction = process.env.NODE_ENV === 'production';
Expand All @@ -36,9 +37,8 @@ if ( ! browserslist.findConfig( '.' ) ) {
}
const hasReactFastRefresh = hasArgInCLI( '--hot' ) && ! isProduction;

const copyWebpackPatterns = process.env.WP_COPY_PHP_FILES_TO_DIST
? '**/{block.json,*.php}'
: '**/block.json';
// Get paths of the `render` props included in `block.json` files
const renderPaths = getRenderPropPaths();
luisherranz marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use wordpress-scripts in our build system, and it doesn't work anymore. The whole thing is more complex, but basically, there is what it does: https://github.com/loxK/wordpress-scripts-test

import wpWebpackConfig from '@wordpress/scripts/config/webpack.config.js'
import CopyWebpackPlugin from 'copy-webpack-plugin'
import {resolve} from "path";

export default {
    ...wpWebpackConfig,
    entry: './dir/file.js',
    output: {
        filename: 'file.js',
        path: resolve( process.cwd(), 'assets/js' ),
    },
    plugins: [
        ...(( wpWebpackConfig?.plugins || [] ).filter(plugin => ! (plugin instanceof CopyWebpackPlugin))),

    ]
}
» npm run build                               

> build
> webpack

[webpack-cli] Failed to load '/home/www/test/webpack.config.js' config
[webpack-cli] TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at new NodeError (node:internal/errors:387:5)
    at validateString (node:internal/validators:121:11)
    at Object.join (node:path:1172:7)
    at fromProjectRoot (/home/www/test/node_modules/@wordpress/scripts/utils/file.js:13:7)
    at hasProjectFile (/home/www/test/node_modules/@wordpress/scripts/utils/file.js:16:14)
    at getRenderPropPaths (/home/www/test/node_modules/@wordpress/scripts/utils/config.js:312:9)
    at Object.<anonymous> (/home/www/test/node_modules/@wordpress/scripts/config/webpack.config.js:41:21)
    at Module._compile (node:internal/modules/cjs/loader:1126:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
    at Module.load (node:internal/modules/cjs/loader:1004:32) {
  code: 'ERR_INVALID_ARG_TYPE'
}

Any clue on what to do to make it work again ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @loxK. I've just cloned your repository, done a npm install (using node 16) and npm run build, and it worked fine.

I'm not sure what may be going on. Could you please try removing the node_module folder and running npm install again? Or giving us more precise instructions to reproduce the problem?

Copy link
Contributor

@loxK loxK Sep 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to upgrade @wordpress/scripts to 24.1.0 by editing packages.json and npm i, in the repository the wordpress-scripts version is fixed to 24.0.0.

I upgraded it in the repository, git pull it and npm i

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha.

For some reason, process.env.WP_SRC_DIRECTORY is undefined in your configuration. I'll investigate why.

It wasn't failing before because you overwrote the entry points (which also access process.env.WP_SRC_DIRECTORY).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, it's because you're using "build": "webpack" and not "build": "wp-scripts build" in your package.json.

wp-scripts build initializes process.env.WP_SRC_DIRECTORY:

process.env.WP_SRC_DIRECTORY = hasArgInCLI( '--webpack-src-dir' )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, but I don't know the answer. You need to find a way to switch back to wp-scripts, or you'd have to take a look at its internals to understand what it's doing before it loads webpack to do that yourself. You're also welcome to open a PR if you think there's something broken inside wp-scripts 🙂

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn breaking change in a minor release version number. Hours of work ahead ...
That sucks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extending webpack is not covered with semver, especially when not used in conjunction with wp-scripts.

From the docs:

Future versions of this package may change what webpack and Babel plugins we bundle, default configs, etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, if you have needs that wp-scripts don't support yet and you think they'd be useful for a wide range of people, I encourage you to open an issue/PR to add those improvements to wp-scripts 🙂

I also encourage you to open a PR if you think there's something broken or that could be done in a better way inside wp-scripts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reference, @loxK opened a PR to change the way the webpack config access the value of process.env.WP_SRC_DIRECTORY:


const cssLoaders = [
{
Expand Down Expand Up @@ -232,7 +232,7 @@ const config = {
new CopyWebpackPlugin( {
patterns: [
{
from: copyWebpackPatterns,
from: '**/block.json',
context: process.env.WP_SRC_DIRECTORY,
noErrorOnMissing: true,
transform( content, absoluteFrom ) {
Expand Down Expand Up @@ -265,6 +265,17 @@ const config = {
return content;
},
},
{
from: '**/*.php',
context: process.env.WP_SRC_DIRECTORY,
noErrorOnMissing: true,
filter: ( filepath ) => {
return (
process.env.WP_COPY_PHP_FILES_TO_DIST ||
renderPaths.includes( filepath )
gziolo marked this conversation as resolved.
Show resolved Hide resolved
);
},
},
],
} ),
// The WP_BUNDLE_ANALYZER global variable enables a utility that represents
Expand Down
56 changes: 56 additions & 0 deletions packages/scripts/utils/config.js
Expand Up @@ -302,10 +302,66 @@ function getWebpackEntryPoints() {
};
}

/**
* Returns the list of paths included in the `render` props by scanning the `block.json` files.
*
* @return {Array} The list of all the `render` prop paths included in `block.json` files.
*/
function getRenderPropPaths() {
// Continue only if the source directory exists.
if ( ! hasProjectFile( process.env.WP_SRC_DIRECTORY ) ) {
return [];
}

// Checks whether any block metadata files can be detected in the defined source directory.
const blockMetadataFiles = glob(
`${ process.env.WP_SRC_DIRECTORY }/**/block.json`,
{
absolute: true,
}
);

const srcDirectory = fromProjectRoot( process.env.WP_SRC_DIRECTORY + sep );

const renderPaths = blockMetadataFiles.map( ( blockMetadataFile ) => {
const { render } = JSON.parse( readFileSync( blockMetadataFile ) );
if ( render && render.startsWith( 'file:' ) ) {
// Removes the `file:` prefix.
const filepath = join(
dirname( blockMetadataFile ),
render.replace( 'file:', '' )
);

// Takes the path without the file extension, and relative to the defined source directory.
if ( ! filepath.startsWith( srcDirectory ) ) {
log(
chalk.yellow(
`Skipping "${ render.replace(
'file:',
''
) }" listed in "${ blockMetadataFile.replace(
fromProjectRoot( sep ),
''
) }". File is located outside of the "${
process.env.WP_SRC_DIRECTORY
}" directory.`
)
);
return false;
}
return filepath;
}
return false;
} );

return renderPaths.filter( ( renderPath ) => renderPath );
}

module.exports = {
getJestOverrideConfigFile,
getWebpackArgs,
getWebpackEntryPoints,
getRenderPropPaths,
hasBabelConfig,
hasCssnanoConfig,
hasJestConfig,
Expand Down
2 changes: 2 additions & 0 deletions packages/scripts/utils/index.js
Expand Up @@ -14,6 +14,7 @@ const {
getJestOverrideConfigFile,
getWebpackArgs,
getWebpackEntryPoints,
getRenderPropPaths,
hasBabelConfig,
hasCssnanoConfig,
hasJestConfig,
Expand All @@ -34,6 +35,7 @@ module.exports = {
getPackageProp,
getWebpackArgs,
getWebpackEntryPoints,
getRenderPropPaths,
hasArgInCLI,
hasBabelConfig,
hasCssnanoConfig,
Expand Down