Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add experimental wildcard remotePatterns config for upstream images #36245

Merged
merged 38 commits into from May 5, 2022
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
184d1dd
Add support for wildcard `images.remotePatterns` config
styfle Apr 18, 2022
8d66c2c
Fix lint
styfle Apr 18, 2022
5bdae63
Run lint
styfle Apr 18, 2022
b80be6e
Add test and telemetry
styfle Apr 18, 2022
12c2ffa
Simplify remote pattern assetPrefix
styfle Apr 18, 2022
7f9d656
Fix test
styfle Apr 19, 2022
98f3738
Fix validation
styfle Apr 19, 2022
bbc9477
Merge branch 'canary' into 27925-add-remote-patterns
styfle Apr 19, 2022
0feee39
Add docs for `remotePatterns`
styfle Apr 20, 2022
00b0bbc
Fix bug
styfle Apr 21, 2022
022db8c
Handle case when remotePatterns is undefined
styfle Apr 21, 2022
c56637a
Add version to table
styfle Apr 21, 2022
442dbca
Merge branch 'canary' into 27925-add-remote-patterns
styfle Apr 28, 2022
853c58c
Move match-remote-patterns to shared
styfle May 2, 2022
9acea01
Fix lint
styfle May 2, 2022
2ff43da
Fix assetPrefix usage
styfle May 2, 2022
c374ffa
Change config to experimental
styfle May 3, 2022
342bb7d
Revert build config
styfle May 3, 2022
d649e27
Merge branch 'canary' into 27925-add-remote-patterns
styfle May 3, 2022
591d8c5
Add back the build config
styfle May 3, 2022
32263da
Revert check for NODE_ENV
styfle May 3, 2022
fa98923
Revert image-from-node-modules test
styfle May 3, 2022
e26fbab
Fix default value
styfle May 4, 2022
4be0719
Fix nullable check
styfle May 4, 2022
33e1385
Fix typo
styfle May 4, 2022
e685198
Revert missing "the" in docs
styfle May 4, 2022
eae786b
Merge branch 'canary' into 27925-add-remote-patterns
styfle May 4, 2022
81e387c
Update docs
styfle May 4, 2022
66e344a
Update experimental note
styfle May 4, 2022
7a087f1
Fix case when domains has wildcard even though it should be exact match
styfle May 4, 2022
55679a3
Deconstruct experimental from nextConfig
styfle May 4, 2022
2c1f723
Remove `.js` suffix from test
styfle May 4, 2022
8d9e8fa
Add runtime check for hostname missing
styfle May 5, 2022
163eae9
Validate remotePatterns config has hostname prop
styfle May 5, 2022
c2c9ebb
Validate double asterisks
styfle May 5, 2022
f67daa0
Update docs
styfle May 5, 2022
72db874
Add ts-ignore
styfle May 5, 2022
e20ce59
Merge branch 'canary' into 27925-add-remote-patterns
styfle May 5, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
56 changes: 55 additions & 1 deletion docs/api-reference/next/image.md
Expand Up @@ -16,6 +16,7 @@ description: Enable Image Optimization with the built-in Image component.

| Version | Changes |
| --------- | ----------------------------------------------------------------------------------------------------- |
| `v12.1.7` | Experimental `remotePatterns` configuration added. |
| `v12.1.1` | `style` prop added. Experimental[\*](#experimental-raw-layout-mode) support for `layout="raw"` added. |
| `v12.1.0` | `dangerouslyAllowSVG` and `contentSecurityPolicy` configuration added. |
| `v12.0.9` | `lazyRoot` prop added. |
Expand Down Expand Up @@ -313,9 +314,62 @@ Other properties on the `<Image />` component will be passed to the underlying

## Configuration Options

### Remote Patterns

> Note: The `remotePatterns` configuration is currently **experimental** and subject to change. Please use [`domains`](#domains) for production use cases.

To protect your application from malicious users, configuration is required in order to use external images. This ensures that only external images from your account can be served from the Next.js Image Optimization API. These external images can be configured with the `remotePatterns` property in your `next.config.js` file, as shown below:

```js
module.exports = {
experimental: {
images: {
remotePatterns: [
{
protocol: 'https',
hostname: 'example.com',
port: '',
pathname: '/account123/**',
},
],
},
},
}
```

> Note: The example above will ensure the `src` property of `next/image` must start with `https://example.com/account123/`. Any other protocol, hostname, port, or unmatched path will respond with 400 Bad Request.

Below is another example of the `remotePatterns` property in the `next.config.js` file:

```js
module.exports = {
experimental: {
images: {
remotePatterns: [
{
protocol: 'https',
hostname: '**.example.com',
},
],
},
},
}
```

> Note: The example above will ensure the `src` property of `next/image` must start with `https://img1.example.com` or `https://me.avatar.example.com` or any number of subdomains. Any other protocol or unmatched hostname will respond with 400 Bad Request.

Wildcard patterns can be used for both `pathname` and `hostname` and have the following syntax:

- `*` match a single path segment or subdomain
- `**` match any number of path segments or subdomains

### Domains

To protect your application from malicious users, you must define a list of image provider domains that you want to be served from the Next.js Image Optimization API. This is configured in with the `domains` property in your `next.config.js` file, as shown below:
Similar to [`remotePatterns`](#remote-patterns), the `domains` configuration can be used to provide a list of allowed hostnames for external images.

However, the `domains` configuration does not support wildcard pattern matching and it cannot restrict protocol, port, or pathname.

Below is an example of the `domains` property in the `next.config.js` file:

```js
module.exports = {
Expand Down
14 changes: 4 additions & 10 deletions docs/basic-features/image-optimization.md
Expand Up @@ -66,7 +66,7 @@ function Home() {

### Remote Images

To use a remote image, the `src` property should be a URL string, which can be [relative](#loaders) or [absolute](#domains). Because Next.js does not have access to remote files during the build process, you'll need to provide the [`width`](/docs/api-reference/next/image.md#width), [`height`](/docs/api-reference/next/image.md#height) and optional [`blurDataURL`](/docs/api-reference/next/image.md#blurdataurl) props manually:
To use a remote image, the `src` property should be a URL string, which can be [relative](#loaders) or [absolute](/docs/api-reference/next/image.md#domains). Because Next.js does not have access to remote files during the build process, you'll need to provide the [`width`](/docs/api-reference/next/image.md#width), [`height`](/docs/api-reference/next/image.md#height) and optional [`blurDataURL`](/docs/api-reference/next/image.md#blurdataurl) props manually:

```jsx
import Image from 'next/image'
Expand All @@ -93,15 +93,9 @@ export default function Home() {

Sometimes you may want to access a remote image, but still use the built-in Next.js Image Optimization API. To do this, leave the `loader` at its default setting and enter an absolute URL for the Image `src`.

To protect your application from malicious users, you must define a list of remote domains that you intend to access this way. This is configured in your `next.config.js` file, as shown below:
To protect your application from malicious users, you must define a list of remote hostnames you intend to allow remote access.

```js
module.exports = {
images: {
domains: ['example.com', 'example2.com'],
},
}
```
> Learn more about [`domains`](/docs/api-reference/next/image.md#domains) configuration.

### Loaders

Expand Down Expand Up @@ -207,7 +201,7 @@ For examples of the Image component used with the various fill modes, see the [I

## Configuration

The `next/image` component and Next.js Image Optimization API can be configured in the [`next.config.js` file](/docs/api-reference/next.config.js/introduction.md). These configurations allow you to [enable remote domains](/docs/api-reference/next/image.md#domains), [define custom image breakpoints](/docs/api-reference/next/image.md#device-sizes), [change caching behavior](/docs/api-reference/next/image.md#caching-behavior) and more.
The `next/image` component and Next.js Image Optimization API can be configured in the [`next.config.js` file](/docs/api-reference/next.config.js/introduction.md). These configurations allow you to [enable remote images](/docs/api-reference/next/image.md#domains), [define custom image breakpoints](/docs/api-reference/next/image.md#device-sizes), [change caching behavior](/docs/api-reference/next/image.md#caching-behavior) and more.

[**Read the full image configuration documentation for more information.**](/docs/api-reference/next/image.md#configuration-options)

Expand Down
2 changes: 2 additions & 0 deletions errors/invalid-images-config.md
Expand Up @@ -17,6 +17,8 @@ module.exports = {
imageSizes: [16, 32, 48, 64, 96, 128, 256, 384],
// limit of 50 domains values
domains: [],
// limit of 50 objects
remotePatterns: [],
// path prefix for Image Optimization API, useful with `loader`
path: '/_next/image',
// loader can be 'default', 'imgix', 'cloudinary', 'akamai', or 'custom'
Expand Down
2 changes: 2 additions & 0 deletions packages/next/build/index.ts
Expand Up @@ -2132,6 +2132,8 @@ export default async function build(
const images = { ...config.images }
const { deviceSizes, imageSizes } = images
;(images as any).sizes = [...deviceSizes, ...imageSizes]
;(images as any).remotePatterns =
config?.experimental?.images?.remotePatterns || []

await promises.writeFile(
path.join(distDir, IMAGES_MANIFEST),
Expand Down
2 changes: 2 additions & 0 deletions packages/next/build/webpack-config.ts
Expand Up @@ -1467,6 +1467,8 @@ export default async function getBaseWebpackConfig(
? {
// pass domains in development to allow validating on the client
domains: config.images.domains,
experimentalRemotePatterns:
config.experimental?.images?.remotePatterns,
}
: {}),
}),
Expand Down
26 changes: 15 additions & 11 deletions packages/next/client/image.tsx
Expand Up @@ -18,8 +18,8 @@ import { ImageConfigContext } from '../shared/lib/image-config-context'
import { warnOnce } from '../shared/lib/utils'
import { normalizePathTrailingSlash } from './normalize-trailing-slash'

const experimentalLayoutRaw = (process.env.__NEXT_IMAGE_OPTS as any)
?.experimentalLayoutRaw
const { experimentalLayoutRaw = false, experimentalRemotePatterns = [] } =
(process.env.__NEXT_IMAGE_OPTS as any) || {}
const configEnv = process.env.__NEXT_IMAGE_OPTS as any as ImageConfigComplete
const loadedImageURLs = new Set<string>()
const allImgs = new Map<
Expand Down Expand Up @@ -1063,7 +1063,10 @@ function defaultLoader({
)
}

if (!src.startsWith('/') && config.domains) {
if (
!src.startsWith('/') &&
(config.domains || experimentalRemotePatterns)
) {
let parsedSrc: URL
try {
parsedSrc = new URL(src)
Expand All @@ -1074,14 +1077,15 @@ function defaultLoader({
)
}

if (
process.env.NODE_ENV !== 'test' &&
!config.domains.includes(parsedSrc.hostname)
) {
throw new Error(
`Invalid src prop (${src}) on \`next/image\`, hostname "${parsedSrc.hostname}" is not configured under images in your \`next.config.js\`\n` +
`See more info: https://nextjs.org/docs/messages/next-image-unconfigured-host`
)
if (process.env.NODE_ENV !== 'test') {
// We use dynamic require because this should only error in development
const { hasMatch } = require('../shared/lib/match-remote-pattern')
if (!hasMatch(config.domains, experimentalRemotePatterns, parsedSrc)) {
throw new Error(
`Invalid src prop (${src}) on \`next/image\`, hostname "${parsedSrc.hostname}" is not configured under images in your \`next.config.js\`\n` +
`See more info: https://nextjs.org/docs/messages/next-image-unconfigured-host`
)
}
}
}
}
Expand Down
3 changes: 3 additions & 0 deletions packages/next/server/config-shared.ts
Expand Up @@ -5,6 +5,7 @@ import {
ImageConfig,
ImageConfigComplete,
imageConfigDefault,
RemotePattern,
} from '../shared/lib/image-config'

export type PageRuntime = 'nodejs' | 'edge' | undefined
Expand Down Expand Up @@ -115,6 +116,7 @@ export interface ExperimentalConfig {
outputStandalone?: boolean
images?: {
layoutRaw: boolean
remotePatterns: RemotePattern[]
}
middlewareSourceMaps?: boolean
emotion?:
Expand Down Expand Up @@ -501,6 +503,7 @@ export const defaultConfig: NextConfig = {
outputStandalone: !!process.env.NEXT_PRIVATE_STANDALONE,
images: {
layoutRaw: false,
remotePatterns: [],
},
},
}
Expand Down
34 changes: 34 additions & 0 deletions packages/next/server/config.ts
Expand Up @@ -234,6 +234,40 @@ function assignDefaults(userConfig: { [key: string]: any }) {
)
}
}

const remotePatterns = result.experimental?.images?.remotePatterns
if (remotePatterns) {
if (!Array.isArray(remotePatterns)) {
throw new Error(
`Specified images.remotePatterns should be an Array received ${typeof remotePatterns}.\nSee more info here: https://nextjs.org/docs/messages/invalid-images-config`
)
}

if (remotePatterns.length > 50) {
throw new Error(
`Specified images.remotePatterns exceeds length of 50, received length (${remotePatterns.length}), please reduce the length of the array to continue.\nSee more info here: https://nextjs.org/docs/messages/invalid-images-config`
)
}

const validProps = new Set(['protocol', 'hostname', 'pathname', 'port'])
const invalidIndex = remotePatterns.findIndex(
ijjk marked this conversation as resolved.
Show resolved Hide resolved
(d: unknown) =>
!d ||
typeof d !== 'object' ||
Object.entries(d).some(
([k, v]) => !validProps.has(k) || typeof v !== 'string'
)
)
const invalid = remotePatterns[invalidIndex]
if (invalid) {
throw new Error(
`Specified images.remotePatterns[${invalidIndex}] should be RemotePattern object received invalid value (${JSON.stringify(
invalid
)}).\nSee more info here: https://nextjs.org/docs/messages/invalid-images-config`
)
}
}

if (images.deviceSizes) {
const { deviceSizes } = images
if (!Array.isArray(deviceSizes)) {
Expand Down
4 changes: 3 additions & 1 deletion packages/next/server/image-optimizer.ts
Expand Up @@ -17,6 +17,7 @@ import chalk from 'next/dist/compiled/chalk'
import { NextUrlWithParsedQuery } from './request-meta'
import { IncrementalCacheEntry, IncrementalCacheValue } from './response-cache'
import { mockRequest } from './lib/mock-request'
import { hasMatch } from '../shared/lib/match-remote-pattern'

type XCacheHeader = 'MISS' | 'HIT' | 'STALE'

Expand Down Expand Up @@ -75,6 +76,7 @@ export class ImageOptimizerCache {
minimumCacheTTL = 60,
formats = ['image/webp'],
} = imageData
const remotePatterns = nextConfig.experimental.images?.remotePatterns || []
const { url, w, q } = query
let href: string

Expand Down Expand Up @@ -104,7 +106,7 @@ export class ImageOptimizerCache {
return { errorMessage: '"url" parameter is invalid' }
}

if (!domains || !domains.includes(hrefParsed.hostname)) {
if (!hasMatch(domains, remotePatterns, hrefParsed)) {
return { errorMessage: '"url" parameter is not allowed' }
}
}
Expand Down
31 changes: 30 additions & 1 deletion packages/next/shared/lib/image-config.ts
Expand Up @@ -8,6 +8,33 @@ export const VALID_LOADERS = [

export type LoaderValue = typeof VALID_LOADERS[number]

export type RemotePattern = {
/**
* Must be `http` or `https`.
*/
protocol?: 'http' | 'https'

/**
* Can be literal or wildcard.
* Single `*` matches a single subdomain.
* Double `**` matches any number of subdomains.
*/
hostname: string

/**
* Can be literal port such as `8080` or empty string
* meaning no port.
*/
port?: string

/**
* Can be literal or wildcard.
* Single `*` matches a single path segment.
* Double `**` matches any number of path segments.
*/
pathname?: string
}

type ImageFormat = 'image/avif' | 'image/webp'

/**
Expand All @@ -28,7 +55,9 @@ export type ImageConfigComplete = {
/** @see [Image loader configuration](https://nextjs.org/docs/api-reference/next/image#loader-configuration) */
path: string

/** @see [Image domains configuration](https://nextjs.org/docs/basic-features/image-optimization#domains) */
/**
* @see [Image domains configuration](https://nextjs.org/docs/api-reference/next/image#domains)
*/
domains: string[]

/** @see [Cache behavior](https://nextjs.org/docs/api-reference/next/image#caching-behavior) */
Expand Down
68 changes: 68 additions & 0 deletions packages/next/shared/lib/match-remote-pattern.ts
@@ -0,0 +1,68 @@
import type { RemotePattern } from './image-config'

export function matchRemotePattern(pattern: RemotePattern, url: URL): boolean {
if (pattern.protocol !== undefined) {
const actualProto = url.protocol.slice(0, -1)
if (pattern.protocol !== actualProto) {
return false
}
}
if (pattern.port !== undefined) {
if (pattern.port !== url.port) {
return false
}
}
if (pattern.pathname !== undefined) {
const patternParts = pattern.pathname.split('/')
const actualParts = url.pathname.split('/')
const len = Math.max(patternParts.length, actualParts.length)
for (let i = 0; i < len; i++) {
if (patternParts[i] === '**' && actualParts[i] !== undefined) {
ijjk marked this conversation as resolved.
Show resolved Hide resolved
// Double asterisk means "match everything until the end of the path"
// so we can break the loop early
break
}
if (patternParts[i] === '*') {
// Single asterisk means "match this part" so we can
// continue to the next part of the loop
continue
}
if (patternParts[i] !== actualParts[i]) {
ijjk marked this conversation as resolved.
Show resolved Hide resolved
return false
}
}
}

if (pattern.hostname !== undefined) {
ijjk marked this conversation as resolved.
Show resolved Hide resolved
const patternParts = pattern.hostname.split('.').reverse()
const actualParts = url.hostname.split('.').reverse()
const len = Math.max(patternParts.length, actualParts.length)
for (let i = 0; i < len; i++) {
if (patternParts[i] === '**' && actualParts[i] !== undefined) {
ijjk marked this conversation as resolved.
Show resolved Hide resolved
// Double asterisk means "match every subdomain"
// so we can break the loop early
break
}
if (patternParts[i] === '*') {
// Single asterisk means "match this subdomain" so we can
// continue to the next part of the loop
continue
}
if (patternParts[i] !== actualParts[i]) {
styfle marked this conversation as resolved.
Show resolved Hide resolved
return false
}
}
}
return true
}

export function hasMatch(
domains: string[],
remotePatterns: RemotePattern[],
url: URL
): boolean {
return (
domains.some((domain) => url.hostname === domain) ||
remotePatterns.some((p) => matchRemotePattern(p, url))
)
}