Support config.includeFiles for @now/next

[BrunoBernardino]

There's a message/thread in Spectrum about this.

Basically I can't fs.readFileSync() inside an pages/api/whatever.ts because the files aren't copied.

I've done some investigation in the code for the builders, and I don't see any parity for the functionality that exists in @now/node. I imagine something should be added around here for that, and am happy to try a PR if you don't think this is too crazy.

Here's the info I posted in Spectrum:

I'm having this same issue with @now/next, so I have this in a now.json file:

{
  "version": 2,
  "builds": [
    {
      "src": "next.config.js",
      "use": "@now/next",
      "config": {
        "includeFiles": ["email-templates/**"]
      }
    }
  ]
}

Where a directory with this structure exists in the root of the app/directory:

email-templates
├── login
│   ├── html.pug
│   └── subject.pug
├── main.pug
└── mixins.pug

And I can't read the files at all (I've tried all kinds of combinations to just find the files, and here's the latest):

const templateFilePath = path.join(__dirname, 'email-templates', 'login', 'html.pug');
const templateContent = fs.readFileSync(templateFilePath).toString('utf8');

The error I get is:

Unhandled Promise Rejection    {"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: ENOENT: no such file or directory, open '/email-templates/login/html.pug'","stack":["Runtime.UnhandledPromiseRejection: Error: ENOENT: no such file or directory, open '/email-templates/login/html.pug'","    at process.on (/var/runtime/index.js:37:15)","    at process.emit (events.js:203:15)","    at emitPromiseRejectionWarnings (internal/process/promises.js:119:20)","    at process._tickCallback (internal/process/next_tick.js:69:34)"],"reason":{"errorType":"Error","errorMessage":"ENOENT: no such file or directory, open '/email-templates/login/html.pug'","code":"ENOENT","stack":["Error: ENOENT: no such file or directory, open '/email-templates/login/html.pug'","    at Object.openSync (fs.js:443:3)","    at Object.readFileSync (fs.js:343:35)","    at Object.sendLoginEmail (/var/task/.next/serverless/pages/api/v0/signup.js:564:71)","    at module.exports.L76w.__webpack_exports__.default (/var/task/.next/serverless/pages/api/v0/signup.js:471:67)","    at process._tickCallback (internal/process/next_tick.js:68:7)"],"errno":-2,"syscall":"open","path":"/email-templates/login/html.pug"},"promise":{}}
2019-09-26T14:35:19.168Z    bea5a417-7224-4397-b4ec-81b6838540a2    ERROR    Unhandled rejection: { Error: ENOENT: no such file or directory, open '/email-templates/login/html.pug'
    at Object.openSync (fs.js:443:3)
    at Object.readFileSync (fs.js:343:35)
    at Object.sendLoginEmail (/var/task/.next/serverless/pages/api/v0/signup.js:564:71)
    at module.exports.L76w.__webpack_exports__.default (/var/task/.next/serverless/pages/api/v0/signup.js:471:67)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/email-templates/login/html.pug' }

And while the path looks weird (I'd expect __dirname to be /var/task, not /), it also doesn't work if I try to read /var/task/email-templates/login/html.pug.

Thanks!

[styfle]

Hi @BrunoBernardino

The __dirname is going to be the current directory of your serverless function.

If you have it in /pages/api/index.js and your templates are /email-templates, then you'll need to go up a couple directories like the following:

Is your

const { readFileSync } = require('fs');
const { join } = require('path');
const content = readFileSync(join(__dirname, '..', '..', 'email-templates', 'login', 'html.pug'), 'utf8');

[BrunoBernardino]

Thanks @styfle !

The file that does the reading is at /lib/email.ts and it's imported + called by /pages/api/v0/signup.ts. Indeed the file I'm trying to read is at /email-templates/login/html.pug.

None of the following paths worked (and readdirSyncing them showed there's nothing there to find indeed):

• path.join(__dirname, 'email-templates', 'login', 'html.pug')
• path.join(__dirname, '..', 'email-templates', 'login', 'html.pug')
• path.join(__dirname, '..', '..', 'email-templates', 'login', 'html.pug')
• path.join(__dirname, '..', '..', '..', 'email-templates', 'login', 'html.pug')
• path.join('email-templates', 'login', 'html.pug')
• path.join('..', 'email-templates', 'login', 'html.pug')
• path.join('..', '..', 'email-templates', 'login', 'html.pug')
• path.join('..', '..', '..', 'email-templates', 'login', 'html.pug')

Do you think it might be that it's not a .js file I'm reading?

NOTE: The fifth version works without problems with $ next and $ now dev.

[styfle]

Hi @BrunoBernardino

Thanks for the detailed feedback!

There is a fix in this PR vercel/next.js#8334

[BrunoBernardino]

@styfle Thanks for mentioning that! It seems that'll solve the __dirname being mapped to /, but not the fact that the files I'm trying to read are not copied (they're not bundled with the lambda), or am I missing something?

In any case, is there an easy way for me to try that (can I just refer the branch from the PR as the next dependency version)?

[styfle]

The PR fixes more than the __dirname being correct; it will also make sure files are included in the build output so they’re accessible at runtime.

[BrunoBernardino]

Thanks for the clarification!

So while technically the problem will be solved, the builder still won't support config.includeFiles for files not "recognized" at build time. If that works I'd assume it would make sense to keep this issue open, but maybe as a feature request instead of a bug.

[gunar]

I just realized require('./file.json') works fine, so we just need to generate a json file at build time:

1. Create a script that reads whatever files you need, and exports them as json
2. Add this to package.json: "build": "node script.js>data.json && next build"
3. Have your serverless function require the json as such require('../../data.json')

In my case it was even simpler. The output was static so I just wrote it directly into /public.

Hope this helps!

[Timer]

This bug has been fixed in the latest Next.js builder version, so this feature request should be obsolete.

[BrunoBernardino]

@Timer

So while technically the problem has been solved, the builder still won't support config.includeFiles for files not "recognized" at build time, right?

That's what this feature request is about since October 2019. I hope that makes sense, and if that's been solved, you can go ahead and close this!

[Timer]

The current plan is still to not support config.includeFiles in the Next.js builder.

Any cases where Next.js does not automatically pick up on file usage should be considered a bug and fixed accordingly.

Per vercel/next.js#8251, all file reading must currently be done relative to the CWD (this is a Next.js itself limitation):

path.resolve(process.cwd(), 'myfile.txt')

[BrunoBernardino]

Thanks! If I run into these issues again in the future I'll add more info.