Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Serverless #274

Closed
Nelrohd opened this issue Apr 23, 2019 · 216 comments
Closed

Support for Serverless #274

Nelrohd opened this issue Apr 23, 2019 · 216 comments
Labels
enhancement New feature or request

Comments

@Nelrohd
Copy link

Nelrohd commented Apr 23, 2019

Is your feature request related to a problem? Please describe.

Currently next-i18next does not support serverless. It appears that the i18n-middleware create some troubles when used on lambda #271

Describe the solution you'd like

Not sure exactly what would be the final solution but:

  • Support i18n without needing custom server.
  • Support with custom server like express (note I didn't find an official repo for NextJS V8 serverless + express).

Describe alternatives you've considered

None.

Additional context

Open to suggestion for the work/final solution to be done/achieved.

@isaachinman
Copy link
Contributor

A few things to consider - just my initial thoughts:

  • Where would locales/translated content live? Seems we are going to have problems with a filesystem approach
  • How would we support locale subpaths, if at all?
  • How would we replace all the functionality that i18next-express-middleware gives us? See nextI18NextMiddleware for Koa? #9 (comment)

I am open to any/all suggestions on this topic, but in general regard it as a major refactor of the project from the ground up. If we can remove our dependency on Express/i18next-express-middleware, I think the package would become a lot more versatile.

However, I am not (currently) doing any work whatsoever with serverless NextJs, and have very little input to add here.

@beheh
Copy link

beheh commented Jun 18, 2019

I've gotten next-i18next working in a serverless environment for one of our sites we're building right now (no full source available unfortunately), but it's pretty hacky because we're basically stitching together various parts from next-i18next in a more vanilla i18next setup.

Our approach is that we basically ship the JSON files once to a CDN (with an assetPrefix) and once inside the Lambda "executables". We reference them via import which makes them real Webpack chunks for the CDN and also allows it to pick them up and include them when packaging the serverless file and we're also running express infront of the bundles so the middleware can just be dropped. I can't really recommend this approach, but it can definitely work.

@isaachinman
Copy link
Contributor

@beheh Sounds like you had some fun. Are there any lessons or approaches you'd like to bring back to next-i18next? Being able to support serverless is the eventual goal.

@yoeran
Copy link

yoeran commented Jun 21, 2019

Stumbled upon this issue because I'm researching for a new NextJS project that should have i18n and is planned to be deployed to Now 2.0. I've not used next-i18next yet, nor am I a serverless/now expert, but if someone has some smart ideas I'm willing to help with the implementation.

@isaachinman
Copy link
Contributor

@yoeran I think the right place to start is rewriting/adapting i18next-express-middleware to use vanilla NodeJs HTTP interfaces instead of relying on Express.

@flearsnetwork
Copy link

Would also appreciate it if someone has the knowledge to optimize the package for serverless solutions.

@gurkerl83
Copy link

[RFC] serverless is still open (vercel/next.js#7208). The last entry mentions that a routing middleware is still missing in the current version of Next. Tracking the progress there is a merge request with the title "Example for API routes with middleware" https://github.com/zeit/next.js/pull/7958/files . I do not know anything about the internal of the express middleware currently utilized by next-i18next, but maybe the example shown in the merge request is enough to start a migration.

@isaachinman
Copy link
Contributor

@gurkerl83 That example appears to show how one could use middleware on an API route, not a UI route.

@emirotin
Copy link

emirotin commented Jul 25, 2019

My request sounds different from the original title but I assume it's related cause it's also about reusing the module outside of next execution context. Sorry if I'm wrong though.

We have a successfully working Next.js project using your next-i18next (thanks!).

Now I also want to create a stand-alone node.js script that would reuse the entire infrastructure we already have and do some interpolations (in the static HTML files if you're curious).

For some reason (I have my brain kinda broken after hours of trying) it doesn't work.

Here's what I'm doing:

const nextConfig = require('next/config');
nextConfig.setConfig(require('./next.config'));

const { i18n } = require('./i18n');

And then trying to use i18n.t directly. I see that it's getting all the proper configs but does not load the data from the JSON files.

And thus an attempt to translate any key returns the key itself.

I'm trying to figure out what else next is doing behind the scene that makes your module fetch the needed JSON data and that I'm missing.


For ref, ./i18n is pretty standard:

const NextI18Next = require('next-i18next').default;
const i18nHelper = require('./lib/i18nHelper');

module.exports = new NextI18Next({
    localePath: 'static-src/locales',
    defaultLanguage: i18nHelper.getDefaultLanguage(),
    otherLanguages: i18nHelper.getOtherLanguages(),
    fallbackLng: i18nHelper.getDefaultLanguage(),
    defaultNS: i18nHelper.getDefaultNamespace(),
    fallbackNS: i18nHelper.getAllNamespaces(),
    ns: i18nHelper.getAllNamespaces(),
    localeSubpaths: 'foreign'
});

And I can confirm that all the i18nHelper methods are returning what expected.

@isaachinman
Copy link
Contributor

@emirotin That's not something we're going to support. Unfortunately you'll need to go it alone. I would suggest simply initialising a normal i18next instance. Note that you can pass your identical next-i18next config object into the i18next.init function.

Let's keep this discussion on topic.

@emirotin
Copy link

emirotin commented Jul 25, 2019

Sorry for offtopic then. I truly assumed it's about the same root.
Just curious what's wrong with my current setup that makes it unusable?

P.S. barebones i18next doesn't work for me (yet) too normal i18next works!

@cryptiklemur
Copy link

cryptiklemur commented Jul 27, 2019

It seems possible to spoof this with the following in a custom _app.js

import {handle} from 'i18next-express-middleware';
import {i18n, config} from '../i18n';

export default extends App {
    // ...    
    static async getInitialProps(context) {
        if (context.ctx.req && context.ctx.res) {
            res.set = res.setHeader;
            req.path = context.ctx.asPath;

            const i18nHandler = handle(i18n, {ignoreRoutes: config.ignoreRoutes});
            await new Promise((resolve, reject) => {
                try {
                    i18nHandler(req as any, res as any, resolve);
                } catch (e) {
                    reject(e);
                }
            });
        }
    }
    // ...    
}

Hopefully ZEIT follows through with adding middleware capabilities, and we can easily add this that way

@cryptiklemur
Copy link

If this is something you are cool with (and actually works? I'm still testing this), I can throw up a PR to introduce a helper function (plus some documentation)

@isaachinman
Copy link
Contributor

@aequasi Not sure that'd work. The req and res objects that come off Next's context are vanilla HTTP interfaces, not Express req and res interfaces. What have you seen in your testing?

Because there are so many diverse use cases for this package, I'd rather we proceed in the least hacky way possible. We do have support from the Next team and they're aware of the request for serverless middleware.

@cryptiklemur
Copy link

The two express specific things (set and path) can easily be "mocked"

set actually needs to be:

req.set = (key, value) => {
    if (value !== undefined) {
        res.setHeader(key, value);
    }
}

@cryptiklemur
Copy link

From my testing, this works. I'm not having any issues with server-rendered content

@isaachinman
Copy link
Contributor

To reiterate, I'd rather we proceed in the least hacky way possible. This means we need to:

  1. Wait for serverless middleware support within NextJs
  2. Fork i18next-express-middleware and refactor out the Express dep, and release a new package, eg i18next-node-middleware
  3. Refactor next-i18next accordingly

If people want to roll their own serverless solutions in the meantime, feel free.

@samohovets
Copy link

samohovets commented Jul 30, 2019

@aequasi nice temporary solution 🤔 But I guess locale subpaths is not working, right?

UPD: yes, server-side locale subpaths is not working (naturally), but it seems fine on the client. I got client-side translations working so far without particularly critical bugs, but it can't work on Zeit Now serverless 🤔

Monkey patching all incoming requests in _app.jsx is not a best way to get things working, so as @isaachinman mentioned - we need to wait for serverless middlware support in Next.js 😞

image

@cryptiklemur
Copy link

Zeit compiles everything that it can find from static analysis into a single file. If i had to make an educated guess, the locales are not making it to the deployment, causing the fs.readdirSync to fail. You could look into: https://zeit.co/docs/v2/advanced/builders/overview#including-additional-files

@gurkerl83
Copy link

@aequasi I experienced the same error when a deployment with now get executed. With your last comment, do you mean specifying i18n-resource files within the includeFiles section of now deployment configuration (now.json) will resolve the fs.readdirSync error?

@isaachinman
Copy link
Contributor

This comment: vercel/next.js#13624 (comment) potentially sheds some light on how to get Vercel to pick up filesystem deps.

@adrai I'm back to running into i18next module resolution issues:

2020-07-08T09:30:10.473Z	4e8d3e9a-1479-4f9d-b650-e1ab625ea951	ERROR	Unhandled error during request: Error: Cannot find module 'i18next-fs-backend/cjs'
Require stack:
- /var/task/node_modules/next-i18next/dist/commonjs/create-i18next-client.js
- /var/task/node_modules/next-i18next/dist/commonjs/index.js

Fairly sure this was previously working. Any ideas?

@adrai
Copy link
Member

adrai commented Jul 8, 2020

I'm back to running into i18next module resolution issues:

strange... is this reproducable with the example in the repo? Can't reproduce it.

@isaachinman
Copy link
Contributor

@adrai I have been deploying this repo to Vercel. Feel free to fork and deploy. This issue isn't reproducible locally – it's definitely something to do with their strict module resolution.

@borispoehland
Copy link

borispoehland commented Jul 8, 2020

I'm back to running into i18next module resolution issues:

strange... is this reproducable with the example in the repo? Can't reproduce it.

It only occurs when deploying to vercel. Here is a project that basically runs minimal configuration and has the filepath already included: https://github.com/borispoehland/next-i18next-boilerplate

Edit: Take Isaac's version

@adrai
Copy link
Member

adrai commented Jul 8, 2020

It only occurs when deploying to vercel.

Then they changed something again... 🤷‍♂️
Maybe now requiring /cjs is not needed anymore?

@adrai
Copy link
Member

adrai commented Jul 8, 2020

Seems eval('require...') is ignored by their bundler.... there are also others having this problem: saltyshiomix/nextron#78
vercel/next.js#13736

@adrai
Copy link
Member

adrai commented Jul 8, 2020

Probably adding something like this would force to bundle it:

const path = require('path');
path.resolve('./node_modules/i18next-fs-backend/')

🤷‍♂️

@borispoehland
Copy link

Probably adding something like this would force to bundle it:

const path = require('path');
path.resolve('./node_modules/i18next-fs-backend/')

🤷‍♂️

Sorry for the newbie question, but what stops you from importing the modules with the regular import syntax (instead of the eval(require...))?
Because with require they are handled as node modules?

@adrai
Copy link
Member

adrai commented Jul 8, 2020

I imagine importing them would bundle these dependencies also for the client...
Alternatively a dynamic import could work... but then that function would have been rewritten to be async...
@isaachinman is this correct?

@isaachinman
Copy link
Contributor

isaachinman commented Jul 8, 2020

Guess what? I just deployed a serverless next-i18next project successfully on Vercel:

https://next-i18next-vercel.now.sh/

Can we get some early adopters, of users both:

  1. Deploying serverless projects with next-i18next@beta
  2. Deploying custom server (Express) projects with next-i18next@beta

We want to make sure that both options are supported well and free of any major bugs.

@borispoehland
Copy link

Yeah, that suggestion works @borispoehland. The eval usage there is a nasty hack to exclude those deps from client bundles. I've always been open to other solutions, but no one has come forward.

Guess what? I just deployed a serverless next-i18next project successfully on Vercel:

https://next-i18next-vercel.now.sh/

Can we get some early adopters, of users both:

  1. Deploying serverless projects with next-i18next@beta
  2. Deploying custom server (Express) projects with next-i18next@beta

We want to make sure that both options are supported well and free of any major bugs.

I thought custom server deploys to vercel are not possible, which was the reason for creating the serverless beta in the first place?

@isaachinman
Copy link
Contributor

I thought custom server deploys to vercel are not possible, which was the reason for creating the serverless beta in the first place?

I wasn't talking about Vercel. This project needs to support deployment on various platforms, and needs to maintain support for users that rely on custom servers.

@borispoehland
Copy link

I thought custom server deploys to vercel are not possible, which was the reason for creating the serverless beta in the first place?

I wasn't talking about Vercel. This project needs to support deployment on various platforms, and needs to maintain support for users that rely on custom servers.

I understand

@borispoehland
Copy link

@isaachinman the updated beta is not pushed yet, right?

@isaachinman
Copy link
Contributor

@borispoehland What updates to the beta release are you expecting? I'm not convinced that platform-specific hacks like using path.resolve belong in the next-i18next source.

@iduuck
Copy link

iduuck commented Jul 8, 2020

Are you able to share the source code for how you solved the issue? The path.resolve('./node_modules/i18next-fs-backend/') "hack" does not tell me a lot.

@isaachinman
Copy link
Contributor

The source code is here: https://github.com/isaachinman/next-i18next-vercel

@borispoehland
Copy link

@borispoehland What updates to the beta release are you expecting? I'm not convinced that platform-specific hacks like using path.resolve belong in the next-i18next source.

Your're absolutely right. Better put it in the i18n.js.

@isaachinman
Copy link
Contributor

isaachinman commented Jul 8, 2020

Update: I've spoken to Joe in the NextJs team, and we've identified fixes for both the locales dir path, and i18next-fs-backend. I will be updating next-i18next@beta such that no path.resolve tricks will be necessary at all – should have this done by the end of the week.

@adrai
Copy link
Member

adrai commented Jul 8, 2020

Update: I've spoken to Joe in the NextJs team, and we've identified fixes for both the locales dir path, and i18next-fs-backend. I will be updating next-i18next@beta such that no path.resolve tricks will be necessary at all – should have this done by the end of the week.

Cool, let me know if there is something that need to be done for other i18next modules...

@isaachinman
Copy link
Contributor

isaachinman commented Jul 9, 2020

So, I’ve undertaken the following changes:

  1. Refactor localePath to be an absolute path (requirement), which users can pass via path.resolve. This is to get around bundling technologies used by Vercel, and other serverless platforms. The way that path.resolve falls back in browser envs is critical.
  2. Refactor the create-client code to have two entry points: one for Node, and one for browsers. This way we can stop using eval hacks, and allow tools to properly analyse our dependencies (fixes the various i18next dep issues).

Changes here: 826a9a5.

I then released next-i18next@5.0.0-beta.3 and updated my example repo:

https://github.com/isaachinman/next-i18next-vercel

It's running smoothly on Vercel here:

https://next-i18next-vercel.now.sh/

So: we can now release next-i18next projects on Vercel (and other serverless platforms) without any compromises or hacks, straight out of the box. The main breaking change will be the requirement of path.resolve for localePath.

Please do let me know if anyone has any questions, or if any of you are able to beta test this release. I've been told by the NextJs team that rewrites will be stable very soon, so in the very near future this functionality will be available without the experimental flag.

I am going to (finally) close this issue in the near future, as serverless support is genuinely here. If anyone encounters bugs/problems with the serverless beta release, please open new issues as you normally would.

Thanks everyone!

@Progressandro
Copy link

Progressandro commented Jul 11, 2020

So, I’ve undertaken the following changes:

  1. Refactor localePath to be an absolute path (requirement), which users can pass via path.resolve. This is to get around bundling technologies used by Vercel, and other serverless platforms. The way that path.resolve falls back in browser envs is critical.
  2. Refactor the create-client code to have two entry points: one for Node, and one for browsers. This way we can stop using eval hacks, and allow tools to properly analyse our dependencies (fixes the various i18next dep issues).

Changes here: 826a9a5.

I then released next-i18next@5.0.0-beta.3 and updated my example repo:

https://github.com/isaachinman/next-i18next-vercel

It's running smoothly on Vercel here:

https://next-i18next-vercel.now.sh/

So: we can now release next-i18next projects on Vercel (and other serverless platforms) without any compromises or hacks, straight out of the box. The main breaking change will be the requirement of path.resolve for localePath.

Please do let me know if anyone has any questions, or if any of you are able to beta test this release. I've been told by the NextJs team that rewrites will be stable very soon, so in the very near future this functionality will be available without the experimental flag.

I am going to (finally) close this issue in the near future, as serverless support is genuinely here. If anyone encounters bugs/problems with the serverless beta release, please open new issues as you normally would.

Thanks everyone!

Hey, thanks for coming up with an equally straightforward solution for the serverless option as well.

I'm trying to implement the update into my new project, however, I'm finding the following issue when importing the i18n file into any page-level component:

Error was not caught TypeError: path.resolve is not a function

I'm not a veteran with Next.js, but I don't see any obvious problem with my code.

my i18n.js:

const path = require('path')
const NextI18Next = require('next-i18next').default
const { localeSubpaths } = require('next/config').default().publicRuntimeConfig
module.exports = new NextI18Next({
  defaultLanguage: 'en',
  otherLanguages: ['es'],
  localeSubpaths,
  localePath: path.resolve('./public/locales'),
})

My "/test" page-level component:

import React from 'react'
import PropTypes from 'prop-types'
import { i18n, withTranslation } from '../i18n'
const Test = ({ t, namespacesRequired }) => {
  return (
    <React.Fragment>
      <main>
        <div>
          <button
            type="button"
            onClick={() => {
              console.log(i18n)
              i18n.changeLanguage(i18n.language === 'en' ? 'de' : 'en')
            }}
          >
            Change
          </button>
          <h1>{t('title')}</h1>
        </div>
      </main>
    </React.Fragment>
  )
}

Test.getInitialProps = async () => ({
  namespacesRequired: ['common'],
})

Test.propTypes = {
  t: PropTypes.func.isRequired,
}

export default withTranslation('common')(Test)

My next.config.js:

const withPlugins = require('next-compose-plugins');
const withOptimizedImages = require('next-optimized-images');
const withFonts = require('next-fonts');
const FilterWarningsPlugin = require('webpack-filter-warnings-plugin');
const { nextI18NextRewrites } = require('next-i18next/rewrites')

const localeSubpaths = {
  es: 'es'
}

const nextConfig = {
  env: {
    GOOGLE_API_KEY: 'API_KEY',
    REACT_APP_GOOGLE_MAP_API_KEY:
      'https://maps.googleapis.com/maps/api/js?v=3.exp&key=API_KEY&libraries=geometry,drawing,places',
    SERVER_API: `http://localhost:3001`,
  },
  webpack: (config, { isServer }) => {
    config.plugins.push(
      new FilterWarningsPlugin({
        exclude: /mini-css-extract-plugin[^]*Conflicting order between:/,
      })
    );
    config.resolve.modules.push(__dirname);

    return config;
  },
  publicRuntimeConfig: {
    localeSubpaths,
  },
  experimental: {
    async rewrites() {
      return [
        ...nextI18NextRewrites(localeSubpaths)
      ]
    }
  }
};

module.exports = withPlugins(
  [
    [
      withOptimizedImages,
      {
        mozjpeg: {
          quality: 90,
        },
        webp: {
          preset: 'default',
          quality: 90,
        },
      },
    ],
    withFonts,
  ],
  nextConfig
);

If you have any other question, I'll be glad to provide more information if possible.

Thanks in advance! 💪

@Lanny
Copy link

Lanny commented Jul 11, 2020

Thanks a ton @isaachinman , was able to deploy the 5.0.0-beta.3 to netlify so just wanted to chime in that other serverless type hosts seem to be working. I did have to explicitly list out my namespaces in the ns value to the NextI18Next constructor, otherwise it would call getAllNamespaces and try to read from the filesystem which failed in a serverless environment. Not sure if this was the intended behavior, or I misconfigured something, or it's a bug but listing out NSs isn't too bad and it's great to be able to run under the serverless target.

@lazidoca
Copy link

lazidoca commented Jul 11, 2020

Thanks a ton @isaachinman , was able to deploy the 5.0.0-beta.3 to netlify so just wanted to chime in that other serverless type hosts seem to be working. I did have to explicitly list out my namespaces in the ns value to the NextI18Next constructor, otherwise it would call getAllNamespaces and try to read from the filesystem which failed in a serverless environment. Not sure if this was the intended behavior, or I misconfigured something, or it's a bug but listing out NSs isn't too bad and it's great to be able to run under the serverless target.

I just deployed to Netlify and it seems to return 404 when accessing the locale URL directly.

The demo is at: https://eager-shannon-a9d1c4.netlify.app/

If you access this URL, it shows 404

https://eager-shannon-a9d1c4.netlify.app/de/second-page

The repository for the demo is forked from @isaachinman with adding target: "serverless" https://github.com/welrn/next-i18next-vercel

@isaachinman
Copy link
Contributor

@Progressandro @Lanny @lazidoca – as I wrote, please open separate issues.

@i18next i18next locked as resolved and limited conversation to collaborators Jul 11, 2020
@isaachinman
Copy link
Contributor

Hey all – NextJs v9.5 came out today, and rewrites are no longer experimental. I've therefore just released next-i18next@v5.0.0, which supports serverless. Enjoy!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests