Future of next-page-tester

[toomuchdesign]

I open this issue to:

• update next-page-tester users about the maintenance status of this library
• announce that I will not be be able to actively maintain next-page-tester anymore

next-page-tester maintenance status

For about one year @Meemaw and I have developed and maintained next-page-tester. I started this library to explore an alternative testing approach with Next.js and @Meemaw brought in so many good ideas.

This testing approach proved to have pros and cons and enabled some valuable testing strategies. Since based on DOM testing, it proved to be able to cover a wide range of Next.js testing scenarios with the same tools used to test React component (eg. testing-library).

next-page-tester development/maintenance revealed to be quite problematic for a few reasons:

• It relies on a few Next.js internals which can change without notice in any moment. This means a lot of maintenance effort to align next-page-tester implementation with Next.js (and staying updated about Next.js internals evolution).

• It simplistically re-implements most of Next.js output (supported by TypeScript). This means any new Next.js feature has to be implemented/adjusted based on users' feedback/issues

• There's no way to know whether next-page-tester will be able to support future Next.js features/versions

• Since next-page-tester is not supported in any way by Next.js, it won't be able to reach a significant degree of adoption (for good reasons)

For the reasons mentioned above next-page-tester is currently stuck over 2 major issues:

• Next.js v12 compatibility #281
• Enable useDocument option again #268

Future of next-page-tester

During the last year I haven't touched Next.js, and it's getting hard to follow it's evolution for the sole purpose of maintaining next-page-tester.

I'm convinced that the main downside of this project consists of its maintenance being 100% dependent of Next.js evolution. I personally came to the conclusion that the only viable solution is letting Vercel decide whether they want to invest on such a testing strategy and let them expose some Next.js bindings to enable DOM testing.

Beside the 2 issues mentioned above, there are new features waiting to be implemented which I couldn't take care of.

In order to make further development easier I moved next-page-tester to a dedicated GitHub org. In the meanwhile @jasonwilliams joined maintainers team 🙌 .

I'm still providing support and sharing my project-specific knowledge to possible contributors.

I big hand up to all the contributors who spent time and love on this project. You proved that open source can be sustainable and human friendly sometimes! ❤️

[rtrembecky]

Is there a discussion somewhere on the next.js side (https://github.com/vercel/next.js/discussions) about the potential adoption of next-page-tester? Has anybody from next.js ever shown interest in this project?
I know there are tons of issues there but I believe the page testing is a pretty important topic to be noticed by the next.js team, so it definitely deserves a place there.

[Meemaw]

@rtrembecky I've been chatting with some folks at Vercel recently and mentioned those issues.

They are aware and recently added something in this direction (vercel/next.js#31246), but this is not nearly what this library is providing.

[leerob]

I'd love to understand what hooks you'd need into Next.js to not have to rely on internals. Could you share more?

[toomuchdesign]

Hi @leerob,
let's say that next-page-tester, given a Next.js project and route, tries to render the expected element tree without spinning up an actual server.

This involves, to mention a few:

• fetching Next.js config file with expected defaults
• importing and rendering expected React context providers (HeadManager, Router)
• replicating Next.js filesystem routing
• replicating _app's and _document's and pages' ctx and render them properly
• injecting expected env vars
• last but not least: simulating client and server environments in the same testing process

Beside this, we often import TS types which are not exposed in public entry points.

From my standpoint, there's nothing specific Next.js might do in order to help next-page-tester maintenance long term, since we're trying to replicate the output of a complex system from outside. If DOM testing is going to remain a valuable testing approach (given that browser testing is becoming more and more easy), Next.js could consider exposing a dedicated DOM testing module with its own tests living, running and evolving close to Next.js ones. But that's another story :)

Maybe @Meemaw has different opinions/suggestions, though.
I hope I was able to provide some valuable insights.

[jasonwilliams]

Hey @toomuchdesign

Did you want to transfer this repo to an organisation?
I think this would help move things forward as right now it’s directly under your name.

I don’t know if @Meemaw is staying on but I can help if need be so that’s at least 2, potentially @leerob and some others.

Of course if Vercel make some decision that would be great but we should be prepared for nothing coming out of that and having a working version for next v12.

[toomuchdesign]

Hi @jasonwilliams,
thanks for writing!

I'm 100% available to transfer this repo to an organization. As I said, I'd be happy not to see the efforts done so far get wasted.

I know very little about how GitHub orgs work, any advice is welcome :)

[jasonwilliams]

https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository

You may have to create the organisation first, then transfer the repo into it. Then you can add people.

[toomuchdesign]

So the zero-creativity outcome would be: next-page-tester/next-page-tester. What about that? :)

[jasonwilliams]

Yep obvious naming is good

[toomuchdesign]

next-page-tester transferred to next-page-tester org.

I invited @Meemaw as a org administrator with full access. Not sure how to proceed with further members.

[jasonwilliams]

Awesome!

There’s a question around publishing but I will leave that with you. I often create a secret on the repo and publish from a GitHub action. Or a new @namespace gets created on npm, I don’t know.

I think it’s worth you staying on as an admin too even if you have no time anymore, just in case something happens or we don’t hear from @Meemaw

[Meemaw]

I'll help as much as time will permit. Can do code reviews for new PRs, but probably won't have time to work on some bigger things, e.g. v12 compatibility.

[toomuchdesign]

As for publishing I guess we're going to need an npm namespace, too and add users with:
https://docs.npmjs.com/cli/v8/commands/npm-owner

[pelhage]

The ability to catch runtime bugs ahead of time is invaluable; I can't recount the amount of bugs discovered at runtime that could've been caught through tests.

next-page-tester was recommended as a testing solution in the official Next.js docs so we went with it- but lack of future support is concerning;

@leerob Is the Vercel team open to contributing to this package until there's a more official solution? Otherwise I'm not sure of any alternatives

Appreciate all the contributions thus far. This is a great package!

[toomuchdesign]

Hi @pelhage,
thanks for writing. I didn't know next-page-tester was mentioned in Next.js docs. During this time I've never received much feedback from users or Vercel itself, except a few contributors (🙌) who collaborated on specific bugs/features.

[jasonwilliams]

@pelhage do you have a link to where this is mentioned?

[Meemaw]

https://nextjs.org/docs/testing#community-packages-and-examples

[jasonwilliams]

See #303