App scaffolded with create-cosmos-app cannot call lockdown successfully #2033

samsiegart · 2024-02-05T06:39:16Z

Describe the bug

As shown in https://github.com/agoric-labs/hardened-create-cosmos-app, when I try using create-cosmos-app to make a cosmos app (which runs on Next.js), I cannot successfully invoke lockdown and run the app.

Error Details:

See the README of the example repo for error details.

Steps to reproduce

Clone https://github.com/agoric-labs/hardened-create-cosmos-app
yarn && yarn dev to install deps and run locally
Load the url outputted by yarn dev
Observe the error

Expected behavior

lockdown can be invoked in the top-level of the application without error.

Platform environment

What OS are you using? MacOS 13.3.1 (22E261)
What version of Node.js? v18.17.1
Is there anything special/unusual about your platform? No
What version of Endo are you using? "ses": "^1.1.0"

Additional context

Create-cosmos-app is a tool for scaffolding cosmos apps with Next.js. It would be beneficial to be able to use SES and Endo APIs in apps built using this tool. The error is perhaps more general to Next.js as a whole, but this tool is of particular interest.

Screenshots

See https://github.com/agoric-labs/hardened-create-cosmos-app

The text was updated successfully, but these errors were encountered:

kriskowal · 2024-02-09T20:12:49Z

This smells like a CommonJS interop issue and I would like to summon @naugtur, if we can borrow his eyes.

kriskowal · 2024-02-09T20:13:38Z

@samsiegart Capturing the error in the description will help us diagnose.

samsiegart · 2024-02-09T20:26:30Z

@samsiegart Capturing the error in the description will help us diagnose.

There's a screenshot and two stack traces in the example repo's README. I've updated the description here to point it out more explicitly

samsiegart · 2024-02-09T20:27:08Z

Oh, it's private, I should make it public

Edit: done

erights · 2024-02-09T22:43:29Z

@kriskowal , I've assigned it to the two of us.

erights · 2024-02-09T22:51:48Z

@naugtur , you too ;)

LuqiPan · 2024-02-12T18:29:36Z

Another data point- shouldn't be a surprise given I'm using the same create-cosmos-app, just with a slightly different template:
I tried to initialize an NFT example with create-cosmos-app --name hardened-nft-example --example nft and I got the same error as @samsiegart, see https://github.com/agoric-labs/hardened-nft-example?tab=readme-ov-file#bug-report

erights · 2024-02-13T03:56:38Z

Reproduced locally! Thanks for the clear and simple instructions.

erights · 2024-02-13T04:08:04Z

Yeah, @naugtur @kriskowal , it is all about the cjs loader. I will definitely need your help looking at this. It was easy to reproduce locally, so could you have a look? If the bug is interesting, perhaps we could discuss it at the upcoming endo meeting?

erights · 2024-02-13T04:26:39Z

Actually, this seems kinda urgent for us. Could we meet tomorrow after you've taken a first look? I am free tomorrow anytime after 2pm Pacific.

kumavis · 2024-02-14T19:14:46Z

@tgrecojs you mentioned on the call you might be familiar with this issue

tgrecojs · 2024-02-14T20:02:03Z

I just took a moment to spin up an example repo which integrates @endo/init into a next.js application. This code can be found here - https://github.com/tgrecojs/nextjs-with-endo-init

Pasting the contents of the README file from that repository below. @samsiegart let me know if this helps as all. One more thing I want to note - I think your repo may be using a version of past next.js. FWIW I have wired up past versions to work with endo so I don't believe that is the issue here but I'm not entirely sure.

README.md

Next.js X @endo/init

View this incredibly rich and interactive demo here - https://hardened-nextjs-repo.vercel.app/.

This repository demonstrates the integration of Endo and Knit within a Next.js application.

Below you can view contents of the project's _document.js file.

import { Html, Head, Main, NextScript } from "next/document";
import Script from "next/script";

export default function Document() {
  return (
    <Html>
      <Head />
      <body>
        <Main />
        <NextScript />
        <Script
          async
          type="module"
          src="https://esm.sh/@endo/init@1.0.2"
          strategy="beforeInteractive"
        />
      </body>
    </Html>
  );
}

The key component within this file is the Next.js specific <Script /> element as it is responsible for loading @endo/init. When used outside of Next.js' _document.js file, this component has been the source of issues when loading the page such as the one seen here (although it is not necessarily responsible for this issue).

Checking for tamper-proof globals.

Below is a screenshot of interaction from https://hardened-nextjs-repo.vercel.app

naugtur · 2024-02-15T15:13:16Z

I reproduced the error and the error starts on the server

$ next dev
  ▲ Next.js 13.5.6
  - Local:        http://localhost:3000

 ✓ Ready in 1626ms
 ✓ Compiled / in 1937ms (2269 modules)
 ⨯ [ReferenceError: __webpack_require__ is not defined
  at Object.eval (webpack-internal:///./node_modules/next/dist/build/webpack/loaders/next-route-loader/index.js?kind=PAGES&page=%2F&preferredRegion=&absolutePagePath=.%2Fpages%2Findex.tsx&absoluteAppPath=private-next-pages%2F_app&absoluteDocumentPath=private-next-pages%2F_document&middlewareConfigBase64=e30%3D!:1:1)
  at eval (eval at makeEvaluate (file:///lab/samples/hardened-create-cosmos-app/node_modules/ses/src/make-evaluate.js:92:27), <anonymous>:12:22)
  at ./node_modules/next/dist/build/webpack/loaders/next-route-loader/index.js?kind=PAGES&page=%2F&preferredRegion=&absolutePagePath=.%2Fpages%2Findex.tsx&absoluteAppPath=private-next-pages%2F_app&absoluteDocumentPath=private-next-pages%2F_document&middlewareConfigBase64=e30%3D! (/lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:22:1)
  at __webpack_require__ (/lab/samples/hardened-create-cosmos-app/.next/server/webpack-runtime.js:33:42)
  at __webpack_exec__ (/lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:361:39)
  at /lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:362:114
  at __webpack_require__.X (/lab/samples/hardened-create-cosmos-app/.next/server/webpack-runtime.js:185:21)
  at /lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:362:47
  at Object.<anonymous> (/lab/samples/hardened-create-cosmos-app/.next/server/pages/index.js:365:3)] {
  page: '/'
}
 ✓ Compiled /_error in 228ms (2271 modules)
[ReferenceError: exports is not defined
  at Object.eval (webpack-internal:///./node_modules/next/dist/pages/_document.js:2:23)
  at eval (eval at makeEvaluate (file:///lab/samples/hardened-create-cosmos-app/node_modules/ses/src/make-evaluate.js:92:27), <anonymous>:12:22)
  at ./node_modules/next/dist/pages/_document.js (/lab/samples/hardened-create-cosmos-app/.next/server/vendor-chunks/next.js:30:1)
  at __webpack_require__ (/lab/samples/hardened-create-cosmos-app/.next/server/webpack-runtime.js:33:42)
  at __webpack_exec__ (/lab/samples/hardened-create-cosmos-app/.next/server/pages/_document.js:52:39)
  at /lab/samples/hardened-create-cosmos-app/.next/server/pages/_document.js:53:83
  at __webpack_require__.X (/lab/samples/hardened-create-cosmos-app/.next/server/webpack-runtime.js:185:21)
  at /lab/samples/hardened-create-cosmos-app/.next/server/pages/_document.js:53:47
  at Object.<anonymous> (/lab/samples/hardened-create-cosmos-app/.next/server/pages/_document.js:56:3)]

the exports reported on the page might as well be a sourcemap translation back from webpack_require OR abug caused by it being missing

tgrecojs · 2024-02-15T17:03:10Z

@naugtur did you get to that error using the _document.js file I referenced? If so, then I am definitely intersted in hearing as much.

I reproduced the error as well in this repo and it did it by changing the strategy attribute to beforeInteractive. (I've since arrived at the belief that _document.js approach is the best solution.)

https://github.com/tgrecojs/hardened-nextjs/blob/b0c77d97f90b3b536e75390c62655afd9ee61c3a/source/shared/HOCs/withHardenedJs.hoc.js#L3-L15

I can see this my solution fixes this in the repo related to this issue if it would be most helpful. cc @samsiegart @erights

naugtur · 2024-02-20T14:51:20Z

@tgrecojs The difference between yours and this repo seems to be this one is running lockdown on the serverside and yours is running it on clientside.

What is the intended result?
re your way of adding it to the clientside - this reminds me of a similar situation in our work on lockdown for react native where we were trying to find the right place to run it. Are you sure this way you're not running lockdown too late?

tgrecojs · 2024-02-21T15:35:27Z

What is the intended result?

This question is best suited for @samsiegart. It's my understanding that the intended result is for this error to go away, and for this application to properly run lockdown.

The difference between yours and this repo seems to be this one is running lockdown on the serverside and yours is running it on clientside.

I‘m assuming you are referring to the repo referenced in my last comment, and not the one I recreated last week. If you're referring to this project, then let me know how you were able to confirm lockdown was being called on the client.

I pushed up a change last night after doing a bit of investigating now it seems as though @endo/init is being loaded on the server side however lockdown doesn't seem to get invoked on the server. If it does, then the behavior isn't consistent.

the next.js docs on Script shows the following:

Although the default behavior of next/script allows you to load third-party scripts in any page or layout, you can fine-tune its loading behavior by using the strategy property:

beforeInteractive: Load the script before any Next.js code and before any page hydration occurs.

afterInteractive: (default) Load the script early but after some hydration on the page occurs.

lazyOnload: Load the script later during browser idle time.
worker: (experimental) Load the script in a web worker.

This indicates that next.js to load this code before any hydration is done leading me to believe that hardened javascript would be loaded on the server before any other code (including next.js internals). There was one big issue though - beforeInteractive without async leads to the very error @samsiegart is mentioning.

The following code loads the script properly, but it still doesn't consistently download endo before next.js code.

    <Script
          async
          type="module"
          src="https://esm.sh/@endo/init@1.0.2"
          strategy="afterInteractive"
        />

Here's a repo to the latest implementation -
https://github.com/tgrecojs/nextjs-with-endo-init/blob/main/pages/_document.js

A live URL can be viewed here - https://nextjs-with-endo-init-kpj7t18sm-thomasgreco.vercel.app/

samsiegart · 2024-02-21T18:37:34Z

It would also be preferred to be able to load ses from a local bundle as opposed to the url https://esm.sh/@endo/init@1.0.2

tgrecojs · 2024-02-21T19:24:28Z

I'm getting the impression that you may have viewed the repo I spun up as a potential solution to the issues your facing with create-cosmos-app. I want to quickly clarify that this was not my intention at all. It was only meant to provide some added context into integrating next.js with SES.

So, as for your ask...

It would also be preferred to be able to load ses from a local bundle as opposed to the url https://esm.sh/@endo/init@1.0.2

I can reference @endo/init locally without issue. But is that the only ask here, or am I missing something. If it's not, then please let me know what your end goal is so I can try and assist.

samsiegart · 2024-02-22T02:39:20Z

Yea the intended result is that we can use endo APIs inside the application on top of next.js+create-cosmos-app. I guess if the local @endo/init is a non-factor, then we're just on the question of whether we need beforeInteractive or not. I can try based on your example and see if it works to call makeAgoricWalletConnection without beforeInteractive. If so your <Script> tag solution could suffice.

gacogo · 2024-03-06T08:06:06Z

I came across this when trying to fork dapp-offer-up. I went around it by disabling server side rendering.

demo: https://dapp-offer-up-next-ui.vercel.app/

tgrecojs · 2024-03-07T00:12:57Z

The goal here seems to be using next.js' SSR capabilities and from what i've gathered the Script tag is the vehicle that will get us to the desired outcome. I've just drafted a PR for this here.. I'm adding the comments from that here for visibility.

I'm not completely over the hill yet but I think there is some solid progress made. For one, i'm almost certain that the approach that needs to be taken is to use beforeInteractive with _document.js. This change makes use of this approach, and everything seems to render properly. There is only one issue - @endo/init doesn't seem to be doing anything?

Below is a screen grab of the application booting up. You can see that @endo/init is the first script loaded out of the bunch, which is the expected behavior of beforeInteractive. But it doesn't seem to actually be calling lockdown.

endo-init-nextjs.mp4

Here is the URL for this deployment - https://create-cosmos-app-three.vercel.app/.

@endo/init seems to be getting imported, but it just stops there resulting in Object.isFrozen(Array.prototype) returns false.

I'm wondering if anyone can give additional input on how @endo/init is functioning here. I'm not sure why lockdown doesn't seem to be getting called? And how can I make sure the @endo/init invokes lockdown (I expected it to invoke lockdown automatically).

It seems like there is something I'm missing here but not exactly sure what 🤔 cc @naugtur @kriskowal @kumavis

gacogo · 2024-03-16T14:10:57Z

Hey @samsiegart . I've created a pull request] intended to fix some of these bugs, using a slightly modified installSesLockdown.ts. I hope you find it helpful.

Issues that still persist during development: Once you make a change on a file and attempt to reload page manually, WebPack will throw errors. After making changes you'll have to run yarn dev manually after making changes. Live reload seems to work however.

Edit: I've hosted it here https://hardened-create-cosmos-app.vercel.app/

tgrecojs · 2024-03-16T20:01:00Z

@gacogo this is a really interesting solution! I do have one question - are you still disabling SSR in your PR?

Also, i'm not sure if you read any of my comments on this issue but I provide a fairly detailed look into how next.js suggests loading scripts (without disabling SSR).

gacogo · 2024-03-17T04:45:10Z

@tgrecojs Am not disabling ssr with this solution. I also tried dynamic imports with {ssr: true} and it worked.

erights · 2024-04-01T00:03:04Z

What's the status of this? Should this be tracked as an endo ecosystem compat issue #2037 ?

samsiegart added the bug Something isn't working label Feb 5, 2024

erights assigned kriskowal and erights Feb 9, 2024

erights assigned naugtur Feb 9, 2024

erights mentioned this issue Apr 1, 2024

Ecosystem Compatibility (2024-02-05 onward) #2037

Open

kriskowal added the ecosystem-compatibility Tracks a compatibility issue in a third-party package or packages. label Apr 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

App scaffolded with create-cosmos-app cannot call lockdown successfully #2033

App scaffolded with create-cosmos-app cannot call lockdown successfully #2033

samsiegart commented Feb 5, 2024 •

edited

kriskowal commented Feb 9, 2024

kriskowal commented Feb 9, 2024

samsiegart commented Feb 9, 2024

samsiegart commented Feb 9, 2024 •

edited

erights commented Feb 9, 2024

erights commented Feb 9, 2024

LuqiPan commented Feb 12, 2024

erights commented Feb 13, 2024

erights commented Feb 13, 2024

erights commented Feb 13, 2024

kumavis commented Feb 14, 2024

tgrecojs commented Feb 14, 2024

naugtur commented Feb 15, 2024 •

edited

tgrecojs commented Feb 15, 2024

naugtur commented Feb 20, 2024

tgrecojs commented Feb 21, 2024 •

edited

samsiegart commented Feb 21, 2024

tgrecojs commented Feb 21, 2024

samsiegart commented Feb 22, 2024

gacogo commented Mar 6, 2024

tgrecojs commented Mar 7, 2024

gacogo commented Mar 16, 2024 •

edited

tgrecojs commented Mar 16, 2024

gacogo commented Mar 17, 2024

erights commented Apr 1, 2024

App scaffolded with create-cosmos-app cannot call lockdown successfully #2033

App scaffolded with create-cosmos-app cannot call lockdown successfully #2033

Comments

samsiegart commented Feb 5, 2024 • edited

Describe the bug

Error Details:

Steps to reproduce

Expected behavior

Platform environment

Additional context

Screenshots

kriskowal commented Feb 9, 2024

kriskowal commented Feb 9, 2024

samsiegart commented Feb 9, 2024

samsiegart commented Feb 9, 2024 • edited

erights commented Feb 9, 2024

erights commented Feb 9, 2024

LuqiPan commented Feb 12, 2024

erights commented Feb 13, 2024

erights commented Feb 13, 2024

erights commented Feb 13, 2024

kumavis commented Feb 14, 2024

tgrecojs commented Feb 14, 2024

README.md

Next.js X @endo/init

Checking for tamper-proof globals.

naugtur commented Feb 15, 2024 • edited

tgrecojs commented Feb 15, 2024

naugtur commented Feb 20, 2024

tgrecojs commented Feb 21, 2024 • edited

samsiegart commented Feb 21, 2024

tgrecojs commented Feb 21, 2024

samsiegart commented Feb 22, 2024

gacogo commented Mar 6, 2024

tgrecojs commented Mar 7, 2024

gacogo commented Mar 16, 2024 • edited

tgrecojs commented Mar 16, 2024

gacogo commented Mar 17, 2024

erights commented Apr 1, 2024

samsiegart commented Feb 5, 2024 •

edited

samsiegart commented Feb 9, 2024 •

edited

naugtur commented Feb 15, 2024 •

edited

tgrecojs commented Feb 21, 2024 •

edited

gacogo commented Mar 16, 2024 •

edited