Web3 4x Upgrade #5393

nazarhussain · 2022-08-04T13:40:00Z

No description provided.

cds-amal · 2022-08-04T13:44:30Z

Ooh nice! Thanks, @nazarhussain 🙇

eggplantzzz · 2022-08-04T14:28:48Z

Yeah thanks @nazarhussain! Just a note, I saw that you skipped the Vyper compiler tests. If you want to be able to run those successfully you will have to install the Vyper compiler locally. https://vyper.readthedocs.io/en/stable/installing-vyper.html I think I had to use the instructions using pip.

nazarhussain · 2022-08-04T14:31:49Z

@eggplantzzz There are few tests which I am still working on. So why I kept the PR in draft mode for now. As soon all tests passes I will mark it ready.

cds-amal · 2022-08-04T16:48:22Z

@nazarhussain the yarncheck failure indicates yarn.lock is out of sync with the various package.jsons. This can be resolved by doing yarn bootstrap and committing the updated yarn.lock

nazarhussain · 2022-08-04T20:33:07Z

@cds-amal After running the yarn bootstrap I am getting following error while building packages. It was building fine before. Any idea?

~/Hub/Chainsafe/Web3/projects/truffle/packages/codec on nh/web3-4x-upgrade *1 !2 .............................................................................................................................................................................................. took 14s at 22:28:38
> yarn build
yarn run v1.22.19
$ ttsc
/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:117612
                throw e;
                ^

TypeError: Cannot read properties of undefined (reading 'impliedNodeFormat')
    at Object.getModeForUsageLocation (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:116234:18)
    at resolveExternalModule (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:51178:90)
    at resolveExternalModuleNameWorker (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:51154:19)
    at getExternalModuleFileFromDeclaration (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:87472:32)
    at Object.getExternalModuleFileFromDeclaration (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:87311:36)
    at tryGetModuleNameFromDeclaration (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:30114:59)
    at Object.getExternalModuleNameLiteral (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:30078:20)
    at createRequireCall (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:105088:33)
    at visitImportDeclaration (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:105039:80)
    at topLevelVisitor (/Users/nazar/Hub/Chainsafe/Web3/projects/truffle/node_modules/typescript/lib/typescript.js:104697:28)

cds-amal · 2022-08-04T23:40:14Z

@nazarhussain it looks like you may have built the yarn.lock from a different registry? Do you use verdaccio, or another npmjs proxy?

$ grep localhost yarn.lock | head -n 5
  resolved "http://localhost:4873/@ethereumjs%2fcommon/-/common-2.6.5.tgz#0a75a22a046272579d91919cb12d84f2756e8d30"
  resolved "http://localhost:4873/@ethereumjs%2ftx/-/tx-3.5.2.tgz#197b9b6299582ad84f9527ca961466fce2296c1c"
  resolved "http://localhost:4873/@ethersproject%2fabi/-/abi-5.6.4.tgz#f6e01b6ed391a505932698ecc0d9e7a99ee60362"
  resolved "http://localhost:4873/@noble%2fsecp256k1/-/secp256k1-1.6.3.tgz#7eed12d9f4404b416999d0c87686836c4c5c9b94"
  resolved "http://localhost:4873/@scure%2fbip39/-/bip39-1.1.0.tgz#92f11d095bae025f166bef3defcc5bf4945d419a"

$ git log -S"http://localhost:4873"
Thu Aug 4 23:01:18 2022 +0200 29671cc03 (HEAD -> nh/web3-4x-upgrade) :art: Fix the issues raised after merging base branch  [Nazar Hussain]
Thu Aug 4 15:34:12 2022 +0200 8c2d7c34e :arrow_up: Update dependenceies  [Nazar Hussain]%

nazarhussain · 2022-08-08T15:58:24Z

@cds-amal Yes I am using local NPM registry because these packages are not yet published. I will update yarn.lock soon after the release.

nikoulai

FYI to Truffle team, we no longer use bignumber.js or bn.js, just BigInts (@spacesailor24)

nikoulai · 2023-05-08T09:56:22Z

packages/codec/docs/tsconfig.json

-    "lib": [
-      "es2017"
-    ],
+    "lib": ["es2017", "dom"],


As mentioned web3/web3.js#6011 (comment), dom was added to handle the use of RequestInit, but there should be a better way to handle this on our end (@spacesailor24)

nikoulai · 2023-05-08T09:57:21Z

packages/config/package.json

@@ -34,7 +34,7 @@
    "@types/configstore": "^4.0.0",
    "@types/find-up": "^2.1.0",
    "@types/lodash": "^4.14.179",
-    "@types/node": "~12.12.0",
+    "@types/node": "~18.11.17",


Not sure this is acceptable (@spacesailor24)

nikoulai · 2023-05-08T09:59:36Z

packages/codec/lib/abi-data/allocate/index.ts

@@ -7,7 +7,7 @@ import type * as Abi from "@truffle/abi-utils";

 import * as Import from "@truffle/codec/abi-data/import";
 import * as AbiDataUtils from "@truffle/codec/abi-data/utils";
-import Web3Utils from "web3-utils";
+import * as Web3Utils from "web3-utils";


@jdevcs We should probably fix this on our end and give them option to import everything under Web3Utils for backward compatibility (@spacesailor24)

nikoulai · 2023-05-08T10:04:14Z

packages/contract-tests/test/util.js

@@ -2,7 +2,7 @@ const debug = require("debug")("test:util");
 const fs = require("fs");
 const ganache = require("ganache");
 const Web3 = require("web3");
-const Web3PromiEvent = require("web3-core-promievent");
+const { Web3PromiEvent } = require("web3-core");


The name of the const should probably be Web3Core instead of Web3PromiEvent, as the later use of new Web3PromiEvent.Web3PromiEvent(); doesn't make sense (@spacesailor24)

It's used directly util.fakePromiEvent = new Web3PromiEvent();, and we don't use anything else from web3-core in the test.

nikoulai · 2023-05-08T10:10:00Z

packages/contract/lib/contract/constructorMethods.js

@@ -127,13 +127,17 @@ module.exports = Contract => ({
    // use that network and use latest block gasLimit
    if (this.network_id && this.networks[this.network_id] != null) {
      const { gasLimit } = await this.interfaceAdapter.getBlock("latest");
-      return { id: this.network_id, blockLimit: gasLimit };
+      return { id: this.network_id, blockLimit: String(gasLimit) };


What is gasLimit being returned as? (@spacesailor24)

nikoulai · 2023-05-08T10:19:40Z

packages/db/src/network/test/mockProvider.ts

            id,
            result
          });
        }
      }
    }
-  };
+  } as Web3BaseProvider;


Not sure if we should type cast this if it's being used elsewhere as a non web3.js provider (@spacesailor24)

Anything mentioned with as <something> is probably fine. Usually used to point out one type from the union. If there is some real type issue we had to double typecast as unknown as <something> which should be avoided.

nikoulai · 2023-05-08T10:20:05Z

packages/db/src/project/test/assignNames.spec.ts

@@ -64,7 +64,7 @@ const helpers = (db: Db, project: Project.Project) => ({
  }
 });

-describe("Project.assignNames", () => {
+describe.skip("Project.assignNames", () => {


Unhandled TODO (@spacesailor24)

nikoulai · 2023-05-08T10:20:32Z

packages/db/src/resources/projects/test/contractInstances.spec.ts

@@ -8,7 +8,7 @@ import * as Project from "@truffle/db/project";
 import { resources, Run } from "@truffle/db/process";
 import type { IdObject, Input, Resource } from "@truffle/db/resources";

-describe("Project.contractInstances", () => {
+describe.skip("Project.contractInstances", () => {


Unhandled TODO (@spacesailor24)

It was failing earlier because of the following issue web3/web3.js#5714.

nikoulai · 2023-05-08T10:21:07Z

packages/db/src/test/source.spec.ts

@@ -1,7 +1,7 @@
 import { generateId, Migrations, WorkspaceClient } from "./utils";
 import { AddSource, GetSource, GetAllSources } from "./source.graphql";

-describe("Source", () => {
+describe.skip("Source", () => {


Unhandled TODO (@spacesailor24)

nikoulai · 2023-05-08T10:22:10Z

packages/decoder/tsconfig.json

@@ -10,19 +10,10 @@
    "sourceMap": true,
    "outDir": "dist",
    "baseUrl": ".",
-    "lib": ["es2019"],
+    "lib": ["es2019", "dom"],


Related to previous comment (@spacesailor24)

nikoulai · 2023-05-08T11:48:12Z

packages/artifactor/test/contracts.js

-  const web3 = new Web3();
-  web3.setProvider(provider);
+
+  const web3 = new Web3.Web3(provider);


This shouldn't be necessary (@spacesailor24)

Yes not anymore.

If we want to use it in the same way we have to change the import to const { Web3 } = require("web3"); ,else I get the error TypeError: Web3 is not a constructor

nikoulai · 2023-05-08T11:48:33Z

packages/contract-tests/test/linking.js

@@ -28,7 +30,7 @@ const provider = Ganache.provider({
    instamine: "strict"
  }
 });
-const web3 = new Web3();
+const web3 = new Web3.Web3();


This shouldn't be necessary (@spacesailor24)

same as above.

nikoulai · 2023-05-08T11:51:00Z

packages/contract/lib/handlers.js

-      context.promiEvent.eventEmitter.emit("error", error);
-      this.removeListener("error", handlers.error);
+      context.promiEvent.emit("error", error);
+      this.off("error", handlers.error);


Shouldn't removeListener work here? What was the issue? (@spacesailor24)

Our new types for pomievent had off and on for event handling.

nikoulai · 2023-05-08T11:54:29Z

packages/deployer/ens.js

@@ -151,7 +151,9 @@ class ENS {

    // Set top-level name
    let builtName = nameLabels[0];
-    await this.devRegistry.setSubnodeOwner("0x0", sha3(builtName), from, {
+    const ZERO_NODE =
+      "0x0000000000000000000000000000000000000000000000000000000000000000";


Should remove if "0x0" suffices (@spacesailor24)

Should be doubled backed with latest base branch.

"0x0" fails the validation (Web3 validator found 1 error[s]: value "0x0" at "/0" must pass "bytes32" validation)

nikoulai · 2023-05-08T11:55:28Z

packages/deployer/test/deployer.js

-      assert(libReceipt.blockNumber === startBlock + 1);
-      assert(exampleReceipt.blockNumber === libReceipt.blockNumber + 3);
+      assert(libReceipt.blockNumber === startBlock + BigInt(1));
+      //todo web3js-migration this fails


Unhandled TODO (@spacesailor24)

Muhammad-Altabba · 2023-05-31T11:14:24Z

packages/decoder/test/current/test/downgrade-test.js

+  //todo web3js-migration error withch schema
+  // Error: schema with key or id "5f79e70fd2b8a3a84f947e1f2376b6311f756dc26ed9ebfe148d5b0ef3350ac3f933c17a46bbf7fa08c2edb8a39381688f05ed60ff4853d4a72ea3fed8a85b58" already exists


It now works after the updates to the validation function that is available in RC1.
However, the timeout needs to be extended. And this could be done for example by calling inside the test function: this.timeout(3000);. Or by calling on the test function itself like in:

it("...", async function () { ... }).timeout(3000);

eggplantzzz · 2023-07-18T17:41:15Z

Hello! What is the status of this PR? Is there any chance you can rebase this? I would be happy to do the rebase if you'd like as it appears to be dependency-related things which hopefully will be easy to handle. Let me know!

Muhammad-Altabba · 2023-07-19T13:53:26Z

Hello @eggplantzzz
Yes please, it would be great, if you could do the rebase.

eggplantzzz · 2023-08-04T14:51:18Z

Sorry for the delay @nazarhussain. Things have been shifting around here quite a bit and I've been preoccupied. I will do the rebase soon!

Update: it looks like I'm having trouble rebasing this PR as I think you are using a different npm registry for some unpublished packages, is that right? Also there were some dependencies that I think require Node >= 16 while current policy for Truffle is to support versions of Node as early as the latest version of 12. Is there any way we can get around this issue? It is failing after updating package.json's, regenerating the yarn.lock, and rebuilding. The library that the error is coming from is @noble/hashes.

Update2: we might actually be able to drop support for Node 14 in the near future, so that actually might not be a problem after we get to that point. We'll have to see how early we can do that.

Muhammad-Altabba · 2023-08-11T10:56:27Z

Sorry for the delay @nazarhussain. Things have been shifting around here quite a bit and I've been preoccupied. I will do the rebase soon!

Update: it looks like I'm having trouble rebasing this PR as I think you are using a different npm registry for some unpublished packages, is that right? Also there were some dependencies that I think require Node >= 16 while current policy for Truffle is to support versions of Node as early as the latest version of 12. Is there any way we can get around this issue? It is failing after updating package.json's, regenerating the yarn.lock, and rebuilding. The library that the error is coming from is @noble/hashes.

Update2: we might actually be able to drop support for Node 14 in the near future, so that actually might not be a problem after we get to that point. We'll have to see how early we can do that.

Thanks @eggplantzzz,
Happy to hear that you are already planning to drop support for Node 14 in the near future.
And it would be great if you will be able to rebase this MR. I did not try yet as I was working on some other related stuff.
The current state of this MR is that you can install dependencies, and run builds and tests, without any errors. So, the following commands succeed on my machine:

yarn
yarn build
yarn test

However, there are still some skipped tests and TODOs.

Could you also please manage to run the CI for this MR? I suggest creating a feature branch at this repo and we merge this MR to that branch. If this would be helpful for both, running the CI and easing your collaboration on this.

Many thanks,

nazarhussain · 2023-08-14T08:41:20Z

Happy to hear that you are already planning to drop support for Node 14 in the near future.

That will be supper cool and will help to maintain Truffle.

@Muhammad-Altabba @eggplantzzz My suggestion would be to look keenly into skipped tests and if these are not critical, fix those in separate PRs, so we can get this big PR merged earlier.

eggplantzzz · 2023-08-14T17:02:22Z

Ok, let me switch to Node 16 and try the rebase again.

Also, do note @nazarhussain @Muhammad-Altabba that the command yarn bootstrap also needs to complete successfully. This is the command that builds all of Truffle's sub-packages.

eggplantzzz · 2023-08-14T21:55:46Z

So I rebased locally and am getting some build errors for @truffle/interface-adapter related to web3-providers-ws. Can you tell me anything about these? I'll paste a screenshot below -

Muhammad-Altabba · 2023-08-15T14:49:35Z

So I rebased locally and am getting some build errors for @truffle/interface-adapter related to web3-providers-ws. Can you tell me anything about these? I'll paste a screenshot below -

Thanks @eggplantzzz
The issue is with "@types/ws" (web3/web3.js#6303)
Could you please try adding

"overrides": {
  "web3-providers-ws": {
    "@types/ws": "8.5.4"
  }
}

Thanks,

eggplantzzz · 2023-08-15T18:14:02Z

Thanks for the quick response! I added the overrides property and deleted the yarn.lock. When I try and build again I get more type errors.

Any other ideas?

luu-alex · 2023-08-17T00:52:18Z

@eggplantzzz Thanks for your efforts, the web3js team appreciates you :) I tried to understand the error but im still not sure completely what is wrong.

But this issue is coming from web3-eth@4.1.0, which are the newest web3js release packages.

While I debug the issue with the build,
I think the solution is using these versions without bumping to the latest release should work.

eggplantzzz · 2023-08-18T14:03:24Z

So I do have all packages bumped to ^v4.0.3, is that what you are recommending?

and update Web3 imports

…arameter

(fix related to web3/web3.js#6312)

nazarhussain force-pushed the nh/web3-4x-upgrade branch from 514a291 to c4cecf1 Compare September 15, 2022 18:35

nazarhussain force-pushed the nh/web3-4x-upgrade branch from c4cecf1 to d9d4bd4 Compare December 21, 2022 01:51

cds-amal mentioned this pull request May 4, 2023

web3js 4.x RC upgrade #6013

Closed

nikoulai reviewed May 8, 2023

View reviewed changes

nikoulai mentioned this pull request May 15, 2023

Update import syntax for web3 web3/web3.js#6006

Closed

1 task

Muhammad-Altabba reviewed May 31, 2023

View reviewed changes

Muhammad-Altabba mentioned this pull request Jul 20, 2023

Web3 4.x Testing with Truffle web3/web3.js#6144

Closed

1 task

Muhammad-Altabba added 4 commits September 19, 2023 12:05

try enable back git diff at CI

909acf7

fix a unit test for core console

94f6a8d

increase timeout at some tests

6f1bfb8

increase timeout at a test

6989d44

Muhammad-Altabba mentioned this pull request Sep 19, 2023

Get the CI for the Truffle MR to run successfully after re-basing again and fixing the unskipped tests web3/web3.js#6442

Closed

Muhammad-Altabba added 6 commits September 19, 2023 23:22

fix and un-skip tests at deployer

7f65bf5

un-skip tests at provider

4f9f0e7

restore mocking files and empty lines at packages/truffle

b858881

restore mocking files and empty lines at packages/truffle

2618ff7

restore deleted files at resolver/test/fixtures

948ebc9

fix and un-skip tests at packages/resolver

556812c

Muhammad-Altabba mentioned this pull request Sep 20, 2023

Investigate keep calling eth_getBlockByNumber at @truffle/artifactor after the upgrade to v4.x web3/web3.js#6447

Open

un-skip passed test at packages/truffle

3d9c450

Muhammad-Altabba mentioned this pull request Sep 20, 2023

Investigate the 34 failing tests and then investigate why it never ends, at packages/truffle, after the upgrade to v4.x web3/web3.js#6448

Closed

add a comment at a test and add modifiers at .sol file

36abbd4

This was referenced Sep 20, 2023

Apply the solutions for the blocker items at #6144 web3/web3.js#6449

Closed

Do a thorough review for trufflesuite/truffle#5393, directly apply any needed enhancements and mark the MR as Ready web3/web3.js#6450

Closed

Muhammad-Altabba added 5 commits September 21, 2023 14:02

set web3.js packages versions to dev

4ace25c

and update Web3 imports

reject promiEvent on error

f8188f2

allow sendTransaction and smart contract calls to process options p…

36f6591

…arameter

resolve and un-skip issues: web3.js#6320 and web3.js#6317

70250d8

remove web3-types from packages/contract

a674046

Muhammad-Altabba force-pushed the nh/web3-4x-upgrade branch from df37368 to a674046 Compare September 22, 2023 16:15

Muhammad-Altabba added 6 commits September 25, 2023 23:04

update dependencies

232be17

fix and un-skip tests for events logs

af140b2

(fix related to web3/web3.js#6312)

allow deploy to optionally accept transaction options

4d25ae7

resolve and un-skip issue: web3.js#6327

a66de71

modify for issue: web3.js#6345

4a9c99f

remove comments related to issue: web3.js#6311

df2a53a

YoutacRandS-VA approved these changes Oct 14, 2023

View reviewed changes

		//todo web3js-migration error withch schema
		// Error: schema with key or id "5f79e70fd2b8a3a84f947e1f2376b6311f756dc26ed9ebfe148d5b0ef3350ac3f933c17a46bbf7fa08c2edb8a39381688f05ed60ff4853d4a72ea3fed8a85b58" already exists

Web3 4x Upgrade #5393

Are you sure you want to change the base?

Web3 4x Upgrade #5393

Conversation

nazarhussain commented Aug 4, 2022

cds-amal commented Aug 4, 2022

eggplantzzz commented Aug 4, 2022

nazarhussain commented Aug 4, 2022

cds-amal commented Aug 4, 2022

nazarhussain commented Aug 4, 2022

cds-amal commented Aug 4, 2022 • edited

nazarhussain commented Aug 8, 2022

nikoulai left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eggplantzzz commented Jul 18, 2023

Muhammad-Altabba commented Jul 19, 2023

eggplantzzz commented Aug 4, 2023 • edited

Muhammad-Altabba commented Aug 11, 2023

nazarhussain commented Aug 14, 2023

eggplantzzz commented Aug 14, 2023 • edited

eggplantzzz commented Aug 14, 2023

Muhammad-Altabba commented Aug 15, 2023

eggplantzzz commented Aug 15, 2023 • edited

luu-alex commented Aug 17, 2023

eggplantzzz commented Aug 18, 2023

cds-amal commented Aug 4, 2022 •

edited

eggplantzzz commented Aug 4, 2023 •

edited

eggplantzzz commented Aug 14, 2023 •

edited

eggplantzzz commented Aug 15, 2023 •

edited