Creating a GeneralStateTest (Yoichi's attempt, rough record)

choosing what to test

I was going to test "RETURNDATACOPY" in the beginning should copy zero's into the memory.

checking out sources

I checked out https://github.com/ethereum/cpp-ethereum/pull/4062 at ~/src/cpp-ethereum, and

mkdir -p ~/src/cpp-ethereum/build
cd ~/src/cpp-ethereum/build
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_COMPILER_LAUNCHER=ccache -DVMTRACE=1 ..
make

This PR was still work in progress, but I thought it would perform correctly for this particular test case.

I also checked out https://github.com/ethereum/tests in ~/src/tests (develop branch).

writing test source

choosing a test file

I looked at ~/src/tests/src/GeneralStateTestsFilter.

stAttackTest/                          stChangedEIP150/              stEIP158Specific/           stMemoryStressTest/         stRecursiveCreate/  stSystemOperationsTest/
stBoundsTest/                          stCodeSizeLimit/              stExample/                  stMemoryTest/               stRefundTest/       stTransactionTest/
stCallCodes/                           stCreateTest/                 stHomesteadSpecific/        stNonZeroCallsTest/         stRevertTest/       stTransitionTest/
stCallCreateCallCodeTest/              stDelegatecallTestHomestead/  stInitCodeTest/             stPreCompiledContracts/     stSolidityTest/     stWalletTest/
stCallDelegateCodesCallCodeHomestead/  stEIP150singleCodeGasPrices/  stLogTests/                 stQuadraticComplexityTest/  stSpecialTest/      stZeroCallsRevert/
stCallDelegateCodesHomestead/          stEIP150Specific/             stMemExpandingEIP150Calls/  stRandom/                   stStackTests/       stZeroCallsTest/

Maybe I need a new directory stReturnDataTest.

I created a file ~/src/tests/src/GeneralTestsFiller/stReturnDataTest/returndatacopy_initialFiller.json.

I took a test filter mstore_dejavu.json and started working from there.

writing the code

I thought I should fill the memory with some random data first, and then perform "RETURNDATACOPY". Also, the result of the RETURNDATACOPY should be visible somewhere in the state, so I have to MLOAD and SSTORE the result of the RETURNDATACOPY.

The program looks like this

; hex  description
;; store some non-zero word into memory [0..31]
30   ADDRESS
6000 PUSH1 [0]
52   MSTORE

;; RETURNDATACOPY should copy the first word into memory [0..31]
6020 PUSH1 [32]
6000 PUSH1 [0]
6000 PUSH1 [0]
0e   RETURNDATACOPY

;; load memory[0..31] onto stack
6000 PUSH1 [0]
51   MLOAD

;; store the topmost stack into storage [0]
6000 PUSH1 [0]
55   SSTORE

;; an implicit `STOP` follows.

So, the callee should contain the code 0x306000526020600060000e600051600055

And then I was told to use LLL and the code became (MSTORE 0 0x112233445566778899aabbccddeeff) (RETURNDATACOPY 32 0 0) (SSTORE 0 (MLOAD 0))

writing a pre-state

I expect the storage index 0 to be changed into zero. So, in the pre-state, the storage index 0 should contain something non-zero.

So, this account should be called.

            "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "0x306000526020600060000e600051600055",
                "nonce" : "0x00",
                "storage" : {
                    "0x00" : "0x01"
                }
            },

writing an expectation

The expectation is that: if it's metropolis, the storage at index 0 should change into zero. If it's before metropolis, the storage at index 0 should remain 1.

        "expect" : [
            {   "indexes" : {
                   "data" : -1,
                   "gas" : -1,
                   "value" : -1
                },
                "network" : ["EIP150", "EIP158", "Frontier", "Homestead"],
                "result" : {
                    "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                        "storage" : {
                            "0x00" : "0x01"
                        }
                    }
                }
            },
            {   "indexes" : {
                   "data" : -1,
                   "gas" : -1,
                   "value" : -1
                },
                "network" : ["Metropolis"],
                "result" : {
                    "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                        "storage" : {
                            "0x00" : "0x00"
                        }
                    }
                }
            }
        ],

Above resulted in a filter file in this commit https://github.com/pirapira/tests/commit/b9b9141b3cdd9f72dc5b59bd00f04339290df10c

generating the test

changing cpp-ethereum to run the added test

I created a branch returndata_test from returndata in cpp-ethereum.

I added a line after https://github.com/ethereum/cpp-ethereum/blob/develop/test/tools/jsontests/StateTests.cpp#L180

Compiled cpp-ethereum, and ran

ETHEREUM_TEST_PATH="../../tests" test/testeth -t StateTestsGeneral/stReturnDataTest -- --filltests --checkstate

Back in the tests tree, I added the filled test into the branch.

git add ../../../GeneralStateTests/stReturnDataTest/returndatacopy_initial.json 
git commit

and then?

cpp-ethereum needs to include the tests as a submodule.

For Travis on cpp-ethereum repo to work correctly, the tests need to be in a branch in ethereum/tests. A branch in anybody-else/tests does not work.

After creating a branch in ethereum/tests, in cpp-ethereum/test/jsontests, type

git fetch
git checkout your-branch

and then, in cpp-ethereum,

git add test/jsontests

When you git commit and push the branch, and make a pull-request, Travis will run the new tests in the PR.