Add client configuration to Retesteth

Add client configuration to Retesteth

To run retesteth you need to clone official test repo: https://github.com/ethereum/tests

Retesteth and your client are both compiled locally

If your client is yet not added to retesteth configurations go edit the folder

~/.retesteth

This folder contains configurations parsed by retesteth. A typical configuration looks like <FolderName>/config

config

{
	"name" : "Ethereum aleth on IPC Debug",
	"socketType" : "ipc-debug",
	"socketAddress" : "/home/wins/.ethereum/geth.ipc"
}

Field name type a string that represent your client (will be printed at the test run) Fields socketType and socketAddress take following values:

1) ipc-debug

"socketType" : "ipc-debug",
"socketAddress" : "/home/wins/.ethereum/geth.ipc"   

ipc-debug type tells retesteth to use already running instance of a client. retesteth will run tests on this instance with one thread only and connect to it's ipc socket via path defined by the socketAddress field.

2) tcp

"socketType" : "tcp",
"socketAddress" : "127.0.0.1:8545"

tcp type tells retesteth to use already running instance of a client. retesteth will run tests on this instance with one thread only and connect to it's tcp socket via address defined by the socketAddress field. socketAddress field could be an array:

"socketType" : "tcp",
"socketAddress" : {
    "127.0.0.1:8545",
    "127.0.0.1:8546",
    "127.0.0.1:8547"
}

The other ports will be used to run tests in parallel when -j <n> flag is provided.

3) ipc

"socketType" : "ipc",
"socketAddress" : "local"

The combination of values ipc and local tells retesteth to instantiate as many client instances as defined by -j <thread count> parameter, when running tests. Each instance will be executed using <FolderName>.sh script file which must be defined in the config folder.

The format of .sh file is following:

#!/bin/bash
onexit() 
{ 
    kill $child 
}
trap onexit SIGTERM
trap onexit SIGABRT

##ARGUMENTS PASSED BY RETESTETH
##
## $1  <db-path> 		A path to database directory that a client should use
## $2  <ipcpath>		A path to the ipc socket file (has /geth.ipc at the end)
#####

aleth --test --db-path $1 --ipcpath $2 --log-verbosity 5 &
child=$! 
wait "$child"

Each time retesteth needs a new instance it will call this file with 2 parameters (temp db path) and (temp ipc path). In this example above aleth is installed in the system and is being executed by bash with corresponding options. Retesteth receive a handle and then once test execution is over will kill the instance.

Retesteth is compiled in docker container and your client is compiled locally

See Docker Instructions: https://github.com/ethereum/retesteth/wiki/Docker-instructions

Default Retesteth Configs

if you don't specify the --clients "..." option, retesteth will read the configs from default folder ~/.retesteth/default
Take a look at genesis subfolder: ~/.retesteth/default/genesis
It contains configs that are used to produce genesis configuration when test_setChainParams method is performed. If you create this subfolder in your custom client test folder, it will override the default configs, allowing you to create custom test_setChainParams. Although it is highly recommend to implement generic genesis standard for test_setChainParams.

Config file

The client for retesteth described by the config file like this:

"name" : "Ethereum GO on TCP",
"socketType" : "tcp",
"socketAddress" : [
    "127.0.0.1:8545",
    "127.0.0.1:8546"
],
"forks" : [
    "Frontier",
    "Homestead",
    "EIP150",
    "EIP158",
    "Byzantium",
    "Constantinople",
    "ConstantinopleFix",
    "Istanbul"
],
"additionalForks" : [
    "FrontierToHomesteadAt5",
    "HomesteadToEIP150At5",
    "EIP158ToByzantiumAt5",
    "HomesteadToDaoAt5",
    "ByzantiumToConstantinopleFixAt5"
],
"exceptions" : {
    "ExtraDataTooBig" : "extra-data too long"
}

• Where name is the client name to be printed n the console.
• socetType is a connection method described above.
• socketAddress can have multiple entires for TCP types. Multiple addresses are used when retesteth is run with -j<threadNumber> multithreaded mode.
• forks define the progression order in which a test case would be generated on. In the test you set >=Frontier and it will be decoded using Forks section into Frontier, Homestead, EIP150... ConstantinopleFix, Istanbul
• additionalForks section defines all other allowed to run on the client forks.
• exceptions define expecred error string from the client when filling the tests by retesteth