Skip to content

Introduction to EMV, gas pricing model and standard gas optimisation techniques.

License

Notifications You must be signed in to change notification settings

mtumilowicz/ethereum-gas-workshop

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ethereum-gas-workshop

preface

  • goals of this workshop
    • introduction to EMV
    • understanding gas pricing model
    • showing standard gas optimisation techniques
  • workshop task
    • improve Inefficient.sol
      1. replace compareStrings with keccak256
      2. use correct qualifiers: view, calldata, etc
      3. use correct data structure: mapping

EVM = Ethereum Virtual Machine

  • refresh: virtual machine
    • is a type of simulation of a CPU
    • has predefined operations, but such operations must be understood by the virtual machine and not by the CPU
    • is a program capable of interpreting a specific language
      • transforming (indirectly) this language into machine language
      • executing what needs to be executed on the CPU.
    • language that the virtual machine understands is called bytecode
      • each virtual machine has its own bytecode with its own definitions
      • bytecode is a series of instructions that the EVM will interpret and execute
      • when we write a smart contract in Solidity or Vyper, the result must be transformed (compiled) to bytecode
      • EVM bytecode is not machine language
        • although it looks like a set of bits, it is only a representation
  • Ethereum is like a single-threaded computer
    • it can process one transaction at a time
    • Sharding of blockchain would improve it and make it like a multithreaded computer
  • virtual, isolated environment where code (smart contracts) can be executed
    • running on the EVM is not directly executed by any single computer, but by all nodes in the Ethereum network
    • you can think of JVM(Java Virtual Machine) as the same mechanism
  • is Turing complete
    • contracts contain very little code and their methods require very few instructions to execute
      • usually less than one thousand
      • regular computers execute several billion instructions per second
    • to prevent infinite loops and resource exhaustion, the EVM requires users to pay for computation and storage
      • Ethereum Virtual Machine is seen only as quasi-Turing complete
      • payment is made in the form of "gas"
        • a unit of measurement for the amount of computational work required to execute operations
      • since the London hard fork, each block has a target size of 15 million units of gas
        • the actual size of a block will vary depending on network demand
          • protocol achieves an equilibrium block size of 15 million on average through the process of tâtonnement
          • if the block size is greater than the target block size, the protocol will increase the base fee for the following block
          • the protocol will decrease the base fee if the block size is less than the target block size
          • maximum size: 30 million gas (2x the target block size)
            • means that a block can only contain transactions which cost 30m gas to execute
      • example
        • 3m gas is at maximum 1.5 million instructions (in a very theoretical, unrealistic scenario)
          • example
            • MSTORE (Memory Store): around 3 gas
            • SSTORE (Storage Operation):
              • writing to a new storage slot: Around 20,000 gas (for a 256 bit word)
                • a kilobyte is thus 640k gas
                  • so if gas ~ 10 gwei
                  • 1KB costs 0.0064 ETH
                  • 1GB costs 6400 eth (for eth 1.5k USD, ~ 12,000,000 USD)
                • so a block can only contain instructions that write to storage about 150 times
              • updating an existing storage slot: Around 5,000 gas
            • SLOAD (Storage Load): around 200 gas
  • is deterministic
    • running the same code with the same inputs will produce the same results every time
  • is a stack-based machine
  • operates using a set of instructions called "opcodes"
    • opcodes are predefined instructions that the EVM interprets
    • some operators have operands, but not all
      • operator that have operand: PUSH
        • push to the stack
      • operator that does not have: ADD
        • takes from the stack, add and push result
    • example
      // Solidity code
      function addIntsInMemory(uint a, uint b) public pure returns (uint) {
          uint result = a + b;
          return result;
      }
      
      is compiled into operands and operators (opcodes)
      PUSH1 0x20      // Load the memory slot size (32 bytes)
      MLOAD           // Load 'a' from memory
      PUSH1 0x40      // Load the memory slot size (32 bytes)
      ADD             // Add 'a' and 'b'
      MSTORE          // Store the result back in memory
      
    • then it is interpreted to bytecode
  • bytecode is a set of bytes that must be executed in order, from left to right
    • each byte can be
      • an operator (represented by a single byte)
      • a complete operand
      • part of an operand (operands can have more than just 1 byte)
        • example: PUSH20 - used to push a 20-byte (160-bit) value onto the stack
    • example
      • ADD opcode is represented as 0x01
      • PUSH1 opcode is represented as 60 and expects a 1-byte operand
      • bytecode to analyse: 0x6001600201 -> 60 01 60 02 01
      • byte 60 is the PUSH1 opcode
        • adds a byte to the Stack
        • The PUSH1 opcode is an operator that expects a 1-byte operand
        • Then the complete statement is 60 01
      • 60 02 // similar
      • final result: number 3 on the Stack
      • digression
        • byte 01 was used both as an operand and operator
        • it’s easy to figure out what it represents in the context of how it was used
    • you cannot generate the exact original Solidity source code from the EVM bytecode
      • process of compiling involves
        • optimizations
        • transformations
        • and potentially even loss of information
          • example
            • during complication the function names and their input parameters are hashed to generate the function selectors
            • to compute function selector
              1. concatenate the function name and parameter types without spaces or commas: `myFunction(uint256,address)``
              2. calculate the keccak-256 (sha3) hash of the concatenated string
              3. take the first 4 bytes of the hash
  • can be described as a global decentralized state machine
    • more than a distributed ledger
      • analogy of a 'distributed ledger' is often used to describe blockchains like Bitcoin
        • ledger maintains a record of activity which must adhere to a set of rules that govern what someone can and cannot do to modify the ledger
          • example: Bitcoin address cannot spend more Bitcoin than it has previously received
          • Ethereum has its own native cryptocurrency (Ether) that follows almost exactly the same intuitive rules
      • it enables a much more powerful function: smart contracts
    • state = the current state of all accounts and smart contracts on the blockchain
      • includes things like account balances, contract storage, and contract code
    • each transaction on a blockchain is a transition of state
    • EVM is the engine that processes transactions and executes the corresponding smart contract code
      • leads to state changes
      • at any given block in the chain, Ethereum has one and only one 'canonical' state
        • EVM is what defines the rules for computing a new valid state from block to block

gas

  • is a measure of computational work required to execute operations or transactions on the network
    • opcodes have a base gas cost used to pay for executing the transaction
      • example: KECCAK256
        • cost: 30 + 6 for every 256 bits of data being hashed
    • there isn't any actual token for gas
      • example: you can't own 1000 gas
      • exists only inside of the Ethereum virtual machine as a count of how much work is being performed
  • is the fee paid for executing transactions on the Ethereum blockchain
    • example
      • simple transaction of moving ETH between two addresses
      • we know that this transaction requires 21,000 units
      • base fee for standard speed at the moment of writing is 20 gwei
      • gas fee = gas units (limit) * gas price per unit (in gwei)
      • 21,000 * 20 = 420,000 gwei
      • 420,000 gwei is 0.00042 ETH, which is at the current prices 0.63 USD (1 ETH = $1500)
  • gas prices change constantly and there are a number of websites where you can check the current price
  • if Ether (ETH) was directly used as the unit of transaction cost instead of gas, it would lead to several potential problems:
    • reduced flexibility
      • gas allows for adjustments to the cost of computation without affecting the underlying value of Ether
      • if Ether were used directly
        • any change in pricing would directly impact the value of the cryptocurrency
        • it would be difficult to prevent attackers from flooding the network with low-cost transactions
        • cost of computation should not go up or down just because the price of ether changes
          • it's helpful to separate out the price of computation from the price of the ether token
    • difficulty in predictability
      • Ether's value can be volatile, which means that transaction costs would fluctuate with the market price
      • this could lead to unpredictable costs for users and could make it more challenging to budget for transactions
  • is used to
    • prevent infinite loops
    • computational resource exhaustion
    • prioritize transactions on the network
    • prevent Sybil attacks
      • by discouraging the creation of a large number of malicious identities
      • solution: prevents an attacker from overwhelming the network with a massive number of transactions
        • as each transaction costs some amount of gas
    • solve halting problem
      • problem = it's generally impossible to determine whether that program will eventually halt or continue running indefinitely
      • solution: program will eventually run out of gas and the transaction will be reverted
  • gas has a price, denominated in ether (ETH)
    • users set the gas price they are willing to pay to have their transaction or smart contract executed
    • miners prioritize transactions with higher gas prices because they earn the fees associated with the gas
    • analogy
      • gas price as the hourly wage for the miner
      • gas cost as their timesheet of work performed
  • every operation consumes a certain amount of gas
    • is paid by users to compensate miners for the computational work they perform
    • total gas fee = gas used * gas price
  • each block has a gas limit
    • maximum amount of gas that can be consumed in a block
    • transaction sender is refunded the difference between the max fee and the sum of the base fee and tip
  • some operations can result in a gas refund
    • example: if a smart contract deletes a storage slot, it gets a gas refund
      • digression
        • London Upgrade through EIP-3529: remove gas refunds for SELFDESTRUCT, and reduce gas refunds for SSTORE to a lower level
        • practically speaking gas refunds for selfdestruct was not encouraging the freeing up of network space
          • was encouraging the speculation on gas prices (in an extremely inefficient manner) via GAS tokens
          • was filling the blockchain with space-consuming gastokens and transactions just to get some cheap gas back
          • example
            1. during a period of lower gas prices you deploy contractA (called usually GasToken)
            2. during gas prices spike you'd self-destruct contractA and receive gas refund
            3. you can use that gas refund for paying for transaction during spike
    • refund is only applied at the end of the transaction
      • the full gas must be made available in order to execute the full transaction
  • it's important to estimate the gas needed for a transaction or smart contract execution
    • if too small => the operation will be reverted and any state changes will be discarded
      • miner still includes it in the blockchain as a "failed transaction", collecting the fees for it
        • sender still pays for the gas consumed up to that point
        • the real work for the miner was in performing the computation
          • they will never get those resources back either
          • it's only fair that you pay them for the work they did, even though your badly designed transaction ran out of gas
    • if too big => the excess gas is refunded (refund = max fee - base fee + tip)
      • max fee (maxFeePerGas)
        • maximum limit to pay for their transaction to be executed
        • must exceed the sum of the base fee and the tip
      • providing too big of a fee is also different than providing too much ether
        • if you set a very high gas price, you will end up paying lots of ether for only a few operations
          • similar to super high transaction fee in bitcoin
        • if you provided a normal gas price, however, and just attached more ether than was needed to pay for the gas that your transaction consumed
          • excess amount will be refunded back to you
          • miners only charge you for the work that they actually do
  • EIP-1559
    • implemented in the London Hard Fork upgrade
    • went live in August 2021
    • introduces a new fee structure that separates transaction fees
      • NOT designed to lower gas fees but to make them more transparent and predictable
      • two components
        • base fee
          • minimum fee required to include a transaction in a block
          • determined by network congestion
            • example
              • when the network is busy, the base fee increases
              • when it's less congested, the base fee decreases
            • increase/decrease is predictable and will be the same for all users
            • removing the need for each and every wallets to generate their own individual gas estimation strategies
          • is burned
            • removed from circulation
            • reducing the overall supply of Ether
            • miners have less control over manipulating transaction fees
              • no reason into bumping base price by putting load on the network
            • benefits all Ether holders equally, rather than exclusively benefiting validators
              • creates what EIP-1559 coordinator Tim Beiko refers to as an “ETH buyback” mechanism
              • ETH is paid back to the protocol and the supply gets reduced
        • priority fee
          • optional tip to incentivize miners to include their transaction in the next block
          • goes directly to the miner
    • similar to a delivery service
      • lower fee for regular delivery or a higher fee for express delivery
      • during busy times, like the holiday season, the delivery service may increase the standard delivery fee
        • increase will be set by the delivery company and will affect all customers equally
    • comparable to Bitcoin’s difficulty adjustment
    • oracles might run into issues under EIP-1559 during periods of high congestion
      • oracles are used when you require off-chain data
        • example: Oraclize or ChainLink
      • they need to provide the pricing information for nearly all of DeFi
        • example: in lending protocols, it influences interest rates and collateral ratios
      • might end up paying incredibly high fees in order to ensure the pricing information reaches the DeFi application in a timely manner
    • context: original Ethereum gas fee system
      • simple auction system: unpredictable and inefficient
        • users bid a random amount of money to pay for each transaction
          • we can see a large divergence of transaction fees paid by different users in a single block
            • many users often overpay by more than 5x
            • example alt text
        • when the network becomes busy, this system causes gas fees to become high and unpredictable
        • not easy to quick-fix
          • possible improvement: users submit bids as normal, then everyone pays only the lowest bid that was included in the block
            • can be easily gamed by miners who will fill up their own blocks in order to increase the minimum fee
            • gameable by transaction senders who collude with miners
      • similar to the way ride-sharing services calculate ride fees
        • when demand for rides is higher, prices go up for everyone who wants a ride
      • problem: Ethereum network becomes busy
        • example
          • CryptoKitty users have reached in excess of 1.5 million(25% of total Ethereal traffic in peak times)
          • trade on a new decentralized cryptocurrency exchange
        • result: users trying to push their transactions by paying absurdly high gas fees
          • gas fees become unpredictable
          • users must guess how much to pay for a transaction
      • as Ethereum has gained new users, the network has become more congested
        • gas fees have become more volatile
        • many users have inadvertently overpaid for their transactions

solidity

  1. minimize on-chain data
    • storage operations are over 100x more costly than memory operations
      • OPcodes mload and mstore only cost 3 gas units while storage operations
      • sload and sstore cost at least 100 units
    • keep all data off-chain
      • save the smart contract’s critical info on-chain
      • save part of the system (metadata, etc .. ) on a centralized server
    • data that does not need to be accessed on-chain can be stored in events
  2. minimize storage read/writes
    • save intermediate results in memory and assign results to storage after all calculations
    • caching the length in for loops
      • reading array length at each iteration of the loop takes 6 gas
        • 3 for mload and 3 to place memory_offset in the stack.
      • caching the array length in the stack saves around 3 gas per iteration
        • storage => extra sload operation
          • 100 additional extra gas (EIP-2929) for each iteration except for the first
        • memory => extra mload operation
          • 3 additional gas for each iteration except for the first
        • calldata => extra calldataload operation
          • 3 additional gas for each iteration except for the first) These
        • extra costs can be avoided by caching the array length (in the stack)
          uint _length = arr.length
          
  3. use storage pointers instead of memory
    • example
      mapping(uint256 => User) public users;
      User storage _user = users[_id];
      
      is cheaper
      mapping(uint256 => User) public users;
      User memory _user = users[_id]; // involves copying
      
  4. use calldata instead of memory
    • more cost-effective to load them immediately from calldata
      • does not require copying variables to memory
  5. avoid loops
    • it consumes a lot of gas
    • it can prevent contract from being carried out beyond the block gas limit
    • use mappings
      • except when iteration is required or it is possible to pack data types (arrays are iterable and packable)
  6. minimize the number of storage slots used
    • gas cost for storage usage is calculated based on the number of storage slots used
    • each storage slot has a size of 256 bits
    • you can pack multiple variables within a single storage slot
  7. use 256 byte types
    • example: uint256
    • EVM performs operations in 256-bit chunks
      • using uint8 means the EVM has to first convert it to uint256
      • conversion costs extra gas
  8. get gas refund for free up storage
    • has the same effect as reassigning the value type with its default value
    • mappings are unaffected by deletion
      • slots of values are random (based on key hash) and generally unknown
  9. use immutable and constant
    • evaluated at compile-time and are stored in the bytecode of the contract
  10. enable the Compiler Optimizer
    • several optimization tools available: solc optimizer, Truffle’s build optimizer, and Remix’s Solidity compiler
  11. use short circuit rule
    • disjunction: if the first function evaluates to true, the second function is not executed
    • conjunction: if the first function evaluates to false, the second function is skipped entirely
  12. move the modifiers require statements into an internal virtual function
    • modifier code is substituted by compiler to every method that uses this modifier
    • example
      modifier onlyOwner() {
          require(msg.sender == owner, "Only owner can call this function");
          _;
      }
      
      into
      modifier onlyOwner() {
          _onlyOwner();
          _;
      
      function _onlyOwner() private {
          require(msg.sender == owner, "Only owner can call this function");
      }
      
  13. use Libraries
    • extract common functions into a single library and then deploy this library just once
  14. use layer 2 solutions
    • works by creating a network of payment channels on top of a blockchain network
    • enable offloading of transaction processing from the main Ethereum chain
    • solutions
      • rollups
        • transactions occur off-chain on the rollup chain itself
          • only a summary or commitment of the transactions is recorded on the Ethereum mainnet
            • smart contract on the Ethereum mainnet checks that the commitment is valid
        • smart contract part can be imagined as ERC20
          • balance of each participant is recorded in the contract
          • hundreds "transfer" would be packaged into one transaction
            • contract can disassemble these "transfer" and verify
          • two merkle trees are used for the record
            1. one is to record addresses, so only an index can represent an address
              • in rollup transaction recipient's address is replaced by an index value much smaller
            2. the other tree records balance and nonce
              • rollup transaction does not require a nonce value since it can be computed from the previous state
        • can be further categorized into two types
          • optimistic rollups
            • assume transaction validity by default unless proven otherwise
              • require dispute resolution mechanisms in case of fraud or incorrect transaction execution
            • example: Optimism
          • zero-knowledge (zk)-rollups
            • use advanced cryptographic techniques to validate transactions
            • compress and store the user state on-chain in a Merkle tree
            • transfer the state transition of the user states to the off-chain
              • zkSNARK proof is used to ensure the correctness of the off-chain state transition
            • example: ZK-Sync
      • sidechains
        • separate blockchains that are interoperable with the Ethereum mainnet
          • has its own consensus mechanism and can have different rules and features
        • to move assets between the main chain and a sidechain, you typically need to use a bridge
          • involves locking assets on the main chain and minting corresponding tokens on the sidechain
        • example: Polygon, xDai
      • channels
        • parties can exchange an unlimited amount of transactions off-chain
          • example
            • Alice sends a signed message to Bob saying "I send you 1 ETH"
            • Bob counter-signs the message, indicating his agreement to the new state
        • only submitting two transactions to the mainchain
          • open the channel
            • deploy a smart contract
            • fund smart contract with an initial deposit
          • close the channel
            • submitting the final state to the smart contract on the mainnet
            • smart contract verifies the final state and distributes the funds accordingly
        • example: Raiden, Celer Network, Connext
  15. use in-line assembly code
    • efficient code that can be executed directly by the EVM without the need for expensive Solidity opcodes
    • more precise control over memory and storage usage
  16. don't assigning default values
    • every variable assignment in Solidity costs gas
    • example
      uint256 value
      
      is cheaper than
      uint256 value = 0
      
  17. memory is very cheap to allocate as long as it is small
    • past a certain point (32 kilobytes) in a single transaction, the memory cost enters into a quadratic section
    • example
      contract smallArraySize {
          // Function Execution Cost = 21,903
          function checkArray() external {
              uint256[100] memory myArr;
          }
      }
      contract LargeArraySize {
          // Function Execution Cost = 276,750
          function checkArray() external {
              uint256[10000] memory myArr;
          }
      }
      contract VeryLargeArraySize {
          // Function Execution Cost = 20,154,094
          function checkArray() external {
              uint256[100000] memory myArr;
          }
      
  18. batching
    • consolidate data retrieval by calling a function that returns all the required data instead of making separate calls for each data element
  19. use external function modifiers
    • function parameters are not copied into memory but are read directly from the call data
  20. use erc1167 to deploy the same contract many time
    • standardized, gas-efficient way to deploy a bunch of contract clones from a factory
    • not only minimizes length, but it is also literally a “minimal” proxy that does nothing but proxying
    • address in EIP1167 is hardcoded in bytecode and remain unchangeable
    • example
      • one of the most famous proxy contract users is Uniswap
        • has a factory pattern to create exchanges for each ERC20 tokens
        • has one exchange instance that contains full bytecode as the program logic, and the remainders are all proxies
        • https://etherscan.io/address/0x09cabec1ead1c0ba254b09efb3ee13841712be14#code
          • a short bytecode, which is unlikely an implementation of an exchange
          • what it does is blindly relay every incoming transaction to the reference contract by delegatecall
        • every proxy is a 100% replica of that contract but serving for different tokens
        • length of the creation code of Uniswap exchange implementation is 12468 bytes
          • proxy contract, however, has only 46 bytes