23.2.0

This is an optional update that primarily updates dependent libraries. It is a required update for Sepolia network nodes.

Downloads

• Available as 23.2.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 536a9bbfdad22b1bd38fa9c5e877abdad1a06aab8e520d77ae534a1c5768b488)
  • zip (sha256: e41c43d0b68654983604b66736c64db8cbfe20bdb530b230e3bfe5a2280fb5b6)

Breaking Changes

• --Xmetrics-block-timing-tracking-enabled option has been removed. The --metrics-block-timing-tracking-enabled option should be used instead (enabled by default).

Additions and Improvements

• Introduced Doppelganger Detection feature
• Changed color of block production duty to cyan, rather than dark blue.
• Introduced a configuration overview printed during start up.
• Introduced a new field el_offline in the GetSyncingStatusResponse which is set to true when the execution layer is offline: Syncing API update
• Added Capella fork information for Sepolia network configuration.
• Removed references to beacon block methods /eth2/beacon_chain/req/beacon_blocks_by_root and /eth2/beacon_chain/req/beacon_blocks_by_range.

Bug Fixes

• Fixed issue which was causing validator keys to be loaded twice, which, in some cases, was causing beacon node to initialize slower.

23.1.1

This is an optional update containing a permissions fix to docker containers. Users not experiencing permissions issues with their docker containers do not need to upgrade to this patch.

Downloads

• Available as 23.1.1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 715ee9bef3e0730558b79ef1d3bce5ae190cf01c4e02c3982e79582fca775995)
  • zip (sha256: 6477cda7143b7675ae36aa1f17bc5556eac44b63b12e15d2e7f9d48e81d7b0ef)

Bug Fixes

• Docker container permissions have been updated #6741

23.1.0

This is a recommended update containing bug fixes, optimisations and new features.

Downloads

• Available as 23.1.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: b0d34bc6a162fd3a8b7be8ad10d55dae800bf73e21749536c64cf968364b78f0)
  • zip (sha256: a63692e63efd943357b905023ec77deba4260149a86da33546d9cf8aa1aa80f7)

Breaking Changes

• The validator_beacon_node_published_attestation_total, validator_beacon_node_published_aggregate_total,
  validator_beacon_node_send_sync_committee_messages_total, validator_beacon_node_send_sync_committee_contributions_total
  and validator_beacon_node_published_block_total metrics have been replaced by the new validator_beacon_node_requests_total metric. An update to the Teku Dashboard that uses the new metric is available.

Additions and Improvements

• Introduces the MINIMAL option for --data-storage-mode which prunes both finalized states and blocks prior to the retention period required by the beacon chain spec (~5 months of blocks).
• Support for the /eth/v1/beacon/blinded_blocks/{block_id} REST API
• Support for the /eth/v1/debug/fork_choice REST API
• Support for the /eth/v1/beacon/deposit_snapshot REST API
• Added finalized metadata field to applicable REST API responses
• Use SSZ encoding for external validator client block creation requests by default. Can be disabled with --beacon-node-ssz-blocks-enabled=false.
• Added a timeout (2 minutes) when attempting to load the initial state from a URL
• Improved logging when sync committee messages fail to publish to the gossip network
• Updated bootnodes for the Gnosis chain
• Updated jblst library to version 0.3.10
• Updated docker images to Ubuntu version 22.04

Bug Fixes

• Fixed issue which could cause command line options to be parsed incorrectly
• Fixed issue where the voluntary-exit subcommand did not exit immediately after completion
• Fixed reported security issue on Netty, updating to version 4.1.87.Final (addressing CVE-2022-41881)

22.12.0

This is an optional update that primarily updates dependent libraries.

Downloads

• Available as 22.12.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 920a0759465d695e2fd921023f204c45531d1f73a69d1418cc16e010539cdbb4)
  • zip (sha256: e30c37756704164a568181fbf69fc5c13f94e0831329ee970ae968a6274548fb)

Breaking Changes

• Dropped support for custom network configurations that do not use the current preset format. Presets have been in use since before the MainNet beaconchain launch so this is not expected to affect any users.

Additions and Improvements

• Updated third party libraries
• Early access: Support for additional DoS protection by using a separate sentry beacon node to publish blocks and attestations.
• Tightened validation of the fork for gossip messages

22.11.0

This is a recommended update containing security updates. It is a required update for Gnosis chain nodes.

Downloads

• Available as 22.11.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 2ab8cf50e29c9ce9a62d5f1250693dc0d5cd588e7ce5bb9c4afef830eec682f3)
  • zip (sha256: 62c05b2a91f7b43ab575742185edb691bbc3745bbc4f7b57d10128a831d7c616)

Breaking Changes

• The logic with which the default configuration is applied when using validators-proposer-config has been updated.
  The logic is now more intuitive but a given proposer config file may result in a different configuration compared with the previous Teku version.
  (Refer to #6325 (comment) for more information).

Additions and Improvements

• Updated protobuf used by libp2p library to resolve a potential DoS vector
• Added merge configuration for Gnosis chain. Note that all Gnosis nodes will need to be made merge-ready
• Improved discv5 compliance
• Changed the builder is online\is offline logs to is available\is not available
• Added /eth/v1/beacon/states/{state_id}/randao to beacon-api
• Block timing tracking is now enabled by default. The --Xmetrics-block-timing-tracking-enabled option has been renamed to --metrics-block-timing-tracking-enabled.

Bug Fixes

• Fixed missing status filters (active, pending, exited, withdrawal) for the /eth/v1/beacon/states/{state_id}/validators endpoint
• Fixed issue which could lead to duplicate processing of some events after gossip is stopped and restarted.
• Fixed issue which could cause sync committee duties to be calculated too early, potentially causing duties to be missed if the scheduling was changed by a reorg.

22.10.2

Downloads

• Available as 22.10.2 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 0a0d8bbce52ab1c023da905492fbf3d1b32885e85b632766bca9b31b9fa44540)
  • zip (sha256: 24c66b76720e22d28477ef9f0f53227ba69ad6785343a3dd4a4f593ff2a4d2a5)

Breaking Changes

There are no breaking changes in this release.

Additions and Improvements

• Enabled new framework for REST API with better performance and less memory usage. The new framework does not cause any breaking changes, however if needed the old version can be restored with --Xrest-api-migrated-enabled=false
• Updated jackson-databind library to version 2.13.4.2 addressing CVE-2022-42003

Bug Fixes

• Fix issue where /readiness endpoint returned 200 when Execution Client was not available.

Upcoming Breaking Changes

• The validator_beacon_node_published_attestation_total, validator_beacon_node_published_aggregate_total,
  validator_beacon_node_send_sync_committee_messages_total, validator_beacon_node_send_sync_committee_contributions_total
  and validator_beacon_node_published_block_total metrics have been deprecated in favour of the new validator_beacon_node_requests_total metric.
  The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads
• The commandline option --network of the validator-client subcommand has been undeprecated and can be used to select a network for standalone validator clients. When set to auto, it automatically
  fetches network configuration information from the configured beacon node endpoint.

22.10.1

This is a recommended upgrade containing bug fixes, optimisations and voluntary-exit command improvements

Downloads

• Available as 22.10.1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 8418bfd435cae9a6d8226343b55c9f43decca3b2717a887ae57a1ca3ec4899d5)
  • zip (sha256: 5c5fbc83590f8d9c3686343974b695d6c214b77b943c0987df2bc8ec8dee1476)

Additions and Improvements

• Improve Execution Layer error logging
• Add new validator client metric validator_remote_beacon_nodes_requests_total which tracks the requests made to remote beacon nodes (useful when there are failovers configured)
• The voluntary-exit subcommand can restrict the exit to a specific list of validators public keys using the option --validator-public-keys.
  Example: teku voluntary-exit --beacon-node-api-endpoint=<ENDPOINT>[,<ENDPOINT>...]... --data-validator-path=<PATH> --include-keymanager-keys=<BOOLEAN> --validator-keys=<KEY_DIR>:<PASS_DIR> | <KEY_FILE>:<PASS_FILE> --validator-public-keys=<PUBKEY>[,<PUBKEY>...]...
  To include validator keys managed via keymanager APIs, the option --include-keymanager-keys could be set to true (The default value is set to false)
• Throttle signing of validator registrations when using an external signer

Bug Fixes

• Filter out unknown validators when sending validator registrations to the builder network
• Fix issue which could cause locally produced aggregates to not be gossiped
• Fix issue where the sync module could cause Unexpected rejected execution due to full task queue in nioEventLoopGroup log messages and high CPU usage

Upcoming Breaking Changes

• The validator_beacon_node_published_attestation_total, validator_beacon_node_published_aggregate_total,
  validator_beacon_node_send_sync_committee_messages_total, validator_beacon_node_send_sync_committee_contributions_total
  and validator_beacon_node_published_block_total metrics have been deprecated in favour of the new validator_beacon_node_requests_total metric.
  The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads
• The commandline option --network of the validator-client subcommand has been undeprecated and can be used to select a network for standalone validator clients. When set to auto, it automatically
  fetches network configuration information from the configured beacon node endpoint.

22.9.1

This is a recommended update containing improvements to post-merge performance issues on MainNet.
There are no changes from the previous 22.9.1-RC1 so users who are running the RC do not need to upgrade.

Downloads

• Available as 22.9.1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: a4ab66e30cc24c99d14f91d31d95443cb0d12c984521837ec72e114af5cb5fd9)
  • zip (sha256: 826d7be0966d86281f1c18dfe9fd58c1881561918ffa2b5aa6b1f33e82422a80)

Breaking Changes

• The --initial-state and --eth1-deposit-contract-address options has been removed from the validator-client subcommand. They have been ignored for some time but are now completely removed.
• LevelDB configuration has been changed to increase the maximum number of files it may open at a time. Systems with low open file handle limits may encounter issues.
  We recommend at least 2048 file handles or preferably unlimited. The ulimit command can be used to check and configure file handle limits.

Performance Improvements

This release significantly improves Teku's performance post-merge, but we will continue to investigate further improvements.

Please note that if the execution client is slow to respond, particularly when importing new blocks, that will increase the rate of incorrect head votes and increase the likelihood of attestations not being included.

To see more information on where time is being spent when importing blocks, users can enable late block alerts with the --Xmetrics-block-timing-tracking-enabled option. This option will log a warning when a block import completes more than 4 seconds into the slot (after attestations will be created) with a break down of where time was spent. For example:

Late Block Import *** Block: e83a322a714b1b364ab59b2cb8666076fa015a8a5b4f1a8b9829306fc249f787 (4718961) proposer 425317 arrival 681ms, pre-state_retrieved +11ms, processed +306ms, execution_payload_result_received +3173ms, begin_importing +0ms, transaction_prepared +0ms, transaction_committed +1ms, completed +15ms

The labels used for timings are somewhat Teku developer focussed, but some key steps are:

• arrival - the time since the start of the slot that the block was first received. Large values here usually indicate that the block was produced late.
• state_retrieved - the time it took to prepare the pre-state to apply the block to.
• processed - the time it took Teku to process the block.
• execution_payload_result_received - the time Teku spent waiting for the execution client to finish importing the block.

The remaining entries are Teku processing time to complete importing the block, updating fork choice records and storing the data to disk.

So in the example above the block was late to import because of a slow response from the execution client.

Additions and Improvements

• Enables asynchronous database updates by default. This ensures slow disk access or LevelDB compactions don't cause delays in the beacon node
• Increased leveldb open files to 1000 files by default. If teku fails to start, update maximum open files to 2048 or unlimited
• Reduced number of blocks held in in-memory cache to reduce need for old-gen GC with the larger post-merge blocks
• Make Validator Client connect to a failover event stream (if failovers are configured) when the current Beacon Node is not synced
• Detect Lodestar clients in libp2p_connected_peers_current metrics
• Reduce CPU and Memory consumption in shuffling, which will improve epoch transition performance
• Faster peer discovery on startup

Bug Fixes

• Fixed an issue where epoch cache priming sometimes happened synchronously, causing delays in time tick processing in the beacon node
• Fix /eth/v1/validator/register_validator responding with a 400 status code and a misleading error message in case of exceptions
• Update snakeyaml dependency to resolve cve-2022-25857 which could result in excessive memory usage when parsing YAML content
• Fixed an issue where the range requested for deposit logs was not reduced when using only --ee-endpoint leading to persistent timeouts with execution clients

Upcoming Breaking Changes

• The validator_beacon_node_published_attestation_total, validator_beacon_node_published_aggregate_total,
  validator_beacon_node_send_sync_committee_messages_total, validator_beacon_node_send_sync_committee_contributions_total
  and validator_beacon_node_published_block_total metrics have been deprecated in favour of the new validator_beacon_node_requests_total metric.
  The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads

22.9.1-RC1

This is a release candidate containing improvements to post-merge performance issues on MainNet that has received less testing than our normal releases.
MainNet users should only upgrade if they are confident in their ability to rollback if required. We encourage all testnet users to upgrade to assist with testing.

Downloads

• Available as 22.9.1-RC1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 6df71f19c681b2f1ca3fbd5f834824f1d2753cc9cc8ce2fe4e94c1ba92c12a83)
  • zip (sha256: 36e0f55e799bf4650d04dee69b5a713db509ddc0655f09b06fc20724e65b7b29)

Breaking Changes

• The --initial-state and --eth1-deposit-contract-address options has been removed from the validator-client subcommand. They have been ignored for some time but are now completely removed.
• LevelDB configuration has been changed to increase the maximum number of files it may open at a time. Systems with low open file handle limits may encounter issues.
  We recommend at least 2048 file handles or preferably unlimited. The ulimit command can be used to check and configure file handle limits.

Additions and Improvements

• Enables asynchronous database updates by default. This ensures slow disk access or LevelDB compactions don't cause delays in the beacon node. Can be disabled if needed via --Xdata-storage-async-enabled=false
• Increased leveldb open files to 1000 files by default. If teku fails to start, update maximum open files to 2048 or unlimited
• Reduced number of blocks held in in-memory cache to reduce need for old-gen GC with the larger post-merge blocks
• Make Validator Client connect to a failover event stream (if failovers are configured) when the current Beacon Node is not synced
• Detect Lodestar clients in libp2p_connected_peers_current metrics
• Reduce CPU and Memory consumption in shuffling, which will improve epoch transition performance
• Faster peer discovery on startup

Bug Fixes

• Fixed an issue where epoch cache priming sometimes happened synchronously, causing delays in time tick processing in the beacon node
• Fix /eth/v1/validator/register_validator responding with a 400 status code and a misleading error message in case of exceptions
• Fix a NullPointerException for the gas limit when a proposer config is used and builder is enabled
• Update snakeyaml dependency to resolve cve-2022-25857 which could result in excessive memory usage when parsing YAML content
• Fixed an issue where the range requested for deposit logs was not reduced when using only --ee-endpoint leading to persistent timeouts with execution clients

Upcoming Breaking Changes

• The validator_beacon_node_published_attestation_total, validator_beacon_node_published_aggregate_total,
  validator_beacon_node_send_sync_committee_messages_total, validator_beacon_node_send_sync_committee_contributions_total
  and validator_beacon_node_published_block_total metrics have been deprecated in favour of the new validator_beacon_node_requests_total metric.
  The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads

22.9.0

This is a required update. All users should upgrade as soon as possible.

IMPORTANT: Mainnet nodes must be made "merge ready"

All Mainnet nodes will need to prepare for the merge, including:

• Connecting to a local execution client's engine API with the --ee-endpoint option.
• Validators will need to set a default fee recipient with --validators-proposer-default-fee-recipient.

Both Teku and your execution client will need to be updated to prepare for the merge. You should not receive any warning or error messages about the execution client being unavailable.

See the Teku guide on preparing for the merge for more information.

Downloads

• Available as 22.9.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 8bdba94f20f19c8779909bae310ced5a23962374fe92ecb0755e60a0d4ce2af0)
  • zip (sha256: e35be79568d225d250606f0d225767a37eff082d2bf68e1c04b91185be72a4d9)

Bug Fixes

• Resolves an issue with public key validation.

Upcoming Breaking Changes

• The validator_beacon_node_published_attestation_total, validator_beacon_node_published_aggregate_total,
  validator_beacon_node_send_sync_committee_messages_total, validator_beacon_node_send_sync_committee_contributions_total
  and validator_beacon_node_published_block_total metrics have been deprecated in favour of the new validator_beacon_node_requests_total metric.
  The old metrics will be removed in a future release. An update to the Teku Dashboard that uses the new metric is available.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads