23.9.1

This is an optional update unless you are going to run a node on the new Holesky network. It contains the updated configuration for the Holesky launch on the 28th of September.

This release contains a leveldb library that requires GLIBC version 2.34.

• Linux users running on an OS that doesn’t provide this library won’t be able to run Teku and need to downgrade to 23.9.0.
• For reference, the minimum Debian version providing the library is 12 (bookworm), the minimum Ubuntu version is 22.0.4 (jammy).
• Windows users or users running teku using official docker images are not affected.

Downloads

• Available as 23.9.1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: b5381056e7d4bf2f118806ec8ffd0dcd586ada744712bc2873e7bc2cbe2c926e)
  • zip (sha256: 8191a3447ba58e8a07d6f938e69324a6bf812fb6e4b6f07e1a648b480af7eb15)

Breaking Changes

• Removed epoch from the POST /eth/v1/validator/liveness/{epoch} response as it was not part of the api spec.

Additions and Improvements

• The latest version of blst will automatically use optimized code paths if they are supported. As a result, JAVA_OPTS="-Dteku.portableBlst=true" is no longer necessary for some older systems.
• The voluntary exit subcommand now accepts --network=<NETWORK> command line option, using it to load the network specification rather than loading configuration from the rest api, if specified.
• Add /teku/v1/beacon/blob_sidecars/{slot} Teku API which returns all blob sidecars (canonical and non-canonical) at a specific slot. This endpoint will become available once Deneb fork is scheduled.
• Updated LevelDb native library which is using LevelDb 1.23 using latest Snappy to 1.1.10 for compression (this change doesn't apply to Windows)
• Updated holesky configuration with new genesis time of 28th September 12:00 UTC, and fork changes.
• Apply proposer boost to first block in case of equivocation (spec PR)

Bug Fixes

• When the rest-api's fail to start up they can now potentially 'fail fast' rather than silently ignoring the issue.

Upcoming Breaking Changes

N/A

23.9.0

This is a recommended update containing networking updates, optimizations and bug fixes. It also contains upcoming Holesky testnet configuration.

Downloads

• Available as 23.9.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: faa72f61accf98eb61de1129a80390373e196b4d8ec6ac0a58bac8aa0fc0e804)
  • zip (sha256: 3e6cc64e0c960c3c1d1197d46cdd49ddf1932c2c9eba0281ad1232b2c7fc32f2)

Breaking Changes

• The Development option --Xp2p-minimum-subnet-subscriptions has been removed and will no longer be recognised as a command line option.
• If running separated Beacon Node and Validator node it is required to upgrade Beacon Node first and then Validator Node. A Validator Node running this release won't start if connecting to an older Beacon Node version.

Known issues

• When switching an existing database to minimal mode, teku can become unresponsive. It will recover after cleaning up the old blocks, but this can take a period of time (an hour or so). If you wish to switch to minimal mode, consider re-initialising the database (delete beacon db and restart from a checkpoint sync) to avoid an outage.

Additions and Improvements

• Default storage mode has been changed from prune to minimal. Existing prune mode databases will need to explicitly set --data-storage-mode=minimal if they wish to switch to minimal data storage. Nodes starting with empty database will activate in minimal mode unless a different value is set via--data-storage-mode.
• Update attestation subnet subscriptions strategy according to the spec changes. All nodes (including non-validating ones) will subscribe to 2 subnets regardless of the number of validators.
• Added /eth/v1/validator/{pubkey}/voluntary_exit Validator API endpoint
• Add support for Holesky test network --network=holesky
• Add support for gzip encoding in REST API

Bug Fixes

• Fixed a bug in network configuration loader which was ignoring MIN_EPOCHS_FOR_BLOCK_REQUESTS parameter.
• Fixed a bug causing proposer config refresh feature to stop reloading the configuration.

Upcoming Breaking Changes

23.8.0

This is a recommended update containing few memory allocation optimizations. It also includes a move to Java 17 as the minimum supported Java version.

Downloads

• Available as 23.8.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 55b7c54aa21d0e31eb661f0e6734ab1214c34ce7d0fa7c45e6fb2ce3146636e9)
  • zip (sha256: 63536d689c51a1fd1ff9a535ef0e431d8bd943d0fa6321a86cfdf19673620f08)

Breaking Changes

• Upgrading the minimum Java version to Java 17, which means users will need to upgrade their Java installation to at least Java 17. The docker versions relying on jdk16 will no longer be published, so docker users explicitly referencing the jdk16 build need to update their package to reference jdk17, as tags develop-jdk16, develop-jdk16-arm64, latest-jdk16 will no longer be updated.
• Users who make heavy use of API calls to fetch non finalized states data other than head may wish to adjust the states-cache if they see excessive regeneration of state messages. This can be accomplished via the --Xstore-state-cache-size, which previously defaulted to 160.

Additions and Improvements

• Introduce --exchange-capabilities-monitoring-enabled parameter. If enabled, EL will be queried periodically for the Engine API methods it supports. If incompatibility is detected, a warning is raised in the logs. The default is true.
• Add support for Lukso network --network=lukso
• The development option --Xfork-choice-update-head-on-block-import-enabled was changed to default to false to ensure fork-choice is run when new blocks arrive.
• The default state-cache size has been changed to 8 (previously 160), and there is now an epoch-states-cache, which defaults to a maximum of 6 elements.

Bug Fixes

• Fixed a possibility of an overflow error when calculating validator duties if the number of active validators is >1M.

Upcoming Breaking Changes

23.6.2

This is an optional update with minor bug fixes and dependencies updates.

Downloads

• Available as 23.6.2 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: c0b021ead22a514fdb8d5e2ce072549dc447dfdeed660eccef0e3a42890e56c2)
  • zip (sha256: 103ae8d5b486f74f764db6432aa1e60300e9e18404b039fcce6f8f944f3239d4)

Breaking Changes

• --exchange-capabilities-enabled option has been removed since it is no longer applicable because of execution-apis spec change.
• Removed kiln and ropsten as --network options

Additions and Improvements

• --validators-external-signer-public-keys parameter now accepts external-signer value. It will enable public key retrieval from external signer standard API, making sure that configured keystore and trustStore will be used, if any.
• Stopped calling engine_exchangeTransitionConfigurationV1 Engine API method, since the method will be deprecated in the future.

Bug Fixes

• Fix Get Attestation Rewards API to fetch beacon state instead of block and beacon state (fixes #7338)

Upcoming Breaking Changes

• Upgrading the minimum Java version to Java 17, which means users will need to upgrade their Java installation to at least Java 17. The docker versions relying on jdk16 will no longer be published, so docker users explicitly referencing the jdk16 build need to update their package to reference jdk17, as tags develop-jdk16, develop-jdk16-arm64, latest-jdk16 will no longer be updated.

23.6.1

This is a required update for Gnosis network operators. It contains Gnosis Capella fork configuration.

Downloads

• Available as 23.6.1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 4a33cf38265ba1b5eb5142eaad230d4b3aaf5130bfaf54d7d37cca9eac10d64c)
  • zip (sha256: 81106eaf1ac11710a47459ee2a5e763478cbbe0e618fdfeb72995062c65e30fa)

Breaking Changes

• The Development options --Xprogressive-balances-mode and --Xee-version have been removed and will no longer be recognised as command line options.
• Network configs were updated following concensus-specs changes. If you run custom network, you will need to add lines with network parameters to your custom config (including the Deneb constants if required). If you are using a remote validator auto network feature, you will need to update both Beacon Node and Validator Client.

Additions and Improvements

• Added Gnosis configuration for the 🦉 Capella 🦉 network fork due at epoch 648704, UTC Tue 01/08/2023, 11:34:20
• Updated Sigmaprime's bootnodes
• Increased the pending pool size for attestations

Upcoming Breaking Changes

• Upgrading the minimum Java version to Java 17, which means users will need to upgrade their Java installation to at least Java 17. The docker versions relying on jdk16 will no longer be published, so docker users explicitly referencing the jdk16 build need to update their package to reference jdk17, as tags develop-jdk16, develop-jdk16-arm64, latest-jdk16 will no longer be updated.

23.6.0

This is a recommended update containing the rewards beacon-api endpoints, and an updated fix for the May non finality issue.

NOTE: Nethermind users on 1.19.X need to be at a minimum version of 1.19.2, due to compatibility issues with 1.19.0, 1.19.1.

Downloads

• Available as 23.6.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: aacd4a4b8396d793163e2cf061146bd1d5436e95440b2f4a8653ef6bc37b4707)
  • zip (sha256: 948abc377dd33560ead4ce94b634372b59c1a6fd61d2f9f92117c26454a0615f)

Breaking Changes

• The command argument --Xengine-exchange-capabilities has been removed, update to use --exchange-capabilities-enabled if you are using this option.
• The command argument --Xdeposit-snapshot-enabled has been removed, just remove it from commandline/configuration if you use it, updated argument --deposit-snapshot-enabled defaults to true now.
• The /eth/v1/debug/beacon/heads endpoint has been removed in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads.
• The /eth/v1/validator/blocks/:slot endpoint has been removed in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot.
• The /eth/v1/beacon/blocks/:block_id endpoint has been removed in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been removed in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/validator/liveness/:epoch endpoint was requiring the wrong body input and now conforms to the beacon-api spec.
• When --rest-api-enabled option is not specified and --rest-api-port is, --rest-api-enabled will now be set as true.
• Log messages previously printing WEI for builder bids and local payload values have been updated to print the ETH value.

Additions and Improvements

• Add support for Chiado (Gnosis testnet): --network=chiado
• Added beacon-apis rewards endpoints.
• Removed the experimental flag from /eth/v1/validator/liveness/:epoch rest-api endpoint.
• Following on from the non finality issue, improved state selection during attestation validation,
  where we failed to correctly identify that we could use the head state for validation of canonical attestation gossip.

Bug Fixes

• Fix a race condition on EL api result handling which may lead to beacon node remain syncing forever

Upcoming Breaking Changes

• Upgrading the minimum Java version to Java 17, which means users will need to upgrade their Java installation to at least Java 17. The docker versions relying on jdk16 will no longer be published, so docker users explicitly referencing the jdk16 build need to update their package to reference jdk17, as tags develop-jdk16, develop-jdk16-arm64, latest-jdk16 will no longer be updated.

Incident Statement - May 2023

As has been well documented by a post-mortem conducted by the Prysm team,
the Consensus layer suffered short periods of non-finality on the 11th and 12th of May, with the 12th of May being notable in that the network started an Inactivity leak.

To mitigate the problem, the Teku team released the 23.5.0 patch, which basically avoided processing some attestations due to
the observed behaviour of a number of clients suffering from memory contention if these attestations were fully validated by Teku.

This release contains a fix for the root cause of the issue, where the processing bug contributing to the original issue has been
rectified, greatly reducing the need for many states to be present while validating gossip.

23.5.0

This is a recommended update containing old attestation filtering.

Downloads

• Available as 23.5.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 9da9fc6c3a0e7ebbfe7c4599295c76113b3631c49da513a677dc35326c5a1a41)
  • zip (sha256: 725144706e5dddc1e5d1003a7e1c8ef12f2e815c93cda5273f109056bb6aa7b1)

Breaking Changes

Additions and Improvements

• Set User-Agent header to "teku/v" (e.g. teku/v23.5.0) when making builder bid requests to help builders identify clients and versions. Use --builder-set-user-agent-header=false to disable.
• Included more context when a request to an external signer fails.
• Added /eth/v1/beacon/rewards/blocks/{block_id} rest api endpoint.

Bug Fixes

• Filtering attestations with old target checkpoint

Upcoming Breaking Changes

• Upgrading source code to Java 17 meaning users will need to upgrade their Java install to at least 17, or use the jdk17 variant of the docker image.
• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads
• The command argument --Xengine-exchange-capabilities will be removed, update to use --engine-exchange-capabilities if you are using this option.
• The command argument --Xdeposit-snapshot-enabled will be removed, just remove it from commandline/configuration if you use it, updated argument --deposit-snapshot-enabled defaults to true now.

23.4.0

This is a recommended update for mainnet, containing a fix for the slow processing of blocks with multiple voluntary exit operations.

Due to some issues with Docker Hub signing service, 23.4.0 docker images have not been signed.

Downloads

• Available as 23.4.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 86b3e673682f410c3c9111dff005b34ff6a787757dccebb893a5ebccf5cb7aa2)
  • zip (sha256: 6bf588011c6fa987e173b3807760f6d0c06cf99be7999422d6b40e5653c5acfa)

Breaking Changes

• The --p2p-discovery-site-local-addresses-enabled option is set to false by default. If you use the client's discovery inside the local network, update its launch command to toggle the option.

Additions and Improvements

• Added the ability for the CL client to negotiate engine capabilities with the connected EL client. This can be disabled by setting --exchange-capabilities-enabled=false if Shanghai is not supported on your EL client.
• Enabled peer scoring by default. Can be disabled explicitly using --Xp2p-gossip-scoring-enabled=false
• When failovers are configured, the validator client will perform a readiness check on startup to avoid retrieving validator statuses from a node which is not ready.
• Enabled deposit tree snapshot bundles for major networks and persists it after finalization to decrease EL pressure and speed up node startup. Use --deposit-snapshot-enabled=false to disable.
• Optimized validator exit processing during state transition, to speed up block import containing multiple validator exits.
• Locally submitted exits and bls changes will now periodically broadcast if they are not actioned, to address operations being lost in remote pools.

Bug Fixes

• Included All forks in fork schedule if they're defined in configuration.
• • Operation pool not removing some Voluntary Exit messages after block inclusion.
• Slow processing of blocks with multiple voluntary exit operations.

Upcoming Breaking Changes

• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads
• The command argument --Xexchange-capabilities-enabled will be removed, update to use --exchange-capabilities-enabled if you are using this option.
• The command argument --Xdeposit-snapshot-enabled will be removed, just remove it from commandline/configuration if you use it, updated argument --deposit-snapshot-enabled defaults to true now.

23.3.1

This is a required update containing mainnet 🦉CAPELLA🦉 fork configuration. All users should upgrade as soon as possible.
NOTE: Upgrade your EL client to the latest version first! This update could be incompatible with old versions.

Known Issues

• CVE-2022-1471 security alert related to snakeyaml-1.33.jar could be ignored because Teku doesn't make use of the affected functionalities. Future dependency updates will fix this security alert.

Downloads

• Available as 23.3.1 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: f774fab2b6b5804a1619d129004805c722a07cb9bece1894c3558b2528ac300b)
  • zip (sha256: fadc91e193923897930a4d2f81290c45c949749754a5fe2fe27962db9729a228)

Breaking Changes

• --Xbeacon-liveness-tracking-enabled has been removed in favour of --beacon-liveness-tracking-enabled

Additions and Improvements

• The commandline option --network of the validator-client subcommand has been undeprecated for values other than auto.
• Send validator registrations to the Beacon node when the Validator client has reconnected to the event stream
• Added optional query parameters require_prepared_proposers and require_validator_registrations to the teku/v1/admin/readiness endpoint
• Added mainnet configuration for the CAPELLA network fork due at epoch 194048, April 12, 2023; 10:27:35pm UTC
• Added experimental feature --Xdeposit-snapshot-enabled to use bundled deposit contract tree snapshot and persist it after finalization to decrease EL pressure when syncing and speed up node startup
• Added --p2p-discovery-site-local-addresses-enabled option to allow discovery connections to local (RFC1918) addresses.
• Added --builder-bid-compare-factor to be applied to the builder bid value when comparing it with locally produced payload.
• Improved Completed regeneration info message so that is more technically accurate.

Bug Fixes

• Fixed a bug in signature verification on fork boundaries when first blocks of the new fork are missing.

Upcoming Breaking Changes

• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads
• The --p2p-discovery-site-local-addresses-enabled option will be set to false by default. If you use the client's discovery inside the local network, update its launch command to toggle the option.

23.3.0

This is an optional update that primarily updates dependent libraries. It is a required update for Goerli network nodes.

Downloads

• Available as 23.3.0 on Dockerhub
• Download the binary distribution:
  • tar.gz (sha256: 82c292b78c9788ba1022ffe43cfd7d41eb8648fdb91bc541cde60629f2c7c4ee)
  • zip (sha256: 882fdf054cf47b4fe6a6e890036d9074f53f8c939f717efb8565394ffaf1826b)

Additions and Improvements

• Trigger the beacon nodes fallback mechanism when the execution layer is offline
• --beacon-liveness-tracking-enabled option (disabled by default) can be used instead of --Xbeacon-liveness-tracking-enabled
• Added an optional query parameter locally_submitted to /eth/v1/beacon/pool/bls_to_execution_changes to allow users to query only bls changes submitted to the current node.
• Added /eth/v1/builder/states/{state_id}/expected_withdrawals rest api endpoint.
• Added /eth/v1/beacon/rewards/sync_committee/{block_id} rest api endpoint.
• Added Capella fork information for Goerli network configuration.
• Added validator_scheduled_sync_committee_duties_current metric - the number of validators that you have active with current sync committee duties
• Added validator_current_sync_committee_last_epoch metric - the last epoch of the current sync committee.

Bug Fixes

• Included All forks in fork schedule if they're defined in configuration.

Upcoming Breaking Changes

• The /eth/v1/debug/beacon/states/:state_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/debug/beacon/states/:state_id
• The /eth/v1/beacon/blocks/:block_id endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/beacon/blocks/:block_id
• The /eth/v1/validator/blocks/:slot endpoint has been deprecated in favor of the v2 Altair endpoint /eth/v2/validator/blocks/:slot
• The /eth/v1/debug/beacon/heads endpoint has been deprecated in favor of the v2 Bellatrix endpoint /eth/v2/debug/beacon/heads
• The commandline option --network of the validator-client subcommand has been undeprecated and can be used to select a network for standalone validator clients. When set to auto, it automatically
  fetches network configuration information from the configured beacon node endpoint.
• --Xbeacon-liveness-tracking-enabled option will be removed. The --beacon-liveness-tracking-enabled option should be used instead (disabled by default)