Refs paritytech/smoldot#1712
Impl libp2p/rust-libp2p#2622

- WebRTC transport is enabled for non-validators and developers by default.
- The transport will generate and store the certificate, which is required for WebRTC identity, in
base dir. In the future, when a new version of `ring` library is released, the certificate will be
deterministically derived from the node's peer ID.

paritytech#12529 (review)

* Disallow burning externally locked nfts

* Update docs

* Make try-runtime checks selectable

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/support/src/traits/try_runtime.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Add Clap wrapper for enum UpgradeCheckSelect

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Revert "Add Clap wrapper for enum UpgradeCheckSelect"

This reverts commit e29538c.

* fix pools sanity check

* Set default for --checks to None

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Make --checks backwards comp

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add clap attr comment

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: kianenigma <kian@parity.io>

* grandpa: pass the actual best block to voting rules

* grandpa: add test for checking best header is passed to voting rule

* Remove Copy from EnsureOp and EnsureOpAssign

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Remove Copy from EnsureFrom and EnsureInto

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix default impl

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Reuse assignment code in Ensure trait

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Require Ensure for all BaseArithmetic types

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix assign impls

Co-authored-by: Luis Enrique Muñoz Martín <lemunozm@gmail.com>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Add success doc tests

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Aura: Do not verify on state import

When we import the state, we can not fetch authorities to verify the seal etc. So, we can directly
skip any verification.

* Skip checks as well for gap sync

* Update client/consensus/aura/src/import_queue.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Review comment

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* upgrade libp2p to 0.50.0

* on_swarm_event and on_connection_handler_event

* replace `Swarm::new` with `Swarm::with_threadpool_executor`

* on_swarm_event and on_connection_handler_event part 2

* on_swarm_event and on_connection_handler_event part 3

* on_swarm_event and on_connection_handler_event part 4

* update libp2p

* libp2p 0.50.0

* rename OutboundQueryCompleted to OutboundQueryProgressed

refs libp2p/rust-libp2p#2712

* remove unused var

* accumulate outbound_query_records until query is finished

* format code

* use p_handler instead of new_handler

paritytech#12734 (comment)

* pass ListenFailure to kademlia

paritytech#12734 (comment)

* use tokio executor in tests

paritytech#12734 (comment)

* use chrono Local::now

instead of deprecated Local::today

* remove unused vars from request_responses tests

* attempt to fix pallet UI tests

* restart CI

* restart CI

* restart CI

* restart CI

* restart CI

* restart CI

* restart CI

* restart CI

* grandpa: remove deprecated afg log target

* grandpa: define log targets in primitives

…ytech#13066)

* Adds test

* Ensure we are using the runtime version of the override/substitute wasm

* Update client/service/src/client/call_executor.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

Co-authored-by: Anton <anton.kalyaev@gmail.com>

Before libp2p 0.50.0 we used a quorum of one to fetch records from the DHT. In the pr that upgraded
to libp2p 0.50.0 we accidentally changed this behavior. This pr brings back the old behavior of
using a qorum of one and thus, a faster discovery. After finding the first value, we directly finish
the query. There was also another behavior change in libp2p, they stopped automatic caching on
remote nodes. This pr also brings back the remote caching on nodes that are nearest to the key from
our point of view of the network.

The pr that changed the behavior in libp2p: libp2p/rust-libp2p#2712

…_env]` macro (paritytech#13032)

* macro to expand traits for host functions documentation

* other way: same Doc trait in seal modules

* added docs for macro, and remove `doc` attribute

* fmt

* Apply suggestions from code review

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* make docs to be generated into re-exported `api_doc` module; fix
unrelated elder docs;

* make it compile without `doc` attr passed to macro

* make alias functions indicated explicitly in docs

* tidy up docs

* refactored a bit

* macro to auto-add doc warning for unstable functions

* invoke macro with no doc generation by default

* addressed review comments

* hide api_doc module behind cfg(doc)

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Update trybuild to avoid random test failures.

A feature was added to trybuild >1.0.70 avoid failing on different variant
counts in the line `and 278 others`

fixes paritytech#12955

* Update all trybuild deps to latest version

* Update Cargo.lock

…itytech#12790)

Co-authored-by: parity-processbot <>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Run frame_system integrity tests in Externalities

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Use feature = 'std'

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* EPM and staking events improvement

* Uses RawOrigin in ElectionCompute event

* Refactors new phase events to PhaseTransition event

* PhaseTransitioned and remove RawOrigin from event

* Adds helpers for epm phase transition and staking force new

* addresses review comments

* nit: removes unecessary clone

* fixes benchmarks

Co-authored-by: parity-processbot <>

…ch#13104)

* add ensure_pow method

* reexport checked_pow and ensure_pow

* Add solution hint to error message

* make it compile

* Update frame/support/procedural/src/pallet/expand/hooks.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/procedural/src/pallet/expand/storage.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/procedural/src/pallet/expand/storage.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/support/procedural/src/pallet/expand/storage.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

…aritytech#13088)

* avoid unintentionally canceling the scheduled crate publishing job

because publish-crates and publish-crates-manual share the resource group
"crates-publishing", any instance of publish-crates-manual cancels a running
instance of publish-crates, as demonstrated by
https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2212179

a workaround for that unintended interaction is to avoid creating instances of
publish-crates-manual and instead require pipelines to be triggered manually by
checking $CI_JOB_MANUAL == "true"

* check manual pipelines by $CI_PIPELINE_SOURCE instead of $CI_JOB_MANUAL

* make crate-publishing pipelines uninterruptible

* use conditional includes to work around interruptible limitations

* organize comments

* remove interruptible from common pipeline

* wip: check include

* wip: check include

* fix include

* fix include

* fix include

* fix yaml

* fix yaml

* remove shared common-pipeline

* wip: retry common-pipeline

* move .default-template to .gitlab-ci.yml

* fix the pipeline

add comments

* fix default-pipeline.yml

* revert publish-crates-manual to when: manual

* move "needs:" back to publish-crates

* avoid manual repetition

* improve previous commit

* try to avoid manual repetition

* fix indentation

* minor adjustments

* move defaults to top of .gitlab-ci.yml

* fix positioning on default in the diff

* comments

* indentation

* Apply suggestions from code review

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

* introduce log-target constant to more frame pallets

* cargo fmt

* make LOG_TARGET in session public

* Update frame/elections-phragmen/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/elections-phragmen/src/migrations/v3.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/elections-phragmen/src/migrations/v3.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update frame/elections-phragmen/src/migrations/v3.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* move LOG_TARGET=runtime::session_historical to migrations module, where it's actually used

Co-authored-by: Bastian Köcher <git@kchr.de>

…3118)

* Update UI tests for 1.66

* Fix `test_enum` assertion for Rust 1.66

* Fix another `test_enum` assertion for Rust 1.66

* Fix another `test_enum` assertion for Rust 1.66

* Fix another `test_enum` assertion for Rust 1.66

paritytech#12529 (comment)

will be moved to polkadot's pipeline

ref https://github.com/paritytech/release-engineering/issues/142

* Removed `has_duplicates` check form the `deposit_event`.
Removed the usage of `Error::DuplicateTopics`.

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_contracts

Co-authored-by: command-bot <>

* Refactor do_mint()

* Track the depositor of item's metadata

* Revert back the access control

* On collection destroy return the metadata deposit

* Clear the metadata on item burn returning the deposit

* Address comments

* Fix clippy

* Don't return Ok on non-existing attribute removal

)

* `IntegrityTest` implementation should be feature gated

The initial implementation for the old declarative macros is still feature gating the
implementation. As we only call this in a test, there is no need to have this compiled for wasm.

* Don't assume that all "consumers" have a `std` feature

Bumps [git2](https://github.com/rust-lang/git2-rs) from 0.14.4 to 0.16.0.
- [Release notes](https://github.com/rust-lang/git2-rs/releases)
- [Commits](rust-lang/git2-rs@0.14.4...git2-curl-0.16.0)

---
updated-dependencies:
- dependency-name: git2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Scheduler is already at V4

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix Referenda log target

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

…tech#12857)

* Sort hashes when generating & verifying MMR proofs

* Remove unused variables

* Double-hash leaves in beefy-merkle-tree

* Revert "Double-hash leaves in beefy-merkle-tree"

This reverts commit f788f5e.

* Retry Polkadot companion CI jobs

Minor change to logs but required to minimize spamming during initial BEEFY deployment.

* Stop keeping track of epoch changes data within the sync gap

* Fix docs

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Fix typo

Co-authored-by: Bastian Köcher <git@kchr.de>

…tech#13131)

Still allows custom message hasher, but ties together the crypto
types used for private+public keys and the signature.

Signed-off-by: acatangiu <adrian@parity.io>

…#13144)

* pallet-offences-benchmarking: Box events in verify

Events in frame are represented by an enum in the pallet and the runtime. The size of an enum in
Rust depends on the size of biggest variant. This means we always need to allocate memory for the
biggest variant when allocating memory for an event. The offences benchmarking is verifying the
benchmarking results by checking the events. To check the events it is generating all the expected
events. With the recent changes in Polkadot the events are too big and lead to issues when running
this verify functions. The solution is to box each event, as the vector holding all the events will
then only need to hold fat pointers * expected events, instead of size_of(event) * expected events.
This issue isn't a problem in production, as we never read the events on chain. When we are reading
the events, it is done in an offchain context and they are only decoded one by one.

Besides that this also enables the benchmarking verification for everyone running these benchmarks.

* FMT

* Disable checking again

* more improvements for the crate publishing pipeline

* move default definitions to the publish-crates script

* add script to check the crate publishing pipeline at the start

* fix yaml references

* move more variables to .crates-publishing-pipeline

* separate .crates-publishing-pipeline from .crates-publishing-variables

* clean up redundant and unused code

* txpool: don't maintain the pool during major sync

Fix shall prevent from wasting the CPU during the major sync. No actions
are actually required in transaction pool during the major sync.

Fixes: paritytech#12903

* passing sync_oracle to maintain method

* fixed: builder, txpool tests

* do not maintain tx-pool if node gone out of sync

* EnactmentAction: all logic moved to EnactmentState

Tests to be done.

* maintain guard logic moved directly to MaintainedTransactionPool

* minor fixes

* EnactmentAction: all logic moved to EnactmentState (again)

* SyncOracle fixes here and there

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* sync_oracle removed

* spelling + fmt + doc

* Review suggestions applied

* log::info -> debug

* Update client/transaction-pool/src/enactment_state.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <git@kchr.de>

* join dns with another instance of WS transport

Secure Websocket transport needs unresolved addresses, so we join DNS transport with
yet another instance of Websocket transport.

Closes paritytech#12024

* WSS transport itself need to wrap DNS transport

in order to resolve addresses before passing them down to TCP transport

Refs libp2p/rust-libp2p#3330

* reverse order

* simplify code: remove WS from WSS inner DNS transport

* remove the 2nd instance of WS transport

* Keep funds with receipt holder

* Counterpart is optional

* Use named reserves

* Tests

* Benchmarks

* Fixes

* Update frame/nis/src/lib.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Update frame/nis/src/lib.rs

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* Update frame/nis/src/lib.rs

* Update frame/nis/src/lib.rs

* Update frame/nis/src/tests.rs

* Update frame/nis/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/nis/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/nis/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update frame/nis/src/lib.rs

* Update frame/nis/src/lib.rs

* Update frame/nis/src/lib.rs

* Update frame/nis/src/lib.rs

* Formatting

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Refactory of `next_slot` method

* Prevents slot worker exit if inherent data provider creation fails
* Failure is not possible anymore
* Fix potential failure after warp-sync where block headers of not already downloaded blocks are used by the inherent data provider

* break out moch runtimes to separate files

* tranaction-payment: break out tests & mock runtime to separate files

…ech#13161)

The macros in frame-benchmarking relied on having all the macros imported, which isn't a behavior
for a proper macro :D This pr fixes this by making all internal macro usages absolute.

)

* Have `KeyIterator` clone the `prefix` it receives

* Stream keys in `storage_size` RPC and add a runtime limit

* Update client/rpc/Cargo.toml

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update client/rpc/src/state/utils.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* Rename the types to signify that the cancellation is due to a timeout

* Move the test into a `mod tests`

* Add a comment regarding `biased` in `tokio::select`

* Make the `clone` explicit when calling `KeyIterator::{new, new_child}`

Co-authored-by: Bastian Köcher <git@kchr.de>

…ytech#12976)

* update RuntimeCosts to Weights V2, update tests

* improve docs

* clearer naming and docs to compat_weight helper

* Apply suggestions from code review

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* save before master merge

* HostFnWeights to Weight

* added to_weight! macro

* Apply suggestions from code review

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* RuntimeCosts::ChainExtension to weight_v2

* chain extension to weight v2

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* zobmienet tests are not supposed to fail

* Update scripts/ci/gitlab/pipeline/zombienet.yml

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>

* integrity test for MaxCodeLen and CallStack::len()

* integrity test for MaxDebugBufferLen

* addressed review comments

* fix append_debug_buffer()

* ci fix

* updated code_len_limit formula after further discussion

* enlarged mem safe margin after discussion

* +doc to Config trait associated types

* Apply suggestions from code review

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* more lil fixes from code review feedback

* lowered max call depth to satisfy mem limits

* fix node runtime pallet params to satisfy integrity check

* fix max call depth value calc

Co-authored-by: Alexander Theißen <alex.theissen@me.com>

* Expose `UnknownBlock` error via `ApiError`

In [certain cases](https://github.com/paritytech/polkadot/issues/5885) a
runtime api is called for an unknown block. For example a block which is
already pruned or on an abandon fork.

In such cases the correct error is returned but it is wrapped in
`ApiError::Application` and the only way to figure out what is the
problem is to inspect the actual message in the error. In polkadot for
example this usually happens when the runtime api version is being
queried. It's beneficial to be able to clearly separate such errors so i
that when they occur the client side can handle them more gracefully.
E.g. log less stressful error message than `State already discarded for
BlockId` or cancel any pending work related on this block.

* Update primitives/api/src/lib.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

* Worker

* Reorganize and unpin onnotification drop

* Pin in state-db, pass block number

* Pin blocks in blockchain db

* Switch to reference counted LRU

* Disable pinning when we keep all blocks

* Fix pinning hint for state-db

* Remove pinning from backend layer

* Improve readability

* Add justifications to test

* Fix justification behaviour

* Remove debug prints

* Convert channels to tracing_unbounded

* Add comments to the test

* Documentation and Cleanup

* Move task start to client

* Simplify cache

* Improve test, remove unwanted log

* Add tracing logs, remove expect for block number

* Cleanup

* Add conversion method for unpin handle to Finalitynotification

* Revert unwanted changes

* Improve naming

* Make clippy happy

* Fix docs

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>

* Use `NumberFor` instead of u64 in API

* Hand over weak reference to unpin worker task

* Unwanted

* &Hash -> Hash

* Remove number from interface, rename `_unpin_handle`, LOG_TARGET

* Move RwLock one layer up

* Apply code style suggestions

* Improve comments

* Replace lru crate by schnellru

* Only insert values for pinned items + better docs

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Improve comments, log target and test

Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>

* Make DispatchError impl MEL

* Upgrade SCALE codec to support `codec(skip)` for MEL

Co-authored-by: Bastian Köcher <info@kchr.de>

* txpool: LOG_TARGET const added

part of: paritytech#12873

* LOG_TARGET added to tests mod

* txpool::api for api

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>

…aritytech#13183)

* Fix potential huge allocation as a result of `validate_block` output

* Address review comments; add more tests

* Update client/executor/wasmtime/src/runtime.rs

* Remove unnecessary comments

Co-authored-by: Bastian Köcher <git@kchr.de>

…ytech#13185)

When running with `--no-private-ipv4` the node should not trying to connect to any private ip
addresses. With the switch to libp2p this behavior was broken. Part of this version upgrade was the
following pr: libp2p/rust-libp2p#2995. This pr changed the default cache
size of `libp2p-identity` from `0` aka disabled to `100`. Together with our implementation that was
calling into `identity` to request addresses for a given peer. Before the switch to libp2p 0.50.0
this was returning zero addresses, but now with the cache enabled it started to return addresses.
This pr fixes this by only letting discovery return addresses for a peer. It also ensures that we
filter private addresses if requested. The cache is also disabled to restore the previous caching
behavior, but it will actually not be called anymore.

* move Metric

* run hardware bench if validiator flag is being used

* fix rustdoc & update node-template

* fix

* unused improt

* warn

* move Requirement

* bench_result

* ensure_requirements

* make the code compile

* check if authority

* Update client/sysinfo/src/sysinfo.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* nit fixes

* warning signs

* Update client/sysinfo/src/sysinfo.rs

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Move slow hardware warning print logic to CLI

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update client/sysinfo/src/sysinfo.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* fmt

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>

The `authoring_blocks` test of BABE was calculating the slot based on the timestamp it sometimes
failed in CI. The problem is that we combine all the notifications and authoring futures in one big
future. This one big future may first polls one authoring future to build a block. Then it polls all
notification futures again to import the block. Then some other authoring future is polled and
builds on the imported block using the same slot and making the import fail. The solution is that we
just artificially increase the slot to make the test work.

…12979)

* improve debug info in assert_has_event and assert_last_event

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

Co-authored-by: Bastian Köcher <git@kchr.de>

* add stub for new benchmark macro

* benchmark syntax

* add #[extrinsic call] separator

* parse #[benchmark] item as a function

* proper emission of error when #[extrinsic_call] annotation is missing

* clean up

* enclosing module via benchmarks! { } working

* use an attribute macro on the module instead of benchmarks! { }

* cargo fmt

* working component implementation

* WIP

* working

* add syntax for Linear<A, B>

* parsing of param ranges (still need to build tuple though)

* params parsing WIP

* clean up (don't need extrinsic call name)

* use proper Result syntax for BenchmarkDef parsing

* proper parsing of Linear<0, 1> style args

* successfully parse and make use of linear component ranges 💥

* rename support variable => home because eventually will be moved

* compile-time check that param range types implement ParamRange

* switch to using balances as example, failing on instance pallet

* successfully set up __origin and __call with balances 💥

* clean up

* use a module

* don't need a variable for transfer

* rename benchmark_transfer -> transfer because no longer conflicts

* clean up

* working with transfer_increasing_users as well 💥

* re-add BareBlock

* add comments for undocumented structs+functions+traits

* refactor in preparation for removing module requirements

* switch to a block instead of a module

* use the outer macro pattern to to enable #[benchmarks] aggregation

* successfully generate SelectedBenchmark 💥

* implement components for SelectedBenchmark

* implement instance for SelectedBenchmark

* properly track #[extra]

* working impl for fn benchmarks()

* run_benchmarks WIP

* finish run_benchmark! impl 💥

* import balances transfer_best_case benchmark

* import transfer_keep_alive balances pallet benchmark

* import set_balance_creating balances pallet benchmark

* import set_balance_killing balances pallet benchmark

* import force_transfer balances pallet benchmark

* add #[extra] annotation and docs to transfer_increasing_users

* import transfer_all balances pallet benchmark

* import force_unreserve balances pallet benchmark

* prepare to implement impl_benchmark_test_suite!

* ensure tests cover #[extra] before and after #[benchmark] tag

* refactor

* clean up

* fix

* move to outer

* switch to benchmarks/instance_benchmarks

* test impl almost done, strange compiler error

* benchmark test suites working 💥

* clean up

* add stub and basic parsing for where_clause

* working except where clause and extrinsic calls containing method chains

* assume option (2) for now wrt paritytech#12924 (comment)

* clean up

* switch to attribute-style

* properly handle where clauses

* fix subtle missing where clause, now just MessageQueue issues

* fix block formatting in message-queue pallet

* switch to block vs non-block parsing of extrinsic call

* working now but some benchmark tests failing

* message-queue tests working (run order issue fixed) 🎉

* add comments and internal docs for fame_support_procedural::benchmark

* fix license years

* docs for lib.rs

* add docs to new support procedural macros

* don't allow #[benchmark] outside of benchmarking module

* add docs

* use benchmark(extra, skip_meta) style args

* update docs accordingly

* appease clippy

* bump ci

* add notes about `extra` and `skip_meta`

* fix doc tests

* re-run CI

* use `ignore` instead of `no_run` on doc examples

* bump CI

* replace some if-lets with if-elses

* more refactoring of if-let statements

* fix remaining if-lets in BenchmarkDef::from()

* fix if-lets in benchmarks()

* fix remaining if-lets, use nested find_map for extrinsic call

* switch to use #[extrinsic_call] or #[block] situationally

* refactor ExtrinsicCallDef => BenchmarkCallDef

* update docs with info about #[block]

* add macro stub for #[extrinsic_call]

* fix docs and add stub for #[block] as well

* remove unused extern crate line

* fix clippy nits

* Use V2 bench syntax in pallet-example-basic

Just testing the dev-ex...

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* carry over comment

* use curly-brace style for impl_benchmark_test_suite!

* remove unneeded parenthesis

* proper handling of _() extrinsic call style

* add docs for _() syntax

* fix crate access

* simplify keyword access

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* simplify module content destructuring

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* fix crate access "frame_benchmarking" => "frame-benchmarking", compiles

* use _() extrinsic call syntax where possible in balances

* simplify attr.path.segments.last()

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* fix compile error being suppressed

* simplify extrinsic call keyword parsing

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* use ? operator instead of return None

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* rename generics => type_use_generics
rename full_generics => type_impl_generics

* simplify extrinsic call extraction with transpose

* bump CI

* nit

* proper handling of too many + too few block/extrinsic call annotations

* change to B >= A

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* remove unneeded ignore

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* remove another ignore

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* add ui tests

* use _() style extrinsic call on accumulate_dummy

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* add range check to ParamRange

* ui test for bad param ranges

* fix failing example

* add ignore back to other failing example

* tweak expr_call span

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* fix typo

* eliminate a match

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* change pub fn benchmarks to return Result<TokenStream>

* fix origin error span

* more informative error for invalid benchmark parameter name

* fix spans on a few benchmark errors

* remove unneeded clone

* refactor inner loop of benchmark function parsing

* preserve mod attributes

* refactor outer loop of benchmark def parsing code, greatly simplified

* simplify to use a ? operator when parsing benchmark attr path

* fix another ? operator

* further simplify benchmark function attr parsing with more ? ops

* refactor extrinsic call handling to use if let rather than match

* replace is_ok => is_err

Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>

* re-use name during expansion of benchmark def

* remove unneeded clone

* fix span for origin missing error

* fix missing semi

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
Co-authored-by: parity-processbot <>

* Rename `*-private-ipv4` to `*-private-ip` CLI args

Renames the `*-private-ipv4` to `*-private-ip` in the CLI interface. The old names are staying as
alias, thus it will not break for anyone. Besides that it also fixes the naming in the rest of the code.

* FMT

* [Fix] CountedMap::set now takes Counter into account

* introduce tests for StorageMap

* BlockId removal: refactor: CallExecutor trait

It changes the arguments of CallExecutor methods:
-  `call`, 'contextual_call', 'runtime_version', 'prove_execution'

from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* ".git/.scripts/commands/fmt/fmt.sh"

Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>

* service: storage monitor added

Storage monitor added. It uses `notify` create to get notifications
about any changes to monitored path (which is database path).
Notifications are consumed in essential task which terminates when
available storage space drops below given threshold.

Closes: paritytech#12399

* Cargo.lock updated

* misspell

* fs events throttling added

* minor updates

* filter out non mutating events

* misspell

* ".git/.scripts/commands/fmt/fmt.sh"

* Update client/service/src/storage_monitor.rs

Co-authored-by: Anton <anton.kalyaev@gmail.com>

* storage-monitor crate added

* cleanup: configuration + service builder

* storage_monitor in custom service (wip)

* copy-paste bad desc fixed

* notify removed

* storage_monitor added to node

* fix for clippy

* publish = false

* Update bin/node/cli/src/command.rs

Co-authored-by: Dmitry Markin <dmitry@markin.tech>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* crate name: storage-monitor -> sc-storage-monitor

* error handling improved

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* publish=false removed

Co-authored-by: command-bot <>
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Co-authored-by: Bastian Köcher <git@kchr.de>

* LOG_TARGET const
* fmt

* pallet-assets: Rename `total_supply` to `amount`

We are actually passing the `amount` on assets being minted and not the total supply.

Closes: paritytech#13210

* ".git/.scripts/commands/fmt/fmt.sh"

* Fix compilation

* FMT

Co-authored-by: command-bot <>

…3135)

* Detect and correct epoch-index for skipped epochs

* Code refactory

* Epoch index should be also be fixed for secondary claims with VRF

* Fix typo

* Make clippy happy

* Fix typo

* Trigger pipeline

* Deprecate random interface

* Revert change to runtime file

* Fix docs

* Fix tests

* Rename to not_deprecated

* Apply suggestions from code review

Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>

* Deprecate `set_rent_allowance`

Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>

…ing functions (paritytech#13236)

* Add upfront weight of merkle trie proofs for storage reading functions

* drive-by fixes

* CI: Rewrite `check-each-crate` in python

This is a much better readable version of the script and it should also work on Macos and not
siltently fail ;)

* Fix dumb bugs and print everything to stderr

* Don't buffer Python output

* 🤦

* 🤦 🤦

* Use check all until we have more macos runners

* Update scripts/ci/gitlab/check-each-crate.py

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Update scripts/ci/gitlab/pipeline/test.yml

Co-authored-by: Vladimir Istyufeev <vladimir@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>

* Fix try-runtime with create-snapshot

* Apply review suggestions

* reduce exec time of fast-unstake benchmarks

* fix test

* fmt

* fix patch the tests

* fix patch the tests

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* add batch size as well

* update some benches to be better

* fix one last test

* fix

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* reduce time even more

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* fix tests

* nit

* remove

* improve the weight calc further

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* fix benchmarks

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* update

* fix

* fix

* fmt

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* lots of changes again...

* smaller input

* update

* fmt

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* cleanup

* small simplification

* fmt

* reduce exec time a bit

* fix

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* test

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* increase again

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

* review comments

* fmt

* fix

* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_fast_unstake

Co-authored-by: command-bot <>