Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[libp2p-dns] Use trust-dns-resolver (with either async-std or tokio). Remove thread pool. #1927

Merged
merged 9 commits into from Mar 16, 2021

Conversation

romanb
Copy link
Contributor

@romanb romanb commented Jan 15, 2021

This PR changes libp2p-dns to use the trust-dns-resolver library for DNS resolution, thereby removing the current use of the thread pool, i.e. closing #1235. This also serves as a first step for #1920, which will build on this work.

Since trust-dns-resolver and related crates already provide decent support for both async-std and tokio, we make use of that here in libp2p-dns, offering analogous async-std and tokio features. Just like with libp2p-tcp, the async-std variant is enabled by default. This PR is only pending a new release that includes hickory-dns/hickory-dns#1354, which is why it is still pointing to a git dependency. Also just like with libp2p-tcp, the libp2p-dns tests are exercised for both runtimes.

Since trust-dns-resolver provides many useful configuration options and error details, central types of trust-dns-resolver like ResolverConfig, ResolverOpts and ResolveError are re-exposed in the API of libp2p-dns. Full encapsulation
does not seem preferable in this case. We also re-export the (currently tokio-only) features for DNS-over-TLS and DNS-over-HTTPS (via rustls).

I also consolidated the top-level utility functions of the libp2p crate: There is now just fn development_transport() (available with default features) and fn tokio_development_transport() (available when the corresponding tokio features are enabled). I think that is sufficient and I removed the minor variation that also included pnet support into the development transport.

Use the `trust-dns-resolver` library for DNS resolution,
thereby removing current use of the thread pool.

Since `trust-dns-resolver` and related crates already
provide support for both `async-std` and `tokio`, we
make use of that here in our own feature flags.

Since `trust-dns-resolver` provides many useful
configuration options and error detail, central
types of `trust-dns-resolver` like `ResolverConfig`,
`ResolverOpts` and `ResolveError` are re-exposed
in the API of `libp2p-dns`. Full encapsulation
does not seem preferable in this case.
@romanb romanb linked an issue Jan 15, 2021 that may be closed by this pull request
Copy link
Member

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look good to me.

I don't hold a strong opinion on OS resolver vs. library resolver. In general I would consider DNS resolution a responsibility of the OS, at the same time https://github.com/bluejekyll/trust-dns seems to be well established and thus worth betting on.

@@ -264,35 +261,27 @@ pub use self::simple::SimpleProtocol;
pub use self::swarm::Swarm;
pub use self::transport_ext::TransportExt;

/// Builds a `Transport` that supports the most commonly-used protocols that libp2p supports.
/// Builds a `Transport` based on TCP/IP that supports the most commonly-used features of libp2p:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am very much in favor of this cleanup.

transports/dns/Cargo.toml Outdated Show resolved Hide resolved
transports/dns/src/lib.rs Outdated Show resolved Hide resolved
transports/dns/src/lib.rs Outdated Show resolved Hide resolved
Copy link
Member

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me.

I would suggest waiting for hickory-dns/hickory-dns#1354 to be released before merging here. What do you think?

@romanb
Copy link
Contributor Author

romanb commented Jan 20, 2021

Yes, of course. I am anyway already working on the follow-up PR for /dnsaddr. Regarding your comment of OS resolver vs library resolver, apart from more features and better configurability from the code, a main motivation for introducing trust-dns-resolver here is that it makes it easy to do the necessary TXT lookups for /dnsaddr. I also think that these DNS lookups via trust-dns-resolver will actually perform better as they're more targeted to specific record types, as opposed to the opaque use of ToSocketAddrs from a background thread.

@romanb
Copy link
Contributor Author

romanb commented Feb 17, 2021

Unfortunately still pending a new release of async-std-resolver.

@mxinden
Copy link
Member

mxinden commented Mar 11, 2021

Unfortunately still pending a new release of async-std-resolver.

@romanb are you aware of a timeline for this release to happen? I am happy to reach out to the authors myself. It is not urgent, though it would be nice to build on top of the general changes introduced via #1931. In addition it would be a bummer for this pull request and current master to drift apart.

@romanb
Copy link
Contributor Author

romanb commented Mar 11, 2021

are you aware of a timeline for this release to happen?

No, but based on the prior release history of the crate I was indeed hoping for a new release some time in February. It would be nice if you could reach out.

@romanb
Copy link
Contributor Author

romanb commented Mar 16, 2021

trust-dns-0.20.1 has been released, so we can go ahead. Thanks for following up on it @mxinden.

@romanb romanb merged commit cd15bc9 into libp2p:master Mar 16, 2021
@romanb romanb deleted the trust-dns branch March 16, 2021 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

deactivating dns-feature doesn't compile Don't use a thread pool for DNS resolution
2 participants