Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add utils to create and prepare socket for tcp client #31009

Merged
merged 16 commits into from
Sep 22, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -2780,6 +2780,7 @@ grpc_cc_library(
"src/core/lib/event_engine/posix_engine/tcp_socket_utils.h",
],
external_deps = [
"absl/cleanup",
"absl/status",
"absl/status:statusor",
"absl/strings",
Expand Down
1 change: 1 addition & 0 deletions CMakeLists.txt

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions build_autogenerated.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

87 changes: 87 additions & 0 deletions src/core/lib/event_engine/posix_engine/tcp_socket_utils.cc
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

#include <grpc/support/port_platform.h>

#include "src/core/lib/event_engine/posix_engine/tcp_socket_utils.h"

#include <arpa/inet.h>
#include <errno.h>
#include <inttypes.h>
Expand Down Expand Up @@ -43,6 +45,7 @@
#include <atomic>
#include <cstring>

#include "absl/cleanup/cleanup.h"
#include "absl/status/status.h"
#include "absl/strings/str_format.h"

Expand Down Expand Up @@ -100,6 +103,49 @@ int CreateSocket(std::function<int(int, int, int)> socket_factory, int family,

const uint8_t kV4MappedPrefix[] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0xff};

absl::Status PrepareTcpClientSocket(PosixSocketWrapper sock,
const EventEngine::ResolvedAddress& addr,
const PosixTcpOptions& options) {
absl::Status status;
auto sock_cleanup = absl::MakeCleanup([&status, &sock]() -> void {
if (!status.ok()) {
close(sock.Fd());
}
});
status = sock.SetSocketNonBlocking(1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend generalizing the RETURN_IF_ERROR macro in src/core/ext/transport/binder/wire_format/wire_reader_impl.cc, it will make this much more readable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this just made another copy of it. My recommendation is to put it in some central location (so it exists exactly once somewhere) and reuse it across gRPC.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved it to src/core/lib/gprpp/status_helper.h and modified all use cases.

if (!status.ok()) {
return status;
}
status = sock.SetSocketCloexec(1);
if (!status.ok()) {
return status;
}

auto is_unix_socket = [](const EventEngine::ResolvedAddress& addr) -> bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why this needs to be a lambda or function, the logic can be inlined here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

const sockaddr* sock_addr =
reinterpret_cast<const sockaddr*>(addr.address());
return sock_addr->sa_family == AF_UNIX;
};

if (!is_unix_socket(addr)) {
status = sock.SetSocketLowLatency(1);
if (!status.ok()) {
return status;
}
status = sock.SetSocketReuseAddr(1);
if (!status.ok()) {
return status;
}
sock.TrySetSocketTcpUserTimeout(options, true);
}
status = sock.SetSocketNoSigpipeIfPossible();
if (!status.ok()) {
return status;
}
return sock.ApplySocketMutatorInOptions(GRPC_FD_CLIENT_CONNECTION_USAGE,
options);
}

#endif /* GRPC_POSIX_SOCKET_UTILS_COMMON */

} // namespace
Expand Down Expand Up @@ -771,6 +817,39 @@ absl::StatusOr<PosixSocketWrapper> PosixSocketWrapper::CreateDualStackSocket(
return PosixSocketWrapper(newfd);
}

absl::StatusOr<PosixSocketWrapper>
PosixSocketWrapper::CreateAndPrepareTcpClientSocket(
PosixTcpOptions& options, const EventEngine::ResolvedAddress& target_addr,
EventEngine::ResolvedAddress& output_mapped_target_addr) {
PosixSocketWrapper::DSMode dsmode;
absl::StatusOr<PosixSocketWrapper> status;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest renaming this variable to posix_socket_wrapper. Best practice is to name StatusOr variables as the variable type, not the status. See http://go/totw/labs/statusor-patterns-and-antipatterns

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.


// Use dualstack sockets where available. Set mapped to v6 or v4 mapped to v6.
if (!SockaddrToV4Mapped(&target_addr, &output_mapped_target_addr)) {
// addr is v4 mapped to v6 or just v6.
output_mapped_target_addr = target_addr;
}
status = PosixSocketWrapper::CreateDualStackSocket(
nullptr, output_mapped_target_addr, SOCK_STREAM, 0, dsmode);
if (!status.ok()) {
return status;
}

if (dsmode == PosixSocketWrapper::DSMode::DSMODE_IPV4) {
// Original addr is either v4 or v4 mapped to v6. Set mapped_addr to v4.
if (!SockaddrIsV4Mapped(&target_addr, &output_mapped_target_addr)) {
output_mapped_target_addr = target_addr;
}
}

auto error =
PrepareTcpClientSocket(*status, output_mapped_target_addr, options);
if (!error.ok()) {
return error;
}
return *status;
}

#else /* GRPC_POSIX_SOCKET_UTILS_COMMON */

bool SockaddrIsV4Mapped(const EventEngine::ResolvedAddress* /*resolved_addr*/,
Expand Down Expand Up @@ -875,6 +954,14 @@ PosixSocketWrapper::CreateDualStackSocket(
int /*protocol*/, DSMode& /*dsmode*/) {
GPR_ASSERT(false && "unimplemented");
}

absl::StatusOr<PosixSocketWrapper>
PosixSocketWrapper::CreateAndPrepareTcpClientSocket(
PosixTcpOptions& /*options*/,
const EventEngine::ResolvedAddress& /*target_addr*/,
EventEngine::ResolvedAddress& /*output_mapped_target_addr*/) {
GPR_ASSERT(false && "unimplemented");
}
}

#endif /* GRPC_POSIX_SOCKET_UTILS_COMMON */
Expand Down
15 changes: 15 additions & 0 deletions src/core/lib/event_engine/posix_engine/tcp_socket_utils.h
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,21 @@ class PosixSocketWrapper {
const experimental::EventEngine::ResolvedAddress& addr, int type,
int protocol, DSMode& dsmode);

// Return a PosixSocketWrapper which manages a configured, unbound,
// unconnected TCP client fd.
// options: may contain custom tcp settings for the fd.
// target_addr: the destination address.
// output_mapped_target_addr: A out parameter. It is target_addr mapped to an
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can output_mapped_target_addr be replaced by a member of the PosixSocketWrapper? Should it always map to PeerAddress? It would be a cleaner API to have either a return value or output parameters, not both.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to keep the memory footprint of PosixSocketWrapper very small - currently it has only one member variable i.e the socket descriptor and all the operations it supports are stateless. This allows creating socket wrappers at will and using them to manipulate socket flags. The peer address is requested only once and I dont think we should be storing it in the socket wrapper. We can store it in the endpoint code that requests for it.

Shall I modify this to return an absl::StatusOr<std::tuple<PosixSocketWrapper, ResolvedAddress>> instead ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. In that case, I'd suggest a return struct, per https://abseil.io/tips/176

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

// address appropriate to the type of socket FD created. For example, if
// target_addr is IPv4 and dual stack sockets are available,
// output_mapped_target_addr will be an IPv4-mapped IPv6 address.
//
// Returns: Not-OK status on error. Out parameters are not set on error.
//
static absl::StatusOr<PosixSocketWrapper> CreateAndPrepareTcpClientSocket(
PosixTcpOptions& options, const EventEngine::ResolvedAddress& target_addr,
EventEngine::ResolvedAddress& output_mapped_target_addr);

private:
int fd_;
};
Expand Down