gRPC tunneling

[ejona86]

Issue for cross-language tunneling coordination, especially reverse tunneling (where the client can become a server).

grpc/grpc-java#3987 (functional Java POC)
#14100

[ejona86]

The internal push for this fell away. We're looking for user interest. I know there is user interest, but it's hard to say how much there is. If you're interested, vote up the initial comment of this issue and make a comment describing the sort of use-case you're looking at (like Android or firewall-punching). Also mention whether the tunnel component would be generic or app-specific (as app-specific is easier to workaround today with a bidi stream).

Yes, that will produce something not that different from "me too" style of comments, but that's actually what we're asking for here.

[rAndom69]

Well, I will need this (as I definitely do not want to work-around with bidi streams as it's just bad interface for server which needs to connect to client)

So I might as well do pull request for C++ implementation in near-future.

[Kasheen]

I think I have a use case for this which is an app which allows an Android phone's file system to be explored / read / manipulated via a PC application. Essentially the phone is a file server or hosts a remote file system of sorts.

I've read a comment (I think by yourself @ejona86) whereby you say that a listening server on the phone can be flaky depending on screen off / rom configuration and therefore it's possibly appealing to have the connection set up in the reverse direction (PC listens, Android connects) and then reversing the relationship if I'm understanding this feature correctly.

Still looking into GRPC and not entirely sure this is a sensible use case for it, but it seems right now it'd rely on Android netty server working (not sure on the status of this), or doing some kind of bidi stream workaround (not sure how much of a mess this would be when the phone is very much the server in this relationship).

Not exactly a concrete need from me here, but just a 2 cents on a possible use case that might be useful to me. Figured I'd post since you're trying to gauge interest.

[MHDante]

The use case is a remote operations management platform (kiosks/iot). Due to nat/firewall, we'd like the ability to register a client kiosk to a server, and, on certain events, have the server invert into a client that issues requests to the kiosk. This can currently be done by multiplexing over a bidirectional stream, but would be much better with language support.

[overmike]

I am interested in this feature for a use case which service A submit a job through a scheduler and we have no idea the job ends up running at, and we need to send events to the job from service A.

But I am concern about how we can scale up service A after having a tunnel established to one instance of service A. May result in a lookaside table or sticky session

[fischman]

I have a use-case for this motivated by routability - my client can dial a server today but the reverse is not true. The server wants to make side-channel requests of the client, but these "reverse" service definition is largely orthogonal to the "forward" service definition, so intermingling them using bidi streaming and flipping the reverse service are unattractive options.
(this is for a python & C++ app in non-mobile environments, where the server and client are in independent network contexts)

[Anachronomicon]

I have basically the same use case as @fischman, right down to the language requirements. Something like Crossbar's router is perfect for our needs, but we've already gotten pretty invested in gRPC, so I'd rather not just drop it all and switch to WAMP.

[srini100]

More users interested in this enhancement grpc/grpc-go#484

[pgrosu]

@ejona86 I'd ask on the (grpc.io) discussion group - the sample size will be larger :)

[marcelmore]

The use case is clients on remote locations behind heavily NATted or downright unreliable connections, like clients which can only connect seldom through mobile wifi hotspots.

[Frank591]

My use case:
Servers in cloud and clients behind firewalls and NATs(for example, terminals in shops).
Client firewalls configured only for outcoimng connection on server port.
So when client starts, it try to initiate long live connection to server.
After connection establiashed, server wants to do RPC request to client.

Remark:
I didn't use gRPC before. I only doing first look on this technology and trying to understand can i use it in my project or no. And now i can't understand, how i can do rpc requests from server to clients using gRPC.
I searched on stackowerflow and did't find any solution. This issue looks like that's what I need.

And if you know, how i can implement server to client RPC using existing gRPC mechanics, pls tell me.

[dan-bar-dov]

I have the same use cases mentioned by fischman and Frank591 above

Opening two sockets (from an origin behind the firewall) and using one for each direction will be acceptable as well

[nrktkt]

Chiming in with an IOT use case where the client IP address is not stable and the client is in an unknown and probably NAT'd network.

Also of interest to those here might be one of the reverse http standards-attemtps that are out there. None have become standardized, but several have been implemented by multiple people.

[dustin-decker]

Github traffic stats shows >15 unique cloners per day on the code in https://github.com/dustin-decker/grpc-firewall-bypass which is a firewall-punching hack that came out of grpc/grpc-go#484

There's definitely some demand among the go users for this functionality.

[nikolain]

Yes, having the native ability to reverse tunnel for C++ / C# would be very useful.

Use case(s) -

1. remote controlling photo / video kiosks for sport events.
2. remote video judging for sport events.

In both cases target servers must be are located on event premises, behind NAT/firewall. And calls coming from outside (from cloud).

The topology where on-premise app (protected by firewall) initiates an outgoing connection to the cloud, and becomes an rpc server accepting calls on that connection would solve most of the problems.

[lothar7]

Would be very useful for remote IOT devices. I need it in my project and not having this feature makes me hesitate regarding gRPC.

[0x53A]

As WCF is deprecated, we want to move to gRPC.

As far as I understand it, this feature request would enable the equivalent to client callbacks.

---

Use Case:

We have a WebApi (C# asp.net core) in the cloud and a backend (c# winforms) hosted on-premise at our customers.

    Internal Network
 +-------------------------------+                  +-------+
 |                               |                  |       |
 |                               |                  |       |
 |  +--------------+WCF Client   |      WCF Server  |       |
 |  | Backend      |-------------------------------->       |
 |  |              |    <<<Wcf Client Callback<<<   |       | HTTP Server
 |  |              |             |                  |       |  <---------------------+ Client
 |  |              |             |                  |       |
 |  +--------------+             |                  | WebApi|
 |                               |                  |       |
 +-------------------------------+                  +-------+

Currently the WebApi (which is just a dumb proxy / tunnel) is both HTTP Server and WCF server.

The backend connects to it from behind the corp firewall as a WCF client and establishes a TCP channel.

If a HTTP request comes in, the webapi calls back to the backend through the client callballback.

---

This architecture has the really big advantage that we don't need any incoming ports opened for the internal customer network.

[ejona86]

@0x53A, to be clear: it is possible to do reverse tunneling today with a bidi stream. However, it can be annoying to tunnel many different things over the same stream and requires you to manually wire things up. This feature is to make type of setup you described easy.

[shaochieh]

we're looking for node to node communication which would enable something like reverse ssh tunneling. so we could have ad hoc or on demand communication with client node, doing remote upgrade, troubleshooting or proactive deployments of sorts. client nodes could be behind NAT.

[AbhishekPantUK]

I have a use case where a host pc can request the Android device for validating some test cases. Android device is always on. Can we run the server on Android? What would be the ip address?

[kskalski]

I'm looking for a solution to allow local gRPC "backend" (behind local network) to register at cloud server and let gRPC clients connecting to that cloud server reach the registered backend (by its registered id, so a certain level of inspection into forwarded traffic would be desirable for routing to different registered backends).
A requirement here is to let encryption / TLS pass through the cloud server uninterrupted (cloud server doesn't know certificates used by backend and client to communicate with each other).

The solution coming to my mind is an ssh tunnel from backend host to cloud server (exposing backend's serving port at cloud server) and some reverse proxy at cloud server that will route client connections to local ports and tunnel them to backends.

Another approach: let backend establish grpc stream connection to cloud server as a "transport" layer, receiving binary traffic as payload in messages and feeding it locally to server hosted in the same app (worst case through loopback network socket, preferably directly to gRPC server implementation). Client would need to do similar trick of sending its traffic to itself so it can be transported by stream connection to cloud server. The advantage here is that client/backend connection to cloud server can use dedicated server's certificate and routing is easy to implement for such transport connection.

[csmorley]

Firewall punching for remote management. Having an out of box way would be great, in meantime any example working code either in java or c# to do this manually?

[kskalski]

@BuBuMan FYI, I updated code and added working project with simple tunnel server S and A/B example client/server ready to check and try. The C# (forwarded) client code is a bit limited, for this part I use Java version, which has more features, but doesn't have a full working example yet.

[BuBuMan]

@kskalski Thanks for the notification. Unfortunately, I decided to use a custom RPC solution since gRPC has quite a few missing features for my use case. I figured adding those missing features to gRPC would take more time and work than just writing a custom solution that meets my exact needs.

[ptsneves]

@kskalski Where can i find the code you describe. Thanks in advance.

[kskalski]

@ptsneves
https://github.com/kskalski/snippets/tree/master/GrpcTunnel/

[ptsneves]

@kskalski This unfortunately will not work because underneath there would to be a server grpc class in an hypothetical device. This makes it a no go given the limitation of Android grpc of making a server. The concept of tunnel also feels a bit contrived.

I thought a bit about the topic and i think the tunnel is a wrong approach. I think the correct approach is to decouple the transport server/client concept from the RPC server/client concept. I think this is the core of the problem. People do not want to change the concept of rpc from server to client, they want to change the transport from server to client. Please do not get me wrong, as I have no idea if the gRPC code base has such rpc/transport decoupling in it's design, although i will probably study it.

My naive approach would be like the following in python - grpc server side/transport client side:

grpc_server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) product_pb2_grpc.add_ProductServicer_to_server(ProductRPC(product), server) server.add_remote_transport('myservice:50051')

In the snipet above as we decouple the grpc from the transport we can then add a remote transport of the form we want. Theoretically we could have our grpc server be available locally with a transport server or as a transport client, just we may have multiple ports.

grpc_server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) product_pb2_grpc.add_ProductServicer_to_server(ProductRPC(product), server) server.add_remote_transport('myservice:50051') server.add_insecure_port('[::]:50051')

[kskalski]

@ptsneves
I think the concepts of "server" and "client" are embedded fairly deep into gRPC's abstractions and approach. However they don't seem to be very limiting and can basically be considered more as "initiator" and "receiver" of communication channel, since after channel is initiated you can exchange messages freely and in any direction (using steaming rpc). One could imagine an extension of gRPC which defines "bi-direction" agents, such that if both are available (on local network? on tunnel service?), then both can initiate communication, but it just seems redundant with the capabilities of current schema.

My implementation has an obvious flaw of "working-around" difficulty of replacing transport in gRPC and it does it by creating regular server and client and then piping them locally to a transport. This is far from good and as you mention might even break on some platforms (fortunately in my case Android runs the client, not a server). The improvement of that implementation would do something similar to what you suggest - create a dedicated transport that would plug server/client directly into communicating with my tunnel service. It might or might not be possible (and maybe on some, but not all platforms) to implement using current grpc libraries, I looked at it and current abstraction of replacing transport is way too high level for the needs of tunnel service (it operates on events like rpc call, start, end, etc., while in my case I need to replace transport below encryption layer).

The tunnel service is probably the only part that is fully flexible and closest to target design, since it is the remote part of your proposed "transport" and just proxies the bytes between interested parties.

[ptsneves]

@kskalski Thanks for the thorough answer, my apologies for keeping dragging the subject. Comments inline:

I think the concepts of "server" and "client" are embedded fairly deep into gRPC's abstractions and approach. However they don't seem to be very limiting and can basically be considered more as "initiator" and "receiver" of communication channel, since after channel is initiated you can exchange messages freely and in any direction (using steaming rpc).

Given my android server limitation I am starting to think this may be the only way forward for me. I see one problem though: If i invert the semantics with the bidirectional abstraction, will i not need to initialise all the possible calls that my server might be interested in? If so, what about having connections open with no traffic for a long time, with possibly no traffic ever going through. This could very fast lead to resource exhaustion, with threads and sockets waiting and allocated in RPCs x connections.

I had some more questions but given the above mentioned concerns of the bidirectional stream methodology and the impossibility of android to be a server thus denying the usability of your tunnel proposal, i think i will look into using protobuf directly like @BuBuMan probably did.

EDIT In thrift the concept of transport separation is built-in to the framework. In about 2 hours i was able to implement a reversed transport that fit my requirements Will try to see if i do not lose other features in the migration.

Pozdrawiam and thanks for the insights :)

[kskalski]

@ptsneves you didn't describe your use-case, but from what you say I suppose you just need a way for cloud server to initiate messages to your android app (e.g. notifications?). This is precisely what bidi streams solve and it's actually quite simple to do it with them - you just open streaming connection and then server will send messages on its stream, client will respond to them. I wouldn't worry too much with resource exhaustion, I believe TCP/gRPC connections can be opened and kept alive (with heartbeats built into gRPC) at quite large scale, e.g. single server can host 1000s of them without problem.

My tunnel code and this issue is handling a rather different use-case: two agents that are not publicly visible or accessible on internet want to "meet" and talk to each other, in this case both of them need to first connect to the 3rd server (on the internet) and communicate through it.

[ptsneves]

@kskalski The use case may be a bit complex but here it goes.

I work in embedded devices. Currently I have website where users are able to remotely control an embedded device through this website. The device has it's functionalities available through a grpc server. Be aware that the grpc server is not the device itself. So the website connects to this grpc server and is able to request actions or data streams.
This is nice if I have a static infrastructure where a computer is always available and provides this discoverable device server. What I want now is to make this infrastructure more flexible.

The grpc server is still providing me RPC calls for my website to use, but I want to make the connection be initiated by an Android application to my website. This allows for the flexible infrastructure that i was speaking about. It also requires that the rpc client/server architecture remains, but the transport be inverted.

The bidirectional case looked simple at first but now I am slightly stuck, because i kind of need connection state management. I need to initiate the connection on the android app(transport client) to the website(transport server) on every defined RPCs and then on the website block until my user asks for an action(for example a physical movement). One of the things i am concerned is about timeouts in this dormant connections. Does grpc allow for an infinite deadline? And if so does it auto manage the keep alive of each bidirectional RPC?

[kskalski]

Your case can be easily handled by streaming call from android app to the website, this issue is probably not the place to discuss details, but in general I wouldn't worry too much, gRPC is well suited for keeping long-running streaming connections open.

[kent-h]

I'm very interested using gRPC for peer-to-peer networking & clustering.

I'm interested in bidirectional RPC for these cases.

For such a case, streaming is a very, very hacky solution. If I'm going to use streams for two-way RPC requests, I'm basically implementing my own protocol on top of gRPC, (which may even be a second layer of gRPC).

If I have to implement my own protocol anyways, might as well cut out this gRPC layer, do away with all the extra complexity, and implement on TCP directly.

If, on the other hand, bidirectional requests were a first-class citizen, gRPC would be the obvious choice for p2p & clustering applications.

As an aside: In applications I've seen, it's common to want a server to make occasional unsolicited calls to a client. This is usually implemented through some event system. Bidrectional streams would greatly simplify this case.

[markdoerr]

We (sila-standard.org) have a large use-case, where e.g. lab devices / IoT devices should be controlled by a client in the cloud, but institution firewalls prevent connecting to the servers. We would be interested in a standardized (non-hacky solution) :)

[pehlert]

Hi @markdoerr @kent-h , we are facing exactly the same situation (IoT scenario, we want bi-directional RPC) which gRPC right now. We are using streams to listen for commands from behind the NAT, but it's a very hacky solution indeed.

I'd be curious to learn what you did to resolve those problems. Any hints?

[markdoerr]

Hi @pehlert , SiLA went (to my impression) a rather ugly way - basically: opening a bidirectional channel, client initiated and then sending back RPC commands to a receiving server through this bidirectional channel, please see this SiLA specification document for a clearer explaination: https://docs.google.com/document/d/1nGGEwbx45ZpKeKYH18VnNysREbr1EXH6FqlCo03yASM/edit

I think the solution is not nice - and should be happening on the gRPC layer transparently.
I would be happy, if this community could add such a feature soon :)

[rajatgoyal247341]

I am having a requirement where my grpc-servers are behind firewall. So all incoming connections to these servers are blocked.

I want to use grpc as tunnel, where these servers will initiate the tcp connection with a publicly exposed load balancer. I chose nginx as load balancer for POC. So my idea is :

a) Grpc servers will initiate a long-lived tcp connection with nginx by calling a RPC. Nginx will have all these servers defined under upstream group. This way each upstream server will have 1 connection with nginx.

b) A Grpc client calls nginx, and nginx will forward the request to any of upstream server. In-doing so nginx should use the connection established in step-a than creating a new connection to upstream server.

But when I tested with above set-up, I see that nginx is creating a new connection than using the already established connection with upstream server.

Can you please suggest if this is possible ? How differently should I run the grpc server / nginx conf. Can any other load-balancer server above purpose than nginx ?

[ash2k]

I have a use case for this as well - server needs to be able to act as a client and call RPCs on the client's internal gRPC server. Client is behind NAT/firewall.

Currently it's implemented using bidi streams. Both client and server run "internal" gRPC servers. Client connects to the server's "external" gRPC API endpoint to establish a bidi connection. When the server needs to perform an RPC, it makes a request to its internal gRPC server as if it's talking to a suitable client directly. What "suitable client" means is passed in the metadata of the request. The handler on that internal server looks up the list of connected clients in Redis to locate the "suitable" ones and to which instances of the server they have an open connection. Each client may establish N connections to M servers so a server may need to perform an RPC to a client that is connected to another server. After the set of suitable connections is found, handler routes the request to the correct server, which then routes the request to the correct connection of the desired client. The client receives the request over that bidi stream it established and then turns it into an RPC to it's own internal server. The response is proxied back.

That whole routing layer is generic and it's trivial to add new endpoints on the server. It's implemented in Go and is available here:

• Most of it: https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/internal/module/reverse_tunnel
• With some pieces that glue everything together: https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/cmd/kas/kasapp (see router*.go files)

[rajatgoyal247341]

Thanks @ash2k for reply. Suppose we have 10 servers ( horizontally scaled app servers) and 3 clients ( behind firewall).
As per my understanding of your project, each of these 3 client will establish bi-di connection(s) to 10 servers directly without any proxy server in between. And when any server needs to talk to any suitable client, it will call the client over the earlier established connection and again there is no proxy server in between.

Can you please correct if I got it wrong.

If we go this way, then we might need to expose our app servers publicly, which doesn't seem a very good idea ?

[ash2k]

A client establishes N gRPC connections (over 1 or more underlying TCP connections) to an address. This address is a load balancer with M app servers behind it. Each app server tracks connected clients in Redis+its own address.

When a server (let's call it "routing server") needs to make a request to a certain client, it:

1. Finds records about all suitable clients in Redis (not all clients may be able to answer a particular request)
2. Tries to make a request to another server ("gateway server") that has a connection from a certain client
3. If it fails, another gateway server is tried
4. If it succeeds, gateway server routes the request to the connected client.

It's described here but there is a lot of unrelated details.

[mankhakb]

Well, I will need this (as I definitely do not want to work-around with bidi streams as it's just bad interface for server which needs to connect to client)

So I might as well do pull request for C++ implementation in near-future.

Is there any pull request for the c++ implementation for Bidi

[znwabudike]

Looking at building this out, would absolutely love to have this implemented somewhere. If you need any Android assistance I'm available.

[andreybavt]

Has anyone tried to build a python client similar to what @ejona86 has done in TunnelClient.java ?

[joshuagruenstein]

I'm expressing interest for this as well for the purpose of firewall-punching. I know many teams that use VPNs to get around this that would greatly appreciate a natively supported tunneling approach as a far simpler alternative.

[link89]

I am wondering is there any other RPC framework that provide such feature if gRPC is not viable now?

[r4ve1]

I am wondering is there any other RPC framework that provide such feature if gRPC is not viable now?

Maybe json-rpc? I'm currently using this in my project (Golang), and all it needs is an established connection (net.Conn)

[frozenice]

I am wondering is there any other RPC framework that provide such feature if gRPC is not viable now?

You can take a look at RSocket. I'm using this in some projects (Kotlin, Go, JS). Here is a quick overview from my experience:

• There is client/server connection-wise (who connects to whom), but both can request and respond equally (just register callbacks on connect/listen)
• Transport via TCP or WebSockets
• Different interaction models for calls (streaming inputs and/or outputs)
• Messages have metadata and data payloads, those are just bytes and can be encoded however you want (e.g. Protobuf, JSON)
• For metadata you probably want to use the Composite Metadata Extension, so you can put several things into a single message's metadata (e.g. Routing, Auth)
• Supports resumption with missed messages playback (depends on the implementation)
• Can be a bit rough around the edges in some places

[pojntfx]

I've recently built a framework for Go & TS that does exactly this (you can call RPCs on both the client and the server, with an arbitrary transport layer & even closure support): https://github.com/pojntfx/dudirekta

[bsideup]

FYI I submitted a "client channels" impl to grpc-java that allows calling services on the client:
grpc/grpc-java#11023

Here is an example:
https://gist.github.com/bsideup/bd62eb19e7d7cbc590fad07b140c9201

I was also able to successfully connect a Go-based client to it, exposing a service:
https://gist.github.com/bsideup/49a031c3360a68e4542e5597669fe51d
(containers both client & server, works well with Java-based client & server)

Unlike Gitlab's solution, it uses raw streams to exchange the bytes (HTTP2 over raw gRPC streams without encoding/decoding) and pretty efficient (should close to raw HTTP2 over TCP performance)