Socket Closed error when opening wallet using gRPC with Python client

[ioncebotari]

Environment

CentOS 7
Python 3.6 with grpcio 1.25.0
btcwallet version 0.11.0-alpha

How to reproduce the bug

This is the code that can be used for reproducing the bug:

from mnemonic import Mnemonic
import os
import grpc
import logging
from crypto.api_pb2_grpc import WalletServiceStub, WalletLoaderServiceStub, VersionServiceStub
import crypto.api_pb2 as req
from base64 import b64encode

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)


class UsernamePasswordCallCredentials(grpc.AuthMetadataPlugin):
    """Metadata wrapper for raw access token credentials."""

    def __init__(self, username, password):
        self._username = username
        self._password = password

    def __call__(self, context, callback):
        credentials = "{}:{}".format(self._username, self._password).encode()
        basic_auth = "Basic {}".format(b64encode(credentials).decode('ascii'))
        metadata = (('authorization', basic_auth),)
        callback(metadata, None)


class BitcoinProcessor:
    """Class used for defining Bitcoin operations

    The class uses gRPC API with btcwallet to access the bitcoin wallets and perform transactions
    :param hostname: The hostname to which to connect
    :param port: Port number
    :param username: Defined in btcwallet.conf. Usually the same username that is used to connect to the btcd API
    :param password: Defined in btcwallet.conf. Usually the same password that is used  to connect to the btcd API
    """

    def __init__(
            self, hostname, port, username, password,
            secure=True,
            ca_path='/data/btc/.btcwallet/ec-ca.crt',
            cert_path='/data/btc/.btcwallet/client.cert',
            key_path='/data/btc/.btcwallet/client.key'
    ):
        self.http_credentials = grpc.metadata_call_credentials(
            UsernamePasswordCallCredentials(username, password))
        if secure:
            certs = [os.path.abspath(ca_path), os.path.abspath(key_path), os.path.abspath(cert_path)]
            cert_data = list()
            for file in certs:
                with open(file, 'rb') as f:
                    cert_data.append(f.read())
            creds = grpc.ssl_channel_credentials(cert_data[0], cert_data[1], cert_data[2])
            self.ic = grpc.secure_channel(hostname+':'+str(port), creds)
        else:
            self.ic = grpc.insecure_channel(hostname+":"+str(port))

    def __del__(self):
        self.ic.close()

    def version(self):
        """Gets the API version

        :return: The response from the gRPC API with the corresponding version description
        """
        stub = VersionServiceStub(self.ic)
        try:
            response = stub.Version.with_call(req.VersionRequest(), credentials=self.http_credentials)
        except grpc.RpcError as e:
            logger.info('Version failed with {0}: {1}'.format(e.code(), e.details()))
            return None
        return response

    def open_wallet(self, public_passphrase=''):
        """Opens a pre-existing wallet

        :param public_passphrase: The public passphrase. If its length is zero, an insecure default is used instead.
        :return: OpenWalletResponse if successful and None in case of error
        """
        stub = WalletLoaderServiceStub(self.ic)
        message = req.OpenWalletRequest(
            public_passphrase=public_passphrase.encode('utf-8')
        )
        try:
            response = stub.OpenWallet.with_call(message, credentials=self.http_credentials)
        except grpc.RpcError as e:
            logger.info('OpenWallet failed with {0}: {1}'.format(e.code(), e.details()))
            return None
        else:
            logger.info("Wallet opened:", response)
            return response

Running the code:

>>> from crypto.bitcoin import BitcoinProcessor
>>> bp = BitcoinProcessor('btcwallet.test.com', 18332, 'myuser', 'SomeDecentp4ssw0rd')
>>> bp.version()
Version failed with StatusCode.UNAVAILABLE: failed to connect to all addresses
>>> bp.version()
(version_string: "2.0.1"
major: 2
patch: 1
, <_Rendezvous of RPC that terminated with:
        status = StatusCode.OK
        details = ""
>)
>>> bp.open_wallet()
OpenWallet failed with StatusCode.UNAVAILABLE: Socket closed

Debug-level output from btcwallet:

# sudo -u btc /opt/go_apps/bin/btcwallet --testnet --noclienttls --debuglevel=debug --noinitialload
2019-12-16 11:31:19.195 [INF] BTCW: Version 0.11.0-alpha
2019-12-16 11:31:19.209 [INF] BTCW: Experimental RPC server listening on 127.0.0.1:18332
2019-12-16 19:55:39.322 [INF] GRPC: transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2019-12-16 19:55:39.331 [INF] GRPC: transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2019-12-16 20:02:26.095 [INF] GRPC: transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2019-12-16 20:02:26.103 [INF] GRPC: transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2019-12-17 12:31:51.262 [INF] GRPC: transport: loopyWriter.run returning. connection error: desc = "transport is closing"
2019-12-17 12:36:00.306 [INF] WLLT: Opened wallet
2019-12-17 12:36:00.306 [CRT] GRPC: Server.RegisterService after Server.Serve for "walletrpc.WalletService"
[root@instance-2 ~]# sudo -u btc /opt/go_apps/bin/btcwallet --testnet --noclienttls --debuglevel=debug --noinitialload
2019-12-17 12:38:26.569 [INF] BTCW: Version 0.11.0-alpha
2019-12-17 12:38:26.581 [INF] BTCW: Experimental RPC server listening on 127.0.0.1:18332
2019-12-17 12:38:49.902 [INF] WLLT: Opened wallet
2019-12-17 12:38:49.902 [CRT] GRPC: Server.RegisterService after Server.Serve for "walletrpc.WalletService"

Opening the wallet using the legacy RPC works properly. I see the following output from btcwallet in legacy mode:

# sudo -u btc /opt/go_apps/bin/btcwallet --testnet --noclienttls --debuglevel=debug
2019-12-17 13:06:06.236 [INF] BTCW: Version 0.11.0-alpha
2019-12-17 13:06:06.247 [INF] RPCS: Listening on 127.0.0.1:18332
2019-12-17 13:06:06.247 [INF] BTCW: Chain server RPC TLS is disabled
2019-12-17 13:06:06.247 [INF] BTCW: Attempting RPC client connection to localhost:18333
2019-12-17 13:06:07.446 [INF] WLLT: Opened wallet

and I can use btcctl to get the current balance:

# sudo -u btc /opt/go_apps/bin/btcctl --rpcuser=myuser --rpcpass=SomeDecentp4ssw0rd --wallet getbalance --testnet --skipverify

[nostdm]

I had the same issue in Go.

It seems that for some reason the way the code is written it runs RegisterWalletServiceServer(...) after server.Serve() which results in an error.

[nostdm]

When the gRPC server is enabled with the --experimantalrpclisten flag it seems that it first registers the server Version and WalletLoader services and then starts the server. But after the wallet is loaded it tries to register the Wallet service and it fails because the gRPC server is already listening. Has this logic worked before? I don't think you can register gRPC services dynamically at runtime. Looking at git logs this implementation has been added to the codebase around 5 years ago 497ffc1 .
Maybe @jrick can shed some light on this since he's the one who implemented the gRPC features.

[jrick]

This is fixed in dcrwallet, you're welcome to backport it as long as you respect the license.