2020, [piegames]: A new core

So I've more or less rewritten core

When looking at the diff, you'll see almost everything got turned upside down. But with a few exceptions, the only thing I did was refactoring existing code, moving it around etc. If you know the old code, you'll easily recognize a lot of the pieces. Think of it as a Ship Of Theseus refactoring.

But let's start things in the correct order:

• I decided that the connection stability was a feature causing a lot of complexity for little return, and thus removed it. The IO handling got a lot simpler (no more multiple Websocket connections) but also the state machines all had twice the amount of states (each one in a connected+disconnected variant) and transitions (connected and connection lost events).
• I removed the input helper state machine as I think it will result in a more idiomatic API to implement this outside of core. I also (temporarily) removed the nameplate listing functionality.
• After this, a lot of state machines had only one or two non-trivial states left and could be inlined into others, simplifying the design even further.
• I removed the capability of handling message sending events before the PAKE was finished, as this was given by the "sane wrapper" anyways. This removed yet more code from boss and one or two message queues.

At this point I started reassembling the remaining parts in a new order.

The new design

There is simply a method called run() that takes in two channels for API communication and the required paramters for initialization. It asynchronously polls for events from the message queues (queued up actions, API and IO input) and processes them while pushing back new events on to the queue. Here's a shortened version:

pub async fn run(
    appid: &AppID,
    versions: serde_json::Value,
    relay_url: &str,
    code_provider: CodeProvider,
    to_api: UnboundedSender<APIEvent>,
    mut to_core: UnboundedReceiver<Vec<u8>>,
) {
    let mut io = io::WormholeIO::new(relay_url).await;
    let mut actions: VecDeque<Event> = VecDeque::new();

    snip!("Initialization");

    loop {
        let e = match actions.pop_front() {
            Some(event) => Ok(event),
            None => futures::select_biased! {
                snip!("Poll for API or IO events");
            },
        };

        match e {
            snip!("Event handling, the actual logic");
        }
    }
    to_api.close_channel();
    to_core.close();
}

Yes, the WormholeCore struct is gone. At the end it only contained one realy state machine (plus the IO which is a local variable). The new main loop matches on the incoming event, then on the current state, handles that event which produces the new/next state. If the state is complex it is factored out to a sub-module with an own inner state machine. This is maybe the biggest design change made: instead of having multiple (product) state machines, there is only one but with possibly nested states.

enum State {
    AllocatingNameplate { wordlist: Arc<Wordlist> },
    ClaimingNameplate { nameplate: Nameplate, code: Code },
    Keying(Box<key::KeyMachine>),
    Running(running::RunningMachine),
    Closing {
        await_nameplate_release: bool,
        await_mailbox_close: bool,
        result: anyhow::Result<()>,
    },
}

This implies that there are a lot less events as well. Let's look at the new main Event (snipped comments, see real code):

pub enum Event {
    FromIO(InboundMessage),
    ToIO(OutboundMessage),
    FromAPI(Vec<u8>),
    ToAPI(APIEvent),

    CloseWebsocket,
    WebsocketClosed,
    BounceMessage(EncryptedMessage),
    ShutDown(anyhow::Result<()>),
}

There are four variants that handle outside communication, and only four more for internal use. As you can see, the IO events have been changed to simply contain the serializable enum of the server_messages module. FromAPI only has one variant—send a message (therefore APIAction is gone). APIEvent sadly could not entirely be reduced (to put it more optimistically, it got to reduced to four variants):

pub enum APIEvent {
    ConnectedToServer { welcome: serde_json::Value, code: Code },
    ConnectedToClient { key: Key, verifier: Vec<u8>, versions: serde_json::Value },
    GotMessage(Vec<u8>),
    GotError(anyhow::Error),
}

P.S.: For maximal confusion, I at one point in time swapped APIEvent with APIAction (and same for IO), as I thought it more intuitive that way. Doesn't matter that much anymore, since most of them are gone.

Error handling and IO

IO (and API communication) got inlined as much as possible into the event loop. So event sending is done by io.send(message).await;. Receiving is done at the top of the loop in futures::select_biased! { … }. All abstraction regarding this got removed and will have to be introduced back if needed.

Error handling is still rough on a lot of edges, but it's improving. If there is a non-recoverable error (IO unexpectedly closed, can't deserialize message), the connection is shut and an error is sent to the API. If the error is recoverable, the usual shutdown procedure is undertaken and then the API channel is closed with the error message. However, the behaviour during shutdown (it's still communicating more or less normally) needs to be examined further.

What about the "sane wrapper"

The core's API has moved a lot into the direction of the public API. Therefore the "sane wrapper" is not doing much anymore except glueing things together. However I don't think one can completely get rid of it without sacrificing the nice type system initialization. Also we still need a background task and communication pipes because of IO.