API Exploration

[darvld]

Last modified: 27-07-2023

This discussion seeks to explore different approaches we can take to improve the current public API offered by Elide, as well as new ways to organize the source code for ease of maintenance and collaboration.

Important
This document is a work in progress, rationale for the proposed APIs will be added in the future, as well as new sections providing details on how the new features will be supported internally.

Separation of concerns

Currently there are three main parts to the Elide project: the Runtime, the Library, and the Framework.

These components are tightly coupled with each other and with external frameworks (e.g. the VMFacade, in the runtime core, has a specific method for rendering HTTP requests, specifically Micronaut's HttpRequest instances).

To solve this issue, first we need to define the scope covered by each component, so that we may refactor as appropriate.

Important
What follows is a hypothetical view of the project, it is not the way the project works now, but rather how it could work in the future.

Runtime

The Elide runtime provides an API for running guest languages with minimal effort (while still allowing advanced configuration of the polyglot engine):

val engine = PolyglotEngine {
  // adjust settings for all languages
  vfs {
    fromResourcesZip(BUILTIN_VFS)
  }

  contextManager {
    maxPoolSize = 4
  }

  // configure the guest languages supported by this engine
  install(JavaScript) {
    // general JS configuration
    esm = true
    
    // enable specific intrinsics such as the Console API
    intrinsics {
      console()
      base64()
      json()
    }
    
    // inject builtin modules backed by JVM code such as 'http' and 'crypto';
    // these will be available as drop-in replacements using regular syntax,
    // e.g. `require("express")`, `import http from "http"`.
    builtinModules {
      http()
      express()
    }
    
    // additional custom bindings provided to the guest language
    bindings {
      put("message", "Hello!")
    }
  }
  
  install(Python) {
    // ...
  }
}

Note
The install methods for adding features to a generic engine are inspired by Ktor's plugin API, which makes it really easy to develop and use extensions for domain-specific features.

The base PolyglotEngine API can be used when there is a need to execute dynamic guest code, i.e. sources that are resolved in runtime, as opposed to compile-time code:

suspend fun polyglot() {
  // acquire a polyglot context to execute guest code; this function
  // suspends until a context is available for use
  val pinned = engine.use { // this: PolyglotContext
    // extension function, shorthand for evaluate(JavaScript, "...")
    val jsFunction = javaScript("(input) => input * 2")

    // we can extract this value and use it later by "pinning" it,
    // the return type of this expression is PinnedValue
    pin(jsFunction)
  }
  
  // to use the pinned value, we explicitly ask the engine to acquire
  // the original context it came from
  engine.use(pinned) { func -> // this: PolyglotContext
    // now we can use the function, inside the originating context, note
    // that if the value is not a function, an exception will be thrown
    func(26)
  }

  // release the reference to this value, calling `engine.use` on it
  // will throw an exception, since the reference no longer exists
  pinned.release()
}

Framework

The Elide framework uses the Runtime API to provide a Polyglot server framework:

• Entry point in the JVM. The runtime will start the application and load all relevant configuration from the environment, before delegating to the guest code for final setup.
• Guest code can interact with the framework using a polyglot API to register routes, update configuration, include source files from other supported languages, etc.
• After guest code is executed in the startup phase, the application is considered fully configured and the server will start listening for connections.

Sample usage

Run the application using:

elide run app.js
# or
elide run app.py

You can define your application in JavaScript:

// app.js

// use Elide as a normal javascript module
import elide from "elide"
// seamless imports for supported languages
import chatbot from "./logic/ai/chatbot.py"

// general server configuration
elide.configure({
  port: 3000,
  requestTimeout: 5000,
  statusPages: true,
})

// call Python code to configure the chat agent
chatbot.init()

// create an Express-style router for creating handler hierarchies
// it is also possible to use the `elide.get(...)` methods directly
const routes = elide.router()

// route registration (express-style), notice the additional "context" value,
// which allows access to additional methods in the scope of this handler
routes.get("/hello", async (request, response, context) => {
  context.logging.info("Received greeting")
  response.sendString("Hello 👋!")
})

routes.post("/chat", async (request, response) => {
  // first-class async/await support in handlers
  const message = await request.bodyAsText()
	
  // async calls across language barriers!
  const completion = await chatbot.getCompletion(message)
	
  // no need to wait for the last operation
  response.sendString(completion)
})

or in Python:

# app.py

# use Elide as a normal Python library
from elide import elide, GET, POST
# seamless imports for supported languages (extension not included)
from logic.ai.chatbot import chatbot

# general server configuration
elide.configure(port=3000, requestTimeout=5000, statusPages=True)

# call javascript code to configure the chat agent
chatbot.init()

# use FastAPI-style route definitions with decorators
@GET("/hello")
async def hello(request, response, context):
  context.logging.info("Received greeting")
  return "Hello 👋"

@POST("/chat")
async def chat(request, response):
  # same API as in JS
  message = await request.bodyAsText() 
	
  # more async calls across languages
  completion = await chatbot.getCompletion(message)
	
  # send the response
  response.sendString(completion)

# or use the base API similar to JS
routes = elide.router()
routes.get("/hello", hello)
routes.post("/chat", chat)

or even in Kotlin:

import cloud.elide.*
// python import (resolved with compiler plugin)
import com.example.logic.ai.chatbot.chatbot

// This is your entrypoint, the runtime will pass an Application instance as
// receiver to allow your code to configure it, similar to how Ktor works
fun Application.configure() {
  // call Python code to configure the agent
  chatbot.init()

  // configure route handlers with idiomatic builders
  // (or manually with the same API as JS and Python)
  configureRouting()
}

// this DSL is a shorthand for using `val router = this.router()` and manually
// calling each route builder on that instance
fun Application.configureRouting() = routing {
  get("/hello") { // the receiver for this function provides extra features
    context.logging.info("Received greeting")
    return "Hello 👋"
  }
    
  post("/chat") {
    // suspending function call
    val message = request.bodyAsText()
    
    // interoperability with coroutines from other languages
    val completion = chatbot.getCompletion(message).await()
    
    response.sendString(completion)
  }
}

Threading

Route handlers defined as in the previous examples will not be able to benefit from concurrency-related features provided by the underlying server engine.

This is because, in order to invoke a single handler from multiple threads at the same time, it would be necessary to re-evaluate the entire initialization script for each VM in the pool.

Isolated (strict) handlers

The solution is to avoid external references in the handler function: any required values may be passed through the context API (configurable by the initialization script):

// when a new context is initialized, this function will be called, the first
// argument is a "key", used for debugging purposes
elide.context.use("chatbot", context => {
  // use a dynamic import to include python code in this new context
  const chatbot = await import("./logic/messages.py")
	
  // initialize the chatbot in this context
  chatbot.init()
	
  // store the value for handlers to use
  context.chatbot = chatbot
})

Then in the handler function we only use values provided by the context:

elide.get("/chat", (request, response, context) => {
  const message = await request.bodyAsText()
	
  // retrieve this context's 'chatbot' instance
  const completion = await context.chatbot.getCompletion(message)
	
  response.sendString(completion)
})

This way, the initialization script is called only once, and only handler dependencies registered via context.use are duplicated for each context, reducing the overhead involved in spinning up a new VM in the pool.

---

Elide project structure

Rather than implementing all features in a single module, we should split language support features to allow users to depend only on the libraries they need for the target language.

For example, the following modules can be used as dependencies by user code:

• elide-core: provides access to the "core" polyglot API, allowing the development of new language integrations, intrinsics, etc.
• elide-js: extends the core API and implements JavaScript support, providing an API for developing custom module replacements, which can be used to develop custom Express.js drop-in solutions, for example.
• elide-js-express: adds a Netty-backed Express.js implementation that plugs into the core polyglot API using the tools provided by the elide-js module.

We should also consider separating the framework into its own repository:

• elide-framework-core: base API for configuring polyglot applications from the guest languages, without HTTP. This module may also include dependency injection features.
• elide-framework-http: introduces an API for polyglot HTTP servers.

What about the library?

Elide as a library currently provides utilities like logging and multiplatform annotations for DI. These features have a place at the core of the Framework, rather than as an independent component.

---

Progressive migration

Rather than a full rewrite of the project, we must take small steps to repurpose and cleanup the code we currently have. What follows is in incomplete list of the stages that must be applied in this migration.

TBD

[sgammon]

From our discussion on Slack, we've agreed that these APIs are a good structure, but should be exposed in a core module which are used by the developers, and by integrators or embedders who need access to internals. This should keep the user-facing API clean while enabling this deeper level of control over assembly of guest VMs.

More feedback etc. coming soon

[darvld]

Additionally, we can use opt-in markers in that code so regular users can be notified by the Kotlin compiler if they are accidentally using delicate APIs

[sgammon]

I'm down for that, yes. That would make it much easier to understand what's experimental and what's not. Core will definitely need a completely stable API

[darvld]

A first implementation for this proposal is being added in #382.

[sgammon]

Update: This has moved from implementation to adoption internally within Elide, and will be closed at such time as it ships in the next release 👍🏻