Plugin to extend the REST API?

[mneil]

Expected Behavior

I'd like to write a plugin that modifies the behavior of the OPA server itself with respects to the /v1/policies path. I've been working with OPA quite a bit for work and have taken some of the things I learned from those experiences and created an OPA server with a DynamoDB back end here https://github.com/mneil/opa-dynamodb. My wish is to modify the /v1/policies behavior to let people put policies directly into the database. I imagine I could do this by parsing the content type and adding additional functionality for json payloads.

Actual Behavior

Looking at the go plugins packages https://godoc.org/github.com/open-policy-agent/opa it doesn't seem possible.

Additional Info

What I think I'm going to have to do is reimplement the run command. That way when OPA is started I can run my own server that mimics the internal data. My other option (how I solved this previously) is to add a new available route and/or run another process to handle it.

I'm really looking for direction or insight by anyone if they have ideas or input if a DynamoDB back end interests them for OPA policy data.

Thanks!

[patrick-east]

My wish is to modify the /v1/policies behavior to let people put policies directly into the database

Can you clarify what you mean by this? The /v1/policies API is intended for managing Rego policies in OPA. If i'm understanding the code in https://github.com/mneil/opa-dynamodb the "policies" are JSON data that is fetched from dynamo with the custom builtin function, right? So would the extension you are asking for essentially just be inserting arbitrary JSON into the database?

That almost feels more like an implementation of the OPA store (with its base documents managed via PUT/PATCH/DELETE on /v1/data versus /v1/policies).

Anyway, regarding custom API's. Typically the extension path is to write an OPA plugin https://www.openpolicyagent.org/docs/latest/extensions/#custom-plugins-for-opa-runtime which starts up a new listener. The main drawback being that it wouldn't be on the same port as the normal OPA API, but there are a few plugins/integrations that take this approach.

What I think I'm going to have to do is reimplement the run command. That way when OPA is started I can run my own server that mimics the internal data. My other option (how I solved this previously) is to add a new available route and/or run another process to handle it.

Those both seem like pretty good options. Shorter term we could potentially expose some helper API's on like the runtime package to expose the server (and in turn its http.handler so you could add new routes), but it would still require essentially re-implementing the run command.

[mneil]

You're correct that I want to be able to manage policies. Ultimately, I'd like to split a rego file into two parts. I'd like the rego file to hold the logic and then store the data into dynamo. I'd logically call these two things attester and policy data. Sort of similar to how bundles can have json data in the bundle. The simplest solution to this would be to extend the rest api to accept json data and that would go to (ant database) while text files would be stored as normal.

I'm looking for the easiest solution from a infrastructure side where I wouldn't want users to have to run two processes or have engineers hit two different ports for the same service. Because of that, being able to extend the rest api would be ideal.

From an opa perspective being able to write a plug-in that implements new content types or new routes opens up possibilities.

I could accept data and store it in any backend. I could pair that with the existing bundle plugin and add prefetching or caching of the data in memory (already possible). And I can register all this as a plug-in itself to make configuration simple. Extensions would feel tightly integrated with opa.

Having the runtime expose the handler would allow me to add new routes and would satisfy this feature request. Would you be open to adding that or letting me take a shot at it?

[tsandall]

@patrick-east allowing users to customize the routes comes up from time to time. One option would be to add a plugin interface for the HTTP server (like we have for the decision logger) that allows users to register a custom plugin and then customize server behaviour.

[jkbschmid]

Hi all,
I'd also like to customise server behaviour, more specifically, I'd like to add routes to the existing handler in the server.
Having a plugin to modify the server's handler would be great!
For my use case, something like
s.Handler = plugin.updateRouter(mainRouter) instead of

opa/server/server.go

Line 644 in 489c581

s.Handler = mainRouter

This modification is done before the AuthHandler is added, since this is already separately configurable.
What do you think, @tsandall ? Would this make sense (and is this in line with the other feature request that come up from time to time)?

Thanks and stay healthy!

[jkbschmid]

As discussed in the community meeting on 11.05.2020, there is the possibility to extend the REST API by instantiating a custom runtime object with a custom mux.Router.
In order for this to work, two small changes are still required:

• Adding a router field to the runtime params
• Making runtime.Serve use the WithRouter function (WithRouter already exists)

I have created a small draft PR #3458 to show what this would look like.
I'd be happy to have your feedback on this approach:)

[mneil]

I looked and that pr didn't completely satisfy the initial ask (can I replace an existing route) but I think it accomplishes the ask within reason and I'm really grateful that you are working to support extending the rest API.

With that change anyone could add additional routes to extend the behavior of opa easily.

Thank you again for taking the time to look at this and for the effort on the project. I would say that pr closes this issue when it's merged.

[stale]

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.