Implement Authorizer

[soenkeliebau]

In order to allow authorizing HDFS access requests with OPA, we need to implement a component to run inside of HDFS and forward requests to OPA, this is a principle very similar to the ones we currently use in Kafka, Druid and which is being merged in upstream Trino as well.

There is some code at https://github.com/stackabletech/hdfs-opa-authorizer where I have played around a bit, but none of this should be taken as gospel, if it seems weird then that is because it is weird and wasn't thought through!

The authorizer should implement HDFS' internal authorizer interface, serialize the relevant information from the context and forward it to a configurable OPA server.

Configuration could either be done via the HDFS config mechanism or in a first stage via environment variables.

Tasks

Beta Give feedback

1. Initial release of HDFS authorizer
2. Add HDFS authorizer to Stackable HDFS docker image
3. Update authorization section in the docs. Document the JSON object that comes in when writing rules. ideally also give some example rules.
4. feat(hadoop): Add patches to fix missing operationType for some operations in authorizer docker-images#555
   0 of 5
   release/2024-03 +1
   
5. feature tracker updated

[lfrancke]

I just looked at the Group Mapper and just wanted to give a heads-up that I'd like Java projects to follow the example of the Druid Authorizer in terms of code style and setup etc.

(I haven't looked at your code yet)

I can help if needed.

[sbernauer]

Alright 👍

[lfrancke]

Please link to documentation here

[sbernauer]

nightly doc link: https://docs.stackable.tech/home/nightly/hdfs/usage-guide/security#_authorization