Add data masking and filtering to the OPA plugin #16
Conversation
- Simplify tests and remove some parameterization - Make more fields final - Remove unnecessary annotations - Make tests more parallelizable - Remove permissioning operations and replace them by a blanket allow/deny setting And rebase - bump pom version
vagaerg
changed the title
|
|
||
| import static java.util.Objects.requireNonNull; | ||
|
|
||
| public record TrinoColumn( |
There was a problem hiding this comment.
Im not sure whats the convention in Trino on javadoc, but it might sense to include your helpful comments on TrinoColumn here:
TrinoColumn is (perhaps counterintuitively) not used for FilterColumns for three easons:
- API stability between the batch & non-batch modes: we don't want to send a full TrinoColumn object (which includes the catalog, schema and table names) per column when performing a FilterColumns operation as this would make the requests rather large
- TrinoColumn may be changed to include additional information about column-specific operations in the future, which are not relevant to FilterColumns
- Backwards compatibility
There was a problem hiding this comment.
That makes a lot of sense, will do!
| public OpaContainer() | ||
| { | ||
| this.container = new GenericContainer<>(DockerImageName.parse("openpolicyagent/opa:latest-rootless")) | ||
| .withCommand("run", "--server", "--addr", ":%d".formatted(OPA_PORT)) |
There was a problem hiding this comment.
Nit: Sorry to bother you with the same again, but we could test we don't fail on the decision_id field being present:
| .withCommand("run", "--server", "--addr", ":%d".formatted(OPA_PORT)) | |
| .withCommand("run", "--server", "--addr", ":%d".formatted(OPA_PORT) | |
| "--set", "decision_logs.console=true") |
There was a problem hiding this comment.
Yup, sorry if I missed this on the other PR. Will add it!
| @@ -709,6 +714,23 @@ public void checkCanDropFunction(SystemSecurityContext systemSecurityContext, Ca | |||
| OpaQueryInputResource.builder().function(TrinoFunction.fromTrinoFunction(functionName)).build()); | |||
| } | |||
|
|
|||
| @Override | |||
| public List<ViewExpression> getRowFilters(SystemSecurityContext context, CatalogSchemaTableName tableName) | |||
There was a problem hiding this comment.
Regarding getRowFilters and getColumnMask: Is my understanding correct that this is currently only implemented by having an OPA call per table (for row filter) / column (for column mask)?
Do you have any plans of implementing a batch variant as well?
Would you even go one step further to assume users using these two functionalities are advanced enough to justify only offering a batched API for code complexity reasons?
There was a problem hiding this comment.
Pinged you on Slack to discuss a bit more, will update this PR comment once I understand the options available a bit better
vagaerg
force-pushed
the
add-open-policy-agent
branch
from
fd6d65c
to
54873d2
Compare
|
This is now being pushed upstream |
trinodb#9787 Implement column masking and row level filtering to the proposed plugin in trinodb#19532
This PR adds row level filtering and column masking. It creates a
TrinoColumntype, which is only used for these operations.TrinoColumnis (perhaps counterintuitively) not used forFilterColumnsfor three easons:TrinoColumnobject (which includes the catalog, schema and table names) per column when performing aFilterColumnsoperation as this would make the requests rather largeTrinoColumnmay be changed to include additional information about column-specific operations in the future, which are not relevant toFilterColumnsThis PR also adds system tests for row level filtering & column masking.
The idea is to take this PR upstream once trinodb#19532 is merged
CC @mosiac1 as co-developer