Allow policy to force http.send to cache across queries

[tsandall]

There are various reasons why users would need to override the caching directives defined by the server (e.g., the server may be outside their control and configured incorrectly.) To workaround these cases, we should allow users to override the caching behaviour and control it themselves. Initially, users should be able to force caching (overriding the directives in the server's response) and control how long the cached response will be valid for:

# execute an HTTP GET caching the response for up to 5 minutes
http.send({
  "method": "get",
  "url": "https://example.service/v1/foo/bar",
  "force_cache": true,
  "force_cache_duration_seconds": 300,
})

If "force_cache" is true, it overrides the "cache" parameter. If the force_cache parameter is supplied they should have to provide other cache control info like duration (force_cache_duration_seconds above).

[pmeng99]

Is this fully implemented in v0.24.0? When I try it, the response does not seem to be cached. The end point got hit every single time. No caching seems to be taking place.

Anything else I need to do besides setting "force_cache" and "force_cache_duration_seconds"?

[ashutosh-narkar]

@pmeng99 how are you running OPA ?

[pmeng99]

@pmeng99 how are you running OPA ?

we are running it as an envoy plugin.

module xxxxxxxxx/xxxxxxx/opa-plugin

go 1.14

require (
github.com/open-policy-agent/opa v0.24.0
github.com/open-policy-agent/opa-envoy-plugin v0.24.0
gopkg.in/square/go-jose.v2 v2.5.1
)

[ashutosh-narkar]

I meant do you run OPA in server mode ? The inter-query cache is only used when OPA is run as a server.

[pmeng99]

Yes, we did, in server mode and as an envoy plugin..

[ashutosh-narkar]

Can you share your deployment manifest ? It will help reproduce the issue.

[pmeng99]

we start the server like this:

$ ./opa-plugin run --server --log-format=json-pretty --set=decision_logs.console=true --set=plugins.envoy_ext_authz_grpc.addr=:8089 --set=plugins.envoy_ext_authz_grpc.path=authz/allow policy.rego data.json

the plugin was built as mentioned in the go.mod file I sent you earlier

Let me know if any other info you need. This feature is critical to us. Please help!

[ashutosh-narkar]

Currently when the OPA-Envoy plugin creates the Rego object it does not set the inter-query cache and hence http.send falls backs to using the intra-query cache. So the plugin does not support inter-query caching atm.

[pmeng99]

Any plans as when to support it?

[ashutosh-narkar]

It should be a pretty small addition something like below and then it should support inter-query caching.

$ git diff internal/internal.go
diff --git a/internal/internal.go b/internal/internal.go
index ed356a3b..771b3c80 100644
--- a/internal/internal.go
+++ b/internal/internal.go
@@ -38,6 +38,8 @@ import (
@@ -38,6 +38,8 @@ import (
        "github.com/open-policy-agent/opa/server"
        "github.com/open-policy-agent/opa/storage"
        "github.com/open-policy-agent/opa/util"
+
+       iCache "github.com/open-policy-agent/opa/topdown/cache"
 )

 const defaultAddr = ":9191"
@@ -104,10 +106,11 @@ func Validate(m *plugins.Manager, bs []byte) (*Config, error) {
 func New(m *plugins.Manager, cfg *Config) plugins.Plugin {

        plugin := &envoyExtAuthzGrpcServer{
-               manager:             m,
-               cfg:                 *cfg,
-               server:              grpc.NewServer(),
-               preparedQueryDoOnce: new(sync.Once),
+               manager:                m,
+               cfg:                    *cfg,
+               server:                 grpc.NewServer(),
+               preparedQueryDoOnce:    new(sync.Once),
+               interQueryBuiltinCache: iCache.NewInterQueryCache(m.InterQueryBuiltinCacheConfig()),
        }

        // Register Authorization Server
@@ -136,11 +139,12 @@ type Config struct {
 }

 type envoyExtAuthzGrpcServer struct {
-       cfg                 Config
-       server              *grpc.Server
-       manager             *plugins.Manager
-       preparedQuery       *rego.PreparedEvalQuery
-       preparedQueryDoOnce *sync.Once
+       cfg                    Config
+       server                 *grpc.Server
+       manager                *plugins.Manager
+       preparedQuery          *rego.PreparedEvalQuery
+       preparedQueryDoOnce    *sync.Once
+       interQueryBuiltinCache iCache.InterQueryCache
 }

 func (p *envoyExtAuthzGrpcServer) Start(ctx context.Context) error {
@@ -366,6 +370,7 @@ func (p *envoyExtAuthzGrpcServer) eval(ctx context.Context, input ast.Value, res
                        rego.EvalParsedInput(input),
                        rego.EvalTransaction(txn),
                        rego.EvalMetrics(result.metrics),
+                       rego.EvalInterQueryBuiltinCache(p.interQueryBuiltinCache),
                )

[pmeng99]

Can you make a 0.24.1 release to include this?

[ashutosh-narkar]

Sure we could pick this up the next couple of weeks. Feel free to create a PR with this feature if you'd like and I can help out as well.

[pmeng99]

I see there is an edge release out. Is this fix in it?

[ashutosh-narkar]

Yes. The change is included in the v0.24.0-envoy-1 release of the opa-envoy plugin. Docker image: openpolicyagent/opa: 0.24.0-envoy-1.