Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

downloader: support for downloading bundles from an OCI registry #4558

Merged
merged 1 commit into from Apr 28, 2022
Merged

downloader: support for downloading bundles from an OCI registry #4558

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
6 changes: 6 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Expand Up @@ -5,6 +5,12 @@ project adheres to [Semantic Versioning](http://semver.org/).

## Unreleased

### OCI support

The addition of the [OCI downloader](https://github.com/open-policy-agent/opa/blob/main/download/oci_download.go) allows the bundle and discovery plugins to download a bundle package from an OCI registry, and that is specified by the service type configuration value.

For more details please refer to the [services configuration](https://www.openpolicyagent.org/docs/edge/configuration/#services) and [#4518](https://github.com/open-policy-agent/opa/issues/4518).

## 0.39.0

This release contains a number of fixes and enhancements.
Expand Down
26 changes: 26 additions & 0 deletions docs/content/configuration.md
View file
Open in desktop
Expand Up @@ -252,10 +252,36 @@ multiple services.
| `services[_].tls.ca_cert` | `string` | No | The path to the root CA certificate. If not provided, this defaults to TLS using the host's root CA set. |
| `services[_].tls.system_ca_required` | `bool` | No (default: `false`) | Require system certificate appended with root CA certificate. |
| `services[_].allow_insecure_tls` | `bool` | No | Allow insecure TLS. |
| `services[_].type` | `string` | No (default: empty) | Optional parameter that allows to use an "OCI" service type. This will allow bundle and discovery plugins to download bundles from an OCI registry. |

Each service may optionally specify a credential mechanism by which OPA will authenticate
itself to the service.

##### Example

Using an OCI service type to download a bundle from an OCI repository.

```yaml
services:
ghcr-registry:
url: https://ghcr.io
type: oci

bundles:
authz:
service: ghcr-registry
resource: ghcr.io/${ORGANIZATION}/${REPOSITORY}:${TAG}
persist: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be persist: false? I think I don't understand yet fully how persistence and OCI dependent on each other 😅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can work with a persist false, but it needs a folder to store the layers it pulls from the OCI registry. I set that to the temporary folder if the persistence directory is not set.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind describing that behaviour and the meaning of the configurable persist when used with type: oci in the docs somewhere?

Is this directory garbage-collected regularly, or would it be the user's responsibility to keep it from growing indefinitely? If so, it needs to be called out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no garbage-collection mechanism in place for this. I added a note in the changelog about the maintenance of the persistence folder.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no garbage-collection mechanism in place for this. I added a note in the changelog about the maintenance of the persistence folder.

Uhm was that lost...? I can't find it 😅

polling:
min_delay_seconds: 60
max_delay_seconds: 120

persistence_directory: ${PERSISTENCE_PATH}
```

When using an OCI service type the downloader uses the persistence path to store the layers of the downloaded repository. This storage path should be maintained by the user.
If persistence is not configured the OCI downloader will store the layers in the system's temporary directory to allow automatic cleanup on system restart.

#### Bearer Token

OPA will authenticate using the specified bearer token and schema; to enable bearer token
Expand Down
57 changes: 57 additions & 0 deletions docs/devel/OCI.md
View file
Open in desktop
@@ -0,0 +1,57 @@
### Building and pushing a policy to an OCI registry

#### Using policy CLI

The [policy CLI](https://www.openpolicyregistry.io/docs/cli/download) tool can be easily used to build and push a policy bundle to a remote OCI registry using just two simple commands:
- `policy build <path_to_src> -t <org>/<repo>:<tag>`
- `policy push <registry>/<org>/<repo>:<tag>`
A full tutorial is available [here](https://www.openpolicyregistry.io/docs/tutorial)

#### Using OPA and ORAS CLIs

To build and push a policy bundle to a remote OCI registry with the [OPA CLI](https://www.openpolicyagent.org/docs/edge/cli/) and [ORAS CLI](https://oras.land/cli/) you can use the following commands:

- `opa build <path_to_src>` will allow you to build a bundle tarball from your OPA policy and data files

Now that we have the tarball we will need to provide a config manifest to the ORAS CLI and the tarball itself:
- `oras push <registry>/<org>/<repo>:<tag> --manifest-config <you_config_json>:application/vnd.oci.image.config.v1+json <the_tarball_obtained_from_opa_build>:application/vnd.oci.image.layer.v1.tar+gzip`
carabasdaniel marked this conversation as resolved.
Show resolved Hide resolved

Using an empty(`{}`) `manifest-config` json file should be sufficient to be able to push and allow the OCI downloader to use the remote policy image.

### OCI Downloader Debugging

Before starting to run a step-by-step debug process on the OCI downloader you will need a policy image pushed to a repository. To do this at the moment the easiest method is to use the [policy CLI](https://github.com/opcr-io/policy) to build your image and push it to a repository. You can use the public [opcr-io](https://opcr.io/) repository, [GHCR](https://ghcr.io) or any other OCI compatible repository.

The easiest method to be able to do a step by step debugging of the OCI downloader is to start from the available `oci_download_test.go` file and replace the `fixture rest client` with an actual rest client component and use an adequate configuration. This client is then fed to the constructor of the `OCI downloader` and it will use the credentials to access the desired repository and download the image.

The `NewOCI` constructor receives the downloader configuration, rest client, the upstream image path (<server>/<organization>/<repository>:<tag>) and the local storage path as parameters.

**Example** of a rest client configuration:
```
{
"name": "foo",
"url": "http://ghcr.io",
"credentials": {
"bearer": {
"token": "secret"
}
}
}
```
The `OCI Downloader` has the same behaviour as the default bundle downloader and the debugging process can follow the same pattern as seen in the test file.

### OCI Downloader Limitations

The OCI Downloader uses the oras go library to pull images and the limitations of the current implementations are:
- it accepts only **one** layer per image that contains the bundle tarball
- it can download only the following application media types:
- `application/vnd.oci.image.layer.v1.tar+gzip`
- `application/vnd.oci.image.manifest.v1+json`
- `application/vnd.oci.image.config.v1+json`
- cannot create/push image using the OPA CLI
- it can download only from OCI compatible registries

### OCI Downloader TODOs

- Remove deprecated `ClearCache` function.
- Implement appropriate `SetCache` function for the OCI downloader.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we cannot have the bundle caching via Etag with OCI ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the OCI spec documentation does not have any mentioning of using the Etag for caching. We can address this in a future PR.

230 changes: 0 additions & 230 deletions download/download_test.go
View file
Open in desktop
Expand Up @@ -8,26 +8,19 @@
package download

import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/http/httptest"
"reflect"
"strconv"
"strings"
"testing"
"time"

"github.com/open-policy-agent/opa/bundle"
"github.com/open-policy-agent/opa/keys"
"github.com/open-policy-agent/opa/logging"
"github.com/open-policy-agent/opa/logging/test"
"github.com/open-policy-agent/opa/metrics"
"github.com/open-policy-agent/opa/plugins"
"github.com/open-policy-agent/opa/plugins/rest"
)

func TestStartStop(t *testing.T) {
Expand Down Expand Up @@ -759,226 +752,3 @@ func TestWarnOnNonBundleContentType(t *testing.T) {
t.Errorf("Expected log entry: %s", expectLogged)
}
}

type testFixture struct {
d *Downloader
client rest.Client
server *testServer
updates []Update
mockBundleActivationError bool
etags map[string]string
}

func newTestFixture(t *testing.T) testFixture {

patch := bundle.PatchOperation{
Op: "upsert",
Path: "/a/c/d",
Value: []string{"foo", "bar"},
}

ts := testServer{
t: t,
expAuth: "Bearer secret",
bundles: map[string]bundle.Bundle{
"test/bundle1": {
Manifest: bundle.Manifest{
Revision: "quickbrownfaux",
},
Data: map[string]interface{}{
"foo": map[string]interface{}{
"bar": json.Number("1"),
"baz": "qux",
},
},
Modules: []bundle.ModuleFile{
{
Path: `/example.rego`,
Raw: []byte("package foo\n\ncorge=1"),
},
},
},
"test/bundle2": {
Manifest: bundle.Manifest{
Revision: deltaBundleMode,
},
Patch: bundle.Patch{Data: []bundle.PatchOperation{patch}},
},
},
}

ts.start()

restConfig := []byte(fmt.Sprintf(`{
"url": %q,
"credentials": {
"bearer": {
"scheme": "Bearer",
"token": "secret"
}
}
}`, ts.server.URL))

tc, err := rest.New(restConfig, map[string]*keys.Config{})

if err != nil {
t.Fatal(err)
}

return testFixture{
client: tc,
server: &ts,
updates: []Update{},
etags: make(map[string]string),
}
}

func (t *testFixture) oneShot(ctx context.Context, u Update) {

t.updates = append(t.updates, u)

if u.Error != nil {
etag := t.etags["test/bundle1"]
t.d.SetCache(etag)
return
}

if u.Bundle != nil {
if t.mockBundleActivationError {
etag := t.etags["test/bundle1"]
t.d.SetCache(etag)
return
}
}

t.etags["test/bundle1"] = u.ETag
}

type testServer struct {
t *testing.T
expCode int
expEtag string
expAuth string
bundles map[string]bundle.Bundle
server *httptest.Server
etagInResponse bool
longPoll bool
}

func (t *testServer) handle(w http.ResponseWriter, r *http.Request) {

if t.longPoll {

var timeout time.Duration

wait := getPreferHeaderField(r, "wait")
if wait != "" {
waitTime, err := strconv.Atoi(wait)
if err != nil {
panic(err)
}
timeout = time.Duration(waitTime) * time.Second
}

// simulate long operation
time.Sleep(timeout)
}

if t.expCode != 0 {
w.WriteHeader(t.expCode)
return
}

if t.expAuth != "" {
if r.Header.Get("Authorization") != t.expAuth {
w.WriteHeader(401)
return
}
}

name := strings.TrimPrefix(r.URL.Path, "/bundles/")
b, ok := t.bundles[name]
if !ok {
w.WriteHeader(404)
return
}

// check to verify if server can send a delta bundle to OPA
if b.Manifest.Revision == deltaBundleMode {
modes := strings.Split(getPreferHeaderField(r, "modes"), ",")

found := false
for _, m := range modes {
if m == deltaBundleMode {
found = true
break
}
}

if !found {
panic("delta bundle requested but OPA does not support it")
}
}

contentTypeShouldBeSend := true
if t.expEtag != "" {
etag := r.Header.Get("If-None-Match")
if etag == t.expEtag {
contentTypeShouldBeSend = false
if t.etagInResponse {
w.Header().Add("Etag", t.expEtag)
}
w.WriteHeader(304)
return
}
}

if t.longPoll && contentTypeShouldBeSend {
// in 304 Content-Type is not send according https://datatracker.ietf.org/doc/html/rfc7232#section-4.1
w.Header().Add("Content-Type", "application/vnd.openpolicyagent.bundles")
} else {
if r.URL.Path == "/bundles/not-a-bundle" {
w.Header().Add("Content-Type", "text/html")
} else {
w.Header().Add("Content-Type", "application/gzip")
}
}

if t.expEtag != "" {
w.Header().Add("Etag", t.expEtag)
}

w.WriteHeader(200)

var buf bytes.Buffer

if err := bundle.Write(&buf, b); err != nil {
w.WriteHeader(500)
}

if _, err := w.Write(buf.Bytes()); err != nil {
panic(err)
}
}

func (t *testServer) start() {
t.server = httptest.NewServer(http.HandlerFunc(t.handle))
}

func (t *testServer) stop() {
t.server.Close()
}

func getPreferHeaderField(r *http.Request, field string) string {
for _, line := range r.Header.Values("prefer") {
for _, part := range strings.Split(line, ";") {
preference := strings.Split(strings.TrimSpace(part), "=")
if len(preference) == 2 {
if strings.ToLower(preference[0]) == field {
return preference[1]
}
}
}
}
return ""
}