Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add wasm filter-chain entity support #987

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

hishamhm
Copy link
Collaborator

@hishamhm hishamhm commented Jul 28, 2023

This brings support for wasm filter chains to deck.

Most of the groundwork for this was laid in the other repos:

Wasm filters were added in 3.4 and are an optional feature. Feature detection in deck works by fetching the root admin API endpoint and inspecting the configuration to see if wasm is enabled (this works for versions <3.4 and versions >=3.4 with wasm disabled). Feature detection does not run in Konnect mode; therefore Konnect is not yet supported, only Kong gateway OSS/EE

For the sake of keeping things reasonable in size (we're at >3k new lines of code if you include the other 2 PRs), there are some validation steps that have been left out:

  • validating filter chain names against those that the admin API tells us are installed/available
  • validating filter configurations against the JSON schema from the admin API

These things are of course still validated by the Kong admin API, but at a later date it will be possible to validate them client-side for better UX.


KAG-4005
KOKO-1275

@CLAassistant
Copy link

CLAassistant commented Mar 14, 2024

CLA assistant check
All committers have signed the CLA.

@flrgh flrgh force-pushed the feat/filter-chains branch 2 times, most recently from a84e1ea to f9c3ca5 Compare March 20, 2024 01:52
@flrgh
Copy link

flrgh commented Mar 20, 2024

See also: Kong/go-database-reconciler#72

@flrgh flrgh force-pushed the feat/filter-chains branch 2 times, most recently from fc9da65 to 071d975 Compare May 13, 2024 21:53
@flrgh
Copy link

flrgh commented May 13, 2024

Kong/go-database-reconciler#72 has been merged. Awaiting the merge of Kong/go-database-reconciler#90 and for a new release of go-database-reconciler now.

Edit: both of these are done.

@flrgh flrgh changed the title feat: WebAssembly filter chains, coming in Kong Gateway 3.4 feat: add wasm filter-chain entity support May 13, 2024
@flrgh flrgh self-assigned this May 13, 2024
@flrgh flrgh force-pushed the feat/filter-chains branch 4 times, most recently from 907e29f to b6dbadd Compare May 14, 2024 20:53
Comment on lines +420 to +424
workspace := client.Workspace()
client.SetWorkspace("")
defer client.SetWorkspace(workspace)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a mega hack, but I get test failures without it because sometimes tests run against a workspace that doesn't exist. Is there a better way?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment related to this here: https://github.com/Kong/deck/pull/987/files#r1629376245

In case we need to do this check, do you think we can modify the fetchKongVersion function to also deal with it?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so fetchKongVersion() is actually where I got the idea for this. It does something similar by trying to fetch / before trying the /{workspace} endpoint.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested to reuse fetchKongVersion() so that we will not have to run the same query twice.

Also, the current code would fail with 401 if decK doesn't have the permissions to query the default workspace.

@flrgh flrgh force-pushed the feat/filter-chains branch 2 times, most recently from f88707c to 9f2c566 Compare May 20, 2024 20:51
@flrgh flrgh marked this pull request as ready for review May 21, 2024 20:58
@flrgh flrgh requested a review from a team as a code owner May 21, 2024 20:58
@flrgh
Copy link

flrgh commented May 21, 2024

re: CI Test / test (pull_request) Failing after 4m, this is due to us getting ratelimited by Codecov:

[2024-05-21T16:04:17.142Z] ['error'] There was an error running the uploader: Error uploading to https://codecov.io: Error: There was an error fetching the storage URL during POST: 429 - {'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 2452s.', code='throttled')}
Error: Codecov: Failed to properly upload: The process '/home/runner/work/_actions/codecov/codecov-action/v3/dist/codecov' failed with exit code 255

@flrgh flrgh requested review from rainest and GGabriele May 21, 2024 21:27
go.mod Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 0% with 50 lines in your changes are missing coverage. Please review.

Project coverage is 22.18%. Comparing base (509fa23) to head (47aac11).

Files Patch % Lines
cmd/common.go 0.00% 42 Missing ⚠️
cmd/gateway_dump.go 0.00% 4 Missing ⚠️
cmd/gateway_validate.go 0.00% 2 Missing ⚠️
validate/validate.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #987      +/-   ##
==========================================
- Coverage   22.42%   22.18%   -0.25%     
==========================================
  Files          54       54              
  Lines        4508     4558      +50     
==========================================
  Hits         1011     1011              
- Misses       3397     3447      +50     
  Partials      100      100              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@flrgh
Copy link

flrgh commented Jun 3, 2024

👋 @GGabriele do you think you could review or help source a reviewer for this PR?

@@ -318,6 +325,9 @@ func syncMain(ctx context.Context, filenames []string, dry bool, parallelism,
if err := checkForRBACResources(*rawState, dumpConfig.RBACResourcesOnly); err != nil {
return err
}
if err := checkFilterChainsAllowed(*rawState, dumpConfig, parsedKongVersion); err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this even needed? If decK attempts to create a filter chain on a Gateway that doesn't support it, can't we just return the API error back to the caller?

Right now decK is mostly best effort when it comes to features driven by Kong configuration, for example decK doesn't check whether a custom plugin exists before attempting to configure it.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't do some kind of feature detection, pretty much all deck commands will throw an error when executing against a Kong node without wasm support (too old version, not turned on, etc) because GET /filter-chains will return 404/405.

So you are correct that checkFilterChainsAllowed is not strictly required, but determineFilterChainSupport (feature detection) is. I would prefer to keep the checkFilterChainsAllowed guard in place because it's nicer on the user to fail fast and protects against things like incomplete sync, but I can remove it if you think it's just not a good fit with the rest of the codebase.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GET /filter-chains will return 404/405.

For reference, we do this already for Consumer Groups: https://github.com/Kong/go-database-reconciler/blob/main/pkg/dump/dump.go#L90

I'm okay if you prefer to do this extra check though.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhhhh interesting. I think I prefer it that way actually. I'll do some testing and push this logic to go-database-reconciler.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

@GGabriele GGabriele left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please add some integration tests too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants