New validation API #236
New validation API #236
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
oxisto
force-pushed
the
new-validation-api
branch
from
115d547
to
1b437d8
Compare
|
An initial version of this is now ready to review. I will add some more tests once we agree on a more or less stable API. |
Some guidelines in designing the new validation API * Previously, the `Valid` method was placed on the claim, which was always not entirely semantically correct, since the validity is concerning the token, not the claims. Although the validity of the token is based on the processing of the claims (such as `exp`). Therefore, the function `Valid` was removed from the `Claims` interface and the single canonical way to retrieve the validity of the token is to retrieve the `Valid` property of the `Token` struct. * The previous fact was enhanced by the fact that most claims implementations had additional exported `VerifyXXX` functions, which are now removed * All validation errors should be comparable with `errors.Is` to determine, why a particular validation has failed * Developers want to adjust validation options. Popular options include: * Leeway when processing exp, nbf, iat * Not verifying `iat`, since this is actually just an informational claim. When purely looking at the standard, this should probably the default * Verifying `aud` by default, which actually the standard sort of demands. We need to see how strong we want to enforce this * Developers want to create their own claim types, mostly by embedding one of the existing types such as `RegisteredClaims`. * Sometimes there is the need to further tweak the validation of a token by checking the value of a custom claim. Previously, this was possibly by overriding `Valid`. However, this was error-prone, e.g., if the original `Valid` was not called. Therefore, we should provide an easy way for *additional* checks, without by-passing the necessary validations This leads to the following two major changes: * The `Claims` interface now represents a set of functions that return the mandatory claims represented in a token, rather than just a `Valid` function. This is also more semantically correct. * All validation tasks are offloaded to a new (optional) `Validator`, which can also be configured with appropriate options. If no custom validator was supplied, a default one is used.
oxisto
force-pushed
the
new-validation-api
branch
from
5735b9c
to
eedf3eb
Compare
|
I haven't review these changes thoroughly, but glancing at the PR description, I feel neutral or favorable about the suggested changes. I'd be happy to review this PR if requested, but that would have to be next week at the earliest due to my current obligations. I'm interested in resolving this PR for two reasons:
|
Your feedback on this PR would be most welcome! |
|
What is ETA for v5? |
There is no fixed ETA. It is kind of relying on the co-maintainers having some time to review this PR :) |
|
@MicahParks did you already have time to look at this a more closely? |
Not yet, sorry. But it's still on my list of things to do. |
|
@oxisto Thank you for the reminder yesterday, I've reviewed the PR today and will be posting it shortly. Apologies in advance for any comments of mine that suggest improper or inconsistent things for this code base. I haven't familiarized myself with its entirety. I'm mostly familiar with the code surrounding Also, it seems some comments were removed on a few exported assets such as |
|
Very nice work! This PR enables parsing and validation of the token using different APIs, so a token can be parsed once and validated by different validators. This is very useful for implementation of token validation profiles described in the JSON Web Token Best Current Practices RFC. |
|
Thanks for the initial bunch of reviews, @MicahParks, @sermojohn. I incorporated most / all of your feedback. In re-enabling the map claim tests I stumbled upon a design flaw of the first iteration, in which the |
|
Anything to add @mfridman ? I would |
|
|
||
| // Claims must just have a Valid method that determines | ||
| // if the token is invalid for any supported reason | ||
| // Claims represent any form of a JWT Claims Set according to |
There was a problem hiding this comment.
Have you considered an alternate scenario where we remove this (and map claims) entirely?
It might be a bad idea, but just throwing this thought out .. the user still has to assert the claims.
There was a problem hiding this comment.
I think a lot of people still use map claims, so I am not 100 % convinced tbh.
There was a problem hiding this comment.
Ye that's fair. My thought was .. there are too many ways to pass "claims"
- map claims (default)
- RegisteredClaims (structured, arguably could be the default?)
- Custom claims that either implement Claims or embed RegisteredClaims
If we remove Claims interface entirely, and take an any .. would that simply the overall API?
Probably a bad idea, figured I'd bring it up but I think you're probably right .. a lot of people use map claims.
There was a problem hiding this comment.
I think it actually gets easier with the new interface, because you do not need to implement any validation on your claim, but rather you can use any kind of struct/map type that supplies the needed standard claim values, such as exp, iat, etc. I think that is a pretty straight forward way of modelling claims now.
I made another cleanup pass and this should be fairly straightforward now. All validation options are now parser options and this has the added benefit that they also work with the token, err := jwt.ParseWithClaims(tokenString, &MyCustomClaims{}, func(token *jwt.Token) (interface{}, error) {
return []byte("AllYourBase"), nil
}, jwt.WithLeeway(5*time.Second))or as part of a new parser parser := NewParser(jwt.WithLeeway(5*time.Second))If necessary, it is still possible to create a new (standalone) validator with `NewValidator: val := NewValidator(jwt.WithLeeway(5*time.Second))This is fairly close to what @dgrijalva originally had in the "old" v4 validation API. |
|
Overall the new API looks good from my side. I like how we're frontloading the options in one place. The only options I'd suggest removing are Happy to give this a stamp and merge it into the |
Yap, looks good. Waiting for your official approval to merge it into |
* New Validation API Some guidelines in designing the new validation API * Previously, the `Valid` method was placed on the claim, which was always not entirely semantically correct, since the validity is concerning the token, not the claims. Although the validity of the token is based on the processing of the claims (such as `exp`). Therefore, the function `Valid` was removed from the `Claims` interface and the single canonical way to retrieve the validity of the token is to retrieve the `Valid` property of the `Token` struct. * The previous fact was enhanced by the fact that most claims implementations had additional exported `VerifyXXX` functions, which are now removed * All validation errors should be comparable with `errors.Is` to determine, why a particular validation has failed * Developers want to adjust validation options. Popular options include: * Leeway when processing exp, nbf, iat * Not verifying `iat`, since this is actually just an informational claim. When purely looking at the standard, this should probably the default * Verifying `aud` by default, which actually the standard sort of demands. We need to see how strong we want to enforce this * Developers want to create their own claim types, mostly by embedding one of the existing types such as `RegisteredClaims`. * Sometimes there is the need to further tweak the validation of a token by checking the value of a custom claim. Previously, this was possibly by overriding `Valid`. However, this was error-prone, e.g., if the original `Valid` was not called. Therefore, we should provide an easy way for *additional* checks, without by-passing the necessary validations This leads to the following two major changes: * The `Claims` interface now represents a set of functions that return the mandatory claims represented in a token, rather than just a `Valid` function. This is also more semantically correct. * All validation tasks are offloaded to a new (optional) `validator`, which can also be configured with appropriate options. If no custom validator was supplied, a default one is used. Co-authored-by: Micah Parks <66095735+MicahParks@users.noreply.github.com>
|
The old |
[](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/golang-jwt/jwt/v4](https://togithub.com/golang-jwt/jwt) | require | major | `v4.5.0` -> `v5.0.0` | --- ### ⚠ Dependency Lookup Warnings ⚠ Warnings were logged while processing this repo. Please check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>golang-jwt/jwt (github.com/golang-jwt/jwt/v4)</summary> ### [`v5.0.0`](https://togithub.com/golang-jwt/jwt/releases/tag/v5.0.0) [Compare Source](https://togithub.com/golang-jwt/jwt/compare/v4.5.0...v5.0.0) ### 🚀 New Major Version `v5` 🚀 It's finally here, the release you have been waiting for! We don't take breaking changes lightly, but the changes outlined below were necessary to address some of the challenges of the previous API. A big thanks for [@​mfridman](https://togithub.com/mfridman) for all the reviews, all contributors for their commits and of course [@​dgrijalva](https://togithub.com/dgrijalva) for the original code. I hope we kept some of the spirit of your original `v4` branch alive in the approach we have taken here. \~[@​oxisto](https://togithub.com/oxisto), on behalf of [@​golang-jwt/maintainers](https://togithub.com/golang-jwt/maintainers) Version `v5` contains a major rework of core functionalities in the `jwt-go` library. This includes support for several validation options as well as a re-design of the `Claims` interface. Lastly, we reworked how errors work under the hood, which should provide a better overall developer experience. Starting from [v5.0.0](https://togithub.com/golang-jwt/jwt/releases/tag/v5.0.0), the import path will be: "github.com/golang-jwt/jwt/v5" For most users, changing the import path *should* suffice. However, since we intentionally changed and cleaned some of the public API, existing programs might need to be updated. The following sections describe significant changes and corresponding updates for existing programs. #### Parsing and Validation Options Under the hood, a new `validator` struct takes care of validating the claims. A long awaited feature has been the option to fine-tune the validation of tokens. This is now possible with several `ParserOption` functions that can be appended to most `Parse` functions, such as `ParseWithClaims`. The most important options and changes are: - Added `WithLeeway` to support specifying the leeway that is allowed when validating time-based claims, such as `exp` or `nbf`. - Changed default behavior to not check the `iat` claim. Usage of this claim is OPTIONAL according to the JWT RFC. The claim itself is also purely informational according to the RFC, so a strict validation failure is not recommended. If you want to check for sensible values in these claims, please use the `WithIssuedAt` parser option. - Added `WithAudience`, `WithSubject` and `WithIssuer` to support checking for expected `aud`, `sub` and `iss`. - Added `WithStrictDecoding` and `WithPaddingAllowed` options to allow previously global settings to enable base64 strict encoding and the parsing of base64 strings with padding. The latter is strictly speaking against the standard, but unfortunately some of the major identity providers issue some of these incorrect tokens. Both options are disabled by default. #### Changes to the `Claims` interface ##### Complete Restructuring Previously, the claims interface was satisfied with an implementation of a `Valid() error` function. This had several issues: - The different claim types (struct claims, map claims, etc.) then contained similar (but not 100 % identical) code of how this validation was done. This lead to a lot of (almost) duplicate code and was hard to maintain - It was not really semantically close to what a "claim" (or a set of claims) really is; which is a list of defined key/value pairs with a certain semantic meaning. Since all the validation functionality is now extracted into the validator, all `VerifyXXX` and `Valid` functions have been removed from the `Claims` interface. Instead, the interface now represents a list of getters to retrieve values with a specific meaning. This allows us to completely decouple the validation logic with the underlying storage representation of the claim, which could be a struct, a map or even something stored in a database. ```go type Claims interface { GetExpirationTime() (*NumericDate, error) GetIssuedAt() (*NumericDate, error) GetNotBefore() (*NumericDate, error) GetIssuer() (string, error) GetSubject() (string, error) GetAudience() (ClaimStrings, error) } ``` ##### Supported Claim Types and Removal of `StandardClaims` The two standard claim types supported by this library, `MapClaims` and `RegisteredClaims` both implement the necessary functions of this interface. The old `StandardClaims` struct, which has already been deprecated in `v4` is now removed. Users using custom claims, in most cases, will not experience any changes in the behavior as long as they embedded `RegisteredClaims`. If they created a new claim type from scratch, they now need to implemented the proper getter functions. ##### Migrating Application Specific Logic of the old `Valid` Previously, users could override the `Valid` method in a custom claim, for example to extend the validation with application-specific claims. However, this was always very dangerous, since once could easily disable the standard validation and signature checking. In order to avoid that, while still supporting the use-case, a new `ClaimsValidator` interface has been introduced. This interface consists of the `Validate() error` function. If the validator sees, that a `Claims` struct implements this interface, the errors returned to the `Validate` function will be *appended* to the regular standard validation. It is not possible to disable the standard validation anymore (even only by accident). Usage examples can be found in [example_test.go](./example_test.go), to build claims structs like the following. ```go // MyCustomClaims includes all registered claims, plus Foo. type MyCustomClaims struct { Foo string `json:"foo"` jwt.RegisteredClaims } // Validate can be used to execute additional application-specific claims // validation. func (m MyCustomClaims) Validate() error { if m.Foo != "bar" { return errors.New("must be foobar") } return nil } ``` #### Changes to the `Token` and `Parser` struct The previously global functions `DecodeSegment` and `EncodeSegment` were moved to the `Parser` and `Token` struct respectively. This will allow us in the future to configure the behavior of these two based on options supplied on the parser or the token (creation). This also removes two previously global variables and moves them to parser options `WithStrictDecoding` and `WithPaddingAllowed`. In order to do that, we had to adjust the way signing methods work. Previously they were given a base64 encoded signature in `Verify` and were expected to return a base64 encoded version of the signature in `Sign`, both as a `string`. However, this made it necessary to have `DecodeSegment` and `EncodeSegment` global and was a less than perfect design because we were repeating encoding/decoding steps for all signing methods. Now, `Sign` and `Verify` operate on a decoded signature as a `[]byte`, which feels more natural for a cryptographic operation anyway. Lastly, `Parse` and `SignedString` take care of the final encoding/decoding part. In addition to that, we also changed the `Signature` field on `Token` from a `string` to `[]byte` and this is also now populated with the decoded form. This is also more consistent, because the other parts of the JWT, mainly `Header` and `Claims` were already stored in decoded form in `Token`. Only the signature was stored in base64 encoded form, which was redundant with the information in the `Raw` field, which contains the complete token as base64. ```go type Token struct { Raw string // Raw contains the raw token Method SigningMethod // Method is the signing method used or to be used Header map[string]interface{} // Header is the first segment of the token in decoded form Claims Claims // Claims is the second segment of the token in decoded form Signature []byte // Signature is the third segment of the token in decoded form Valid bool // Valid specifies if the token is valid } ``` Most (if not all) of these changes should not impact the normal usage of this library. Only users directly accessing the `Signature` field as well as developers of custom signing methods should be affected. #### What's Changed - Added GitHub Actions Markdown by [@​oxisto](https://togithub.com/oxisto) in [golang-jwt/jwt#260 - Remove `StandardClaims` in favor of `RegisteredClaims` by [@​oxisto](https://togithub.com/oxisto) in [#​235](https://togithub.com/golang-jwt/jwt/issues/235) - Adding more coverage by [@​oxisto](https://togithub.com/oxisto) in [#​268](https://togithub.com/golang-jwt/jwt/issues/268) - More consistent way of handling validation errors by [@​oxisto](https://togithub.com/oxisto) in [#​274](https://togithub.com/golang-jwt/jwt/issues/274) - New Validation API by [@​oxisto](https://togithub.com/oxisto) in [golang-jwt/jwt#236 - `v5` Pre-Release by [@​oxisto](https://togithub.com/oxisto) in [golang-jwt/jwt#234 - no need for string slice and call to strings.join by [@​moneszarrugh](https://togithub.com/moneszarrugh) in [golang-jwt/jwt#115 - Update MIGRATION_GUIDE.md by [@​liam-verta](https://togithub.com/liam-verta) in [golang-jwt/jwt#289 - Moving `DecodeSegement` to `Parser` by [@​oxisto](https://togithub.com/oxisto) in [golang-jwt/jwt#278 - Adjusting the error checking example by [@​oxisto](https://togithub.com/oxisto) in [golang-jwt/jwt#270 - add documentation to hmac `Verify` & `Sign` to detail why string is not an advisable input for key by [@​dillonstreator](https://togithub.com/dillonstreator) in [golang-jwt/jwt#249 - Add golangci-lint by [@​mfridman](https://togithub.com/mfridman) in [golang-jwt/jwt#279 - Added dependabot updates for GitHub actions by [@​oxisto](https://togithub.com/oxisto) in [golang-jwt/jwt#298 - Bump actions/checkout from 2 to 3 by [@​dependabot](https://togithub.com/dependabot) in [golang-jwt/jwt#299 - Bump actions/setup-go from 3 to 4 by [@​dependabot](https://togithub.com/dependabot) in [golang-jwt/jwt#300 - Added coverage reporting by [@​oxisto](https://togithub.com/oxisto) in [golang-jwt/jwt#304 - Last Documentation cleanups for `v5` release by [@​oxisto](https://togithub.com/oxisto) in [golang-jwt/jwt#291 - enable jwt.ParsePublicKeyFromPEM to parse PKCS1 Public Key by [@​twocs](https://togithub.com/twocs) in [golang-jwt/jwt#120 #### New Contributors - [@​moneszarrugh](https://togithub.com/moneszarrugh) made their first contribution in [golang-jwt/jwt#115 - [@​liam-verta](https://togithub.com/liam-verta) made their first contribution in [golang-jwt/jwt#289 - [@​dillonstreator](https://togithub.com/dillonstreator) made their first contribution in [golang-jwt/jwt#249 - [@​dependabot](https://togithub.com/dependabot) made their first contribution in [golang-jwt/jwt#299 - [@​twocs](https://togithub.com/twocs) made their first contribution in [golang-jwt/jwt#120 **Full Changelog**: golang-jwt/jwt@v4.5.0...v5.0.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/woodpecker-ci/woodpecker). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi44My4wIiwidXBkYXRlZEluVmVyIjoiMzYuODMuMCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: qwerty287 <ndev@web.de>
](https://renovatebot.com){kind=link}
Some guidelines in designing the new validation API
Validmethod was placed on the claim, which was always not entirely semantically correct, since the validity is concerning the token, not the claims. Although the validity of the token is based on the processing of the claims (such asexp). Therefore, the functionValidwas removed from theClaimsinterface and the single canonical way to retrieve the validity of the token is to retrieve theValidproperty of theTokenstruct.VerifyXXXfunctions, which are now removederrors.Isto determine, why a particular validation has failediat, since this is actually just an informational claim. When purely looking at the standard, this should probably the default (Make issued at (iat) claim validation optional #175)audby default, which actually the standard sort of demands. We need to see how strong we want to enforce thisRegisteredClaims.Valid. However, this was error-prone, e.g., if the originalValidwas not called. Therefore, we should provide an easy way for additional checks, without by-passing the necessary validationsThis leads to the following two major changes:
Claimsinterface now represents a set of functions that return the mandatory claims represented in a token, such asexp, and so on, rather than just aValidfunction. This is also more semantically correct and makes our codebase smaller and way more maintainable.Validator, which can also be configured with appropriate options. If no custom validator was supplied, a default one is used.Some smaller goodies:
ParseXXX(naming to change) functions, that make it easy to parse a key as a certain JWT special type such as numeric date.Remaining tasks
iatTimeFuncto the validator instead of a global functionaudverification default-ish, at least if a new validator is provided