New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add AllPages pagination helper #1875
base: main
Are you sure you want to change the base?
Conversation
0d4604e
to
9067982
Compare
Oh, this is embarrassing, I totally missed #1741 when searching for previous work 😕 This implementation is considerably different though, so I'll leave the PR open for now. |
Looks cool but what I am missing from my PR (#1741) is the ability to 1. somehow allow the pagination to abort on some criteria and 2. the ability to immediately process each result. Both are a bit of edge cases but can come in handy in a few situations. |
I guess the iterator approach basically solves both your points? |
Ah true, I misread that. Seems that the iterator approach could really do those things. That would be fantastic and way less clunky than my previous version. |
Just for the record: note that the iterator doesn't technically require the If go1.23 turns out to break the |
It took me at least 5min to understand this line |
Just 5min? Then you were faster than me! 😅 Seriously though, I think renaming the type parameters to something more verbose may help? And I think as long as the usage is straightforward (which IMHO it still is), then the implementation is allowed to look a bit gnarly. As a side-note, I'm personally still a bit torn on the rangefunc stuff. Powerful and elegant (at least from a runtime perspective) but twists my brain in ways I'm not yet comfortable with. |
Yep ! For me, that feeling started with the introduction of generics. As a "user"/consumer, they are a joy to use. As a code author/maintainer, they are a pain. Some functions take an additional func XXYYY[O, T any](pid interface{}, f func(interface{}, *O, ...gitlab.RequestOptionFunc) ([]*T, *gitlab.Response, error)) Paginatable[O, T] {
return func(opt *O, optionFunc ...gitlab.RequestOptionFunc) ([]*T, *gitlab.Response, error) {
return f(pid, opt, optionFunc...)
}
} It's a function that returns a closure, whose signature matches Would that make sense to include such function in that |
iterator.go
Outdated
return func(yield func(*T, error) bool) { | ||
nextLink := "" | ||
for { | ||
page, resp, err := f(opt, WithKeysetPaginationParameters(nextLink)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The function f
accepts a variable number of RequestOptionFunc
arguments, but in this call here, it passes a single one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe something like that
// for user, err := range gitlab.PageIterator(gl.Users.List, nil, gitlab.WithContext(ctx)) {
// if err != nil {
// // handle error
// }
// // process individual user
// }
func PageIterator[O, T any](f Paginatable[O, T], opt *O, optionFunc ...gitlab.RequestOptionFunc) iter.Seq2[*T, error] {
return func(yield func(*T, error) bool) {
nextLink := ""
for {
page, resp, err := f(opt, append(optionFunc, gitlab.WithKeysetPaginationParameters(nextLink))...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah yes, good point! Thankfully a quick fix. PTAL
141d142
to
cb6dd4c
Compare
cb6dd4c
to
fb83965
Compare
Also a good point. However, I'm not sure about the closure approach. It's elegant, but I fear it may not be directly clear for consumers. The two versions unfortunately only cover around 60% of list-like methods: if my grep-foo is correct, there are currently 187 list-like methods in total, 25 of the "simple" opt+optFunc type and 85 of the id+opt+optFunc type. |
This small PR finally addresses #1320, picking up where the discussion left off.
Naming is - of course - up for discussion! :)
Update: the PR now also includes an experimental iterator based on the
rangefunc
experiment.