[4.x]馃悰 Fixed member exports timing out for large sites #14876
Conversation
Codecov Report
@@ Coverage Diff @@
## 4.x #14876 +/- ##
==========================================
- Coverage 59.81% 59.72% -0.10%
==========================================
Files 580 581 +1
Lines 48038 48121 +83
Branches 4220 4221 +1
==========================================
+ Hits 28735 28739 +4
- Misses 19262 19341 +79
Partials 41 41
Continue to review full report at Codecov.
|
SimonBackx
force-pushed
the
improved-members-export-performance-4-x
branch
2 times, most recently
from
e8ebd4e
to
e3fd191
Compare
SimonBackx
force-pushed
the
improved-members-export-performance-4-x
branch
from
e3fd191
to
00b6757
Compare
| const page = await models.Member.findPage(options); | ||
| ids = page.data.map(d => d.id); | ||
|
|
||
| /* |
There was a problem hiding this comment.
Could you add a comment line explaining why this is commented out?
There was a problem hiding this comment.
Yeah, I tried last minute to improve the id fetching. But it didn't work reliably (I think filtering with search didn't work). I still think the commented code would have better performance if that is fixed because it 'escapes' from bookshelf. So I quickly commented it out before switching branches to work on the tests for #14873 (switching between 4.x and 5.x takes a bit longer with the client/admin rename). Could have stashed the changes, but then I might forget about it 馃槵
| @@ -1554,7 +1554,7 @@ describe('Members API', function () { | |||
| 'content-disposition': anyString | |||
| }); | |||
|
|
|||
| res.text.should.match(/id,email,name,note,subscribed_to_emails,complimentary_plan,stripe_customer_id,created_at,deleted_at/); | |||
There was a problem hiding this comment.
Is there a reason to keep these 2 tests in members.test.js when we've got a suite in members-exporter.test.js now? Seems like they are duplicates?
There was a problem hiding this comment.
I don't have any tests yet for the search option in the other file, so I think it is best to keep this one until we have a new one for search
There was a problem hiding this comment.
One of them is no search, I just noticed. We can clean it up later :)
refs TryGhost/Product#1641 This commit adds a custom query for the members export, to improve the performance and to prevent any timeouts from happening when exporting large amounts of members.
refs TryGhost/Product#1641 This commit adds a custom query for the members export, to improve the performance and to prevent any timeouts from happening when exporting large amounts of members. Co-authored-by: Simon Backx <simon@ghost.org> Co-authored-by: Matt Hanley <git@matthanley.co.uk>
refs https://github.com/TryGhost/Team/issues/1641