-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Metadata limited to 500 items #10990
Comments
Thank you for taking the time to report a bug. We prioritise bugs depending on the severity and implications, so please ensure that you have provided as much information as possible. If you haven’t already, it really helps us to investigate the bug you have reported if you provide ‘Steps to Replicate’ and any associated screenshots. |
Hi @Ian-T-Price Could I ask what the use case for adding 20,000 schemas is? Do you have 20,000 The endpoint does not currently support paging as we never foresaw a use case for someone having over 500 schemas. Thanks, |
This issue has been updated and moved to our ‘Near Term’ column (typically completed within 0 - 4 months). We have assessed the effort required and outlined a technical specification - please take the time to review this detail. When we're ready to schedule the issue, it will be assigned to the relevant board where you can continue to track its progress to completion. |
It is not 20,000 entityTypes - heavens, and probably ReapIT, forbid. There is only one production schema. It is 20,000 items/records in an entityType. We currently have 10,000 items but I'm working hard to cut down both the items and the fields within the item; this will possibly get as low as 5,000 items. We'll be adding approx. 1,000 - 1,500 items per year after that. |
Hi @Ian-T-Price Thanks for confirming that makes sense now. So you are hitting the metadata endpoint to retrieve entities for your custom type 'BeresfordsReferrals20240329' I've updated the spec on the ticket as we'll need to add paging support to the endpoint. Thanks |
Hello @RWilcox-Reapit That is correct. So my next question is, when will this be scheduled? A rough timeline of days, weeks, months would be great. I am, inevitably of course, ready to go live with our app. so I need to manage expectations with Beresford. Our metadata item is no more than 2k each so we are well within the 1Mb fetch limit for DynamoDB given that we can get a max. of 100 items on a page. I therefore assumed that the 500 item limit was an artificial limit for the SandBox account only. Is this not the case? Many Thanks, Ian |
HI @Ian-T-Price Our systems have flagged a huge number of errors from your app Beresfords Referrals System today, specifically trying to query metadata against the Sandbox. The volume is so large it looks like some kind of automated process. The issue can be resolved by wrapping the string value you are filtering metadata for in single quotes as outlined in the documentation The call you are making is currently: https://platform.reapit.cloud/metadata/?PageNumber=1&PageSize=100&entityType=BeresfordsReferrals20240329&filter=metadata.ReapitID+%24eq+WHY230094 Please can you arrange to make the relevant fix to your system? To answer your previous question, it may be a month or two before we can get to this unfortunately, however I'll flag with our product owner for their attention. |
Sincere apologises for that; I'll reply via your email which I have received late this afternoon. Full explanations in the email. Many thanks for the timeline update. We can live with a month or two and I'm working on methods to allow for the limit as well. Cheers Ian |
Describe the bug
It is only possible to return 500 items from a Metadata Schema.
If a 501st item is added to the metadataSchema a seemingly random(?) item is displaced by the new item.
If an unrelated item is deleted then the displaced item reappears.
Thus, all items are being stored but only the last 500 items are available to be used.
To Reproduce
Steps to reproduce the behaviour:
Expected behaviour
All items should be available. We are looking to store upwards of 20,000 items.
Screenshots
N/A
Device (please complete the following information for web issues):
Additional context
Add any other context about the problem here.
Specification
Service: Reapit.Services.Metadata, Reapit.Lambdas.MetadataProxy
pagedResult
object so users can pass this to retrieve more recordsThe text was updated successfully, but these errors were encountered: