Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change ElementProps acquisition to a prepared statement query #6549

Draft
wants to merge 41 commits into
base: master
Choose a base branch
from

Conversation

6ar8nas
Copy link

@6ar8nas 6ar8nas commented Mar 25, 2024

This changes how we fetch ElementProps from the nativeDB. It allows bypassing the creation of a DgnElement on the native side and various json mapping adapters, which should result in both more accurate DB data representation, as well as considerably better Elements API performance.

To support this, a mapping function has been drafted that would allow all consuming applications to use the following dollar syntax query without the need to match the JSON representation to the accurate ElementProps extending interface.

@6ar8nas 6ar8nas marked this pull request as ready for review March 25, 2024 09:39
@6ar8nas 6ar8nas requested a review from a team as a code owner March 25, 2024 09:39
@pmconne
Copy link
Member

pmconne commented Mar 25, 2024

iModel-native PR: iTwin/imodel-native#688

Instructions for correctly linking your PRs. Don't include "PR", and I'm not sure if it's case-sensitive.

@6ar8nas
Copy link
Author

6ar8nas commented Mar 25, 2024

/azp run iTwin.js

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@6ar8nas 6ar8nas marked this pull request as draft March 27, 2024 10:41
@kabentley
Copy link
Contributor

Can you please explain the motivation behind this change?

@6ar8nas
Copy link
Author

6ar8nas commented May 23, 2024

Can you please explain the motivation behind this change?

Sorry, didn't expect much input on a draft PR yet, so didn't notice the question.

To my knowledge, the motivation is not only it being faster. We already have an interface how we can get data as a stringified JSON without some intermittent layers using the dollar syntax. This is why the expected outcome is that it should be faster, though some testing is soon to be carried out comparing with the current benchmark.

This change would result in a more accurate representation of actual data stored in DB. To illustrate, previously in some cases it seemed to not portray null struct information correctly #itwinjs-backlog/1057; even though code value is unset, previously it was always returned as an empty string. The current getElement output is using EC instance JSON adapters and it seems like it's not really that well extendable and we've had some discussions that ideally we would like to move away from it.

Applications seldom had their own reasons to do the mappings themselves (e.g. iTwin/imodel-transformer in the past). This would streamline mapping by delivering a new mapping function.

I might be missing something, but these are some of the key points. If anything else raises any concerns, please let me know.

@kabentley
Copy link
Contributor

kabentley commented May 24, 2024

To my knowledge, the motivation is not only it being faster.

I am very skeptical that this change will result in appreciable performance difference - and it's not clear to me which way will be faster. The DgnEelement "cost" you endeavor to eliminate adds caching, which for editing applications (where the same set of elements is read repeatedly) helps tremendously. I'm not sure eliminating it is a good idea at all.

This change would result in a more accurate representation of actual data stored in DB.

If there are bugs in the native code, we should fix them.

Applications seldom had their own reasons to do the mappings themselves (e.g. iTwin/imodel-transformer in the past). This would streamline mapping by delivering a new mapping function.

My strong suggestion is that this change should be implemented in native code, not here. I suppose you can run your tests with this change and see what the profiler says.

@6ar8nas 6ar8nas marked this pull request as ready for review May 24, 2024 12:23
@6ar8nas
Copy link
Author

6ar8nas commented May 24, 2024

/azp run iTwin.js

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@pmconne
Copy link
Member

pmconne commented May 24, 2024

more accurate DB data representation

Can you be more specific?

considerably better Elements API performance

Have you measured this?

@6ar8nas 6ar8nas marked this pull request as draft May 29, 2024 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants