New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add an example to create a virtual dataset with FancySelection #1772
base: master
Are you sure you want to change the base?
Conversation
…tion on VirtualSource
c42a1f8
to
df16186
Compare
Codecov Report
@@ Coverage Diff @@
## master #1772 +/- ##
==========================================
+ Coverage 88.51% 89.91% +1.40%
==========================================
Files 17 17
Lines 2273 2311 +38
==========================================
+ Hits 2012 2078 +66
+ Misses 261 233 -28
Continue to review full report at Codecov.
|
In fact I am not sure this is the best way to go. In this example I need to acces |
Does What's the performance like with this? I recall that making many small separate mappings for a virtual dataset caused poor performance when reading from it, and I'd be somewhat concerned that fancy indexing might be similar, as it's lots of small hyperslabs in the source selection. |
In fact it does. It call also 'select'. The same operation could be done by (for the first selection) v_source_1 = h5py.VirtualSource(file_path, "original_data/d1", shape=(10, 10, 10))
v_source_1 = v_source_1[[2, 3]] which is in fact more convenient thank you. I will update the example.
Make sense. I didn't had a look at performances yet. |
…ing getitem instead
I think the remaining bit before this can be merged is the removal of |
Thanks. In fact I was no more using the 'h5py._hl.selections' directly. I removed the import and improved file formatting (black - flake). Let me know if there is more improvements to be done. |
A simple example on how we can create FancySelection for VirtualSource (and as a consequence virtual dataset).
It would have been useful to me so maybe it could help someone else in the future.