Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use groupby instead of vec allocation #1582

Merged
merged 1 commit into from Oct 4, 2022
Merged

Conversation

PSeitz
Copy link
Contributor

@PSeitz PSeitz commented Oct 3, 2022

get_sorted_field_values is quite expensive and usually visible in traces. The allocations are unnecessary though.

Before (13 mulitivalue fast fields)
-    9,72%     1,21%  thrd-tantivy-in  tantivy               [.] tantivy::indexer::segment_writer::SegmentWriter::add_document                                                         
   - 8,51% tantivy::indexer::segment_writer::SegmentWriter::add_document                                                                                                               
      + 3,03% tantivy::store::store_compressor::BlockCompressor::compress_block_and_write                                                                                              
      + 2,42% tantivy::schema::document::Document::get_sorted_field_values                                                                                                             
      + 1,89% cfree                                                                                                                                                                    
        0,95% tantivy::fastfield::multivalued::writer::MultiValuedFastFieldWriter::add_document                                                                                        


After
-    8,10%     1,65%  thrd-tantivy-in  tantivy               [.] tantivy::indexer::segment_writer::SegmentWriter::add_document                                                        
   - 6,45% tantivy::indexer::segment_writer::SegmentWriter::add_document                                                                                                               
      + 3,52% tantivy::store::store_compressor::BlockCompressor::compress_block_and_write                                                                                              
        1,28% tantivy::fastfield::multivalued::writer::MultiValuedFastFieldWriter::add_document                                                                                        
        1,21% cfree                                                                                                                                                                    
   + 1,65% __clone                                                                                                                                                                     

@codecov-commenter
Copy link

Codecov Report

Merging #1582 (b5919ff) into main (44e0379) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1582      +/-   ##
==========================================
- Coverage   93.79%   93.78%   -0.02%     
==========================================
  Files         251      251              
  Lines       46329    46332       +3     
==========================================
- Hits        43456    43454       -2     
- Misses       2873     2878       +5     
Impacted Files Coverage Δ
src/indexer/segment_writer.rs 96.39% <100.00%> (+0.01%) ⬆️
src/schema/document.rs 92.81% <0.00%> (-1.31%) ⬇️
src/fastfield/multivalued/mod.rs 97.23% <0.00%> (-1.19%) ⬇️
src/postings/segment_postings.rs 98.95% <0.00%> (-0.53%) ⬇️
.../termdict/sstable_termdict/sstable/block_reader.rs 82.14% <0.00%> (+1.78%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@PSeitz PSeitz requested a review from fulmicoton October 3, 2022 10:26
@PSeitz PSeitz merged commit 0f4a478 into main Oct 4, 2022
@PSeitz PSeitz deleted the faster_sorted_field_values branch October 4, 2022 01:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants