Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add progress update. #308

Merged
merged 1 commit into from Oct 28, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/phase1.hpp
Expand Up @@ -598,6 +598,7 @@ std::vector<uint64_t> RunPhase1(
uint8_t const flags)
{
std::cout << "Computing table 1" << std::endl;
std::cout << "Progress update: 0.01" << std::endl;
globals.stripe_size = stripe_size;
globals.num_threads = num_threads;
Timer f1_start_time;
Expand Down Expand Up @@ -644,6 +645,7 @@ std::vector<uint64_t> RunPhase1(

// For tables 1 through 6, sort the table, calculate matches, and write
// the next table. This is the left table index.
double progress_percent[] = {0.06, 0.12, 0.2, 0.28, 0.36, 0.42};
for (uint8_t table_index = 1; table_index < 7; table_index++) {
Timer table_timer;
uint8_t const metadata_size = kVectorLens[table_index + 1] * k;
Expand All @@ -664,6 +666,7 @@ std::vector<uint64_t> RunPhase1(
}

std::cout << "Computing table " << int{table_index + 1} << std::endl;
std::cout << "Progress update: " << progress_percent[table_index - 1] << std::endl;
// Start of parallel execution

FxCalculator f(k, table_index + 1); // dummy to load static table
Expand Down
3 changes: 2 additions & 1 deletion src/phase2.hpp
Expand Up @@ -94,10 +94,11 @@ Phase2Results RunPhase2(
// note that we don't iterate over table_index=1. That table is special
// since it contains different data. We'll do an extra scan of table 1 at
// the end, just to compact it.
double progress_percent[] = {0.43, 0.48, 0.51, 0.55, 0.58, 0.61};
for (int table_index = 7; table_index > 1; --table_index) {

std::cout << "Backpropagating on table " << table_index << std::endl;

std::cout << "Progress update: " << progress_percent[7 - table_index] << std::endl;
Timer scan_timer;

next_bitfield.clear();
Expand Down
2 changes: 2 additions & 0 deletions src/phase3.hpp
Expand Up @@ -160,11 +160,13 @@ Phase3Results RunPhase3(
// new_pos), where new_pos is the position in the table, where it's sorted by line_point,
// and the line_points are written to disk to a final table. Finally, table_i is sorted by
// sort_key. This allows us to compare to the next table.
double progress_percent[] = {0.66, 0.73, 0.79, 0.85, 0.92, 0.98};
for (int table_index = 1; table_index < 7; table_index++) {
Timer table_timer;
Timer computation_pass_1_timer;
std::cout << "Compressing tables " << table_index << " and " << (table_index + 1)
<< std::endl;
std::cout << "Progress update: " << progress_percent[table_index - 1] << std::endl;

// The park size must be constant, for simplicity, but must be big enough to store EPP
// entries. entry deltas are encoded with variable length, and thus there is no
Expand Down