Manually trigger major compaction

[v0y4g3r]

What problem does the new feature solve?

Currently GrepgtimeDB supports triggering compaction using SELECT COMPACT_TABLE, but this statement only triggers regular compaction that assigns SSTs into windows and compact those files inside same window together to alleviate the overhead of compactions. We also need to support more compaction strategies when it's manually triggered so that we can enforce a more efficient data layout.

What does the feature do?

Adds options to SELECT COMPACT_TABLE statements so that we can trigger major compactions using strategies like:

• merge all SST files in some time range into one single file
• split SST files strictly by time windows
  • the time window can be inferred from level 0 files or manually specified

We can start with the easy one which splits SST files according to time windows
The syntax would be like:

SELECT COMPACT_TABLE(some_table, type="strict_window");

Before the compaction, suppose the data layout is:

We can find that these 3 files overlaps with each other.

And we expect the compaction output is like (given time window is 50):

Implementation challenges

We have to accommodate different compaction startegies in current compaction framework and reuse the scheduling and IO component.