Bump org.rocksdb:rocksdbjni from 8.6.7 to 8.7.3

[dependabot]

Bumps org.rocksdb:rocksdbjni from 8.6.7 to 8.7.3.

Release notes

Sourced from org.rocksdb:rocksdbjni's releases.

RocksDB 8.7.3

8.7.3 (2023-10-30)

Behavior Changes

• Deleting stale files upon recovery are delegated to SstFileManger if available so they can be rate limited.

8.7.2 (2023-10-25)

Public API Changes

• Add new Cache APIs GetSecondaryCacheCapacity() and GetSecondaryCachePinnedUsage() to return the configured capacity, and cache reservation charged to the secondary cache.

Bug Fixes

• Fixed a possible underflow when computing the compressed secondary cache share of memory reservations while updating the compressed secondary to total block cache ratio.
• Fix an assertion failure when UpdeteTieredCache() is called in an idempotent manner.

8.7.1 (2023-10-20)

Bug Fixes

• Fix a bug in auto_readahead_size where first_internal_key of index blocks wasn't copied properly resulting in corruption error when first_internal_key was used for comparison.
• Add bounds check in WBWIIteratorImpl and make BaseDeltaIterator, WriteUnpreparedTxn and WritePreparedTxn respect the upper bound and lower bound in ReadOption. See 11680.

8.7.0 (2023-09-22)

New Features

• Added an experimental new "automatic" variant of HyperClockCache that does not require a prior estimate of the average size of cache entries. This variant is activated when HyperClockCacheOptions::estimated_entry_charge = 0 and has essentially the same concurrency benefits as the existing HyperClockCache.
• Add a new statistic COMPACTION_CPU_TOTAL_TIME that records cumulative compaction cpu time. This ticker is updated regularly while a compaction is running.
• Add GetEntity() API for ReadOnly DB and Secondary DB.
• Add a new iterator API Iterator::Refresh(const Snapshot *) that allows iterator to be refreshed while using the input snapshot to read.
• Added a new read option merge_operand_count_threshold. When the number of merge operands applied during a successful point lookup exceeds this threshold, the query will return a special OK status with a new subcode kMergeOperandThresholdExceeded. Applications might use this signal to take action to reduce the number of merge operands for the affected key(s), for example by running a compaction.
• For NewRibbonFilterPolicy(), made the bloom_before_level option mutable through the Configurable interface and the SetOptions API, allowing dynamic switching between all-Bloom and all-Ribbon configurations, and configurations in between. See comments on NewRibbonFilterPolicy()
• RocksDB now allows the block cache to be stacked on top of a compressed secondary cache and a non-volatile secondary cache, thus creating a three-tier cache. To set it up, use the NewTieredCache() API in rocksdb/cache.h..
• Added a new wide-column aware full merge API called FullMergeV3 to MergeOperator. FullMergeV3 supports wide columns both as base value and merge result, which enables the application to perform more general transformations during merges. For backward compatibility, the default implementation implements the earlier logic of applying the merge operation to the default column of any wide-column entities. Specifically, if there is no base value or the base value is a plain key-value, the default implementation falls back to FullMergeV2. If the base value is a wide-column entity, the default implementation invokes FullMergeV2 to perform the merge on the default column, and leaves any other columns unchanged.
• Add wide column support to ldb commands (scan, dump, idump, dump_wal) and sst_dump tool's scan command

Public API Changes

• Expose more information about input files used in table creation (if any) in CompactionFilter::Context. See CompactionFilter::Context::input_start_level,CompactionFilter::Context::input_table_properties for more.
• Options::compaction_readahead_size 's default value is changed from 0 to 2MB.
• When using LZ4 compression, the acceleration parameter is configurable by setting the negated value in CompressionOptions::level. For example, CompressionOptions::level=-10 will set acceleration=10
• The NewTieredCache API has been changed to take the total cache capacity (inclusive of both the primary and the compressed secondary cache) and the ratio of total capacity to allocate to the compressed cache. These are specified in TieredCacheOptions. Any capacity specified in LRUCacheOptions, HyperClockCacheOptions and CompressedSecondaryCacheOptions is ignored. A new API, UpdateTieredCache is provided to dynamically update the total capacity, ratio of compressed cache, and admission policy.
• The NewTieredVolatileCache() API in rocksdb/cache.h has been renamed to NewTieredCache().

Behavior Changes

• Compaction read performance will regress when Options::compaction_readahead_size is explicitly set to 0
• Universal size amp compaction will conditionally exclude some of the newest L0 files when selecting input with a small negative impact to size amp. This is to prevent a large number of L0 files from being locked by a size amp compaction, potentially leading to write stop with a few more flushes.
• Change ldb scan command delimiter from ':' to '==>'.
• For non direct IO, eliminate the file system prefetching attempt for compaction read when Options::compaction_readahead_size is 0

Bug Fixes

• Fix a bug where if there is an error reading from offset 0 of a file from L1+ and that the file is not the first file in the sorted run, data can be lost in compaction and read/scan can return incorrect results.
• Fix a bug where iterator may return incorrect result for DeleteRange() users if there was an error reading from a file.
• Fix a bug with atomic_flush=true that can cause DB to stuck after a flush fails (#11872).
• Fix a bug where RocksDB (with atomic_flush=false) can delete output SST files of pending flushes when a previous concurrent flush fails (#11865). This can result in DB entering read-only state with error message like IO error: No such file or directory: While open a file for random read: /tmp/rocksdbtest-501/db_flush_test_87732_4230653031040984171/000013.sst.
• Fix an assertion fault during seek with async_io when readahead trimming is enabled.
• When the compressed secondary cache capacity is reduced to 0, it should be completely disabled. Before this fix, inserts and lookups would still go to the backing LRUCache before returning, thus incurring locking overhead. With this fix, inserts and lookups are no-ops and do not add any overhead.

... (truncated)

Changelog

Sourced from org.rocksdb:rocksdbjni's changelog.

8.7.3 (10/30/2023)

Behavior Changes

• Deleting stale files upon recovery are delegated to SstFileManger if available so they can be rate limited.

8.7.2 (10/25/2023)

Public API Changes

• Add new Cache APIs GetSecondaryCacheCapacity() and GetSecondaryCachePinnedUsage() to return the configured capacity, and cache reservation charged to the secondary cache.

Bug Fixes

• Fixed a possible underflow when computing the compressed secondary cache share of memory reservations while updating the compressed secondary to total block cache ratio.
• Fix an assertion failure when UpdeteTieredCache() is called in an idempotent manner.

8.7.1 (10/20/2023)

Bug Fixes

• Fix a bug in auto_readahead_size where first_internal_key of index blocks wasn't copied properly resulting in corruption error when first_internal_key was used for comparison.
• Add bounds check in WBWIIteratorImpl and make BaseDeltaIterator, WriteUnpreparedTxn and WritePreparedTxn respect the upper bound and lower bound in ReadOption. See 11680.

8.7.0 (09/22/2023)

New Features

• Added an experimental new "automatic" variant of HyperClockCache that does not require a prior estimate of the average size of cache entries. This variant is activated when HyperClockCacheOptions::estimated_entry_charge = 0 and has essentially the same concurrency benefits as the existing HyperClockCache.
• Add a new statistic COMPACTION_CPU_TOTAL_TIME that records cumulative compaction cpu time. This ticker is updated regularly while a compaction is running.
• Add GetEntity() API for ReadOnly DB and Secondary DB.
• Add a new iterator API Iterator::Refresh(const Snapshot *) that allows iterator to be refreshed while using the input snapshot to read.
• Added a new read option merge_operand_count_threshold. When the number of merge operands applied during a successful point lookup exceeds this threshold, the query will return a special OK status with a new subcode kMergeOperandThresholdExceeded. Applications might use this signal to take action to reduce the number of merge operands for the affected key(s), for example by running a compaction.
• For NewRibbonFilterPolicy(), made the bloom_before_level option mutable through the Configurable interface and the SetOptions API, allowing dynamic switching between all-Bloom and all-Ribbon configurations, and configurations in between. See comments on NewRibbonFilterPolicy()
• RocksDB now allows the block cache to be stacked on top of a compressed secondary cache and a non-volatile secondary cache, thus creating a three-tier cache. To set it up, use the NewTieredCache() API in rocksdb/cache.h..
• Added a new wide-column aware full merge API called FullMergeV3 to MergeOperator. FullMergeV3 supports wide columns both as base value and merge result, which enables the application to perform more general transformations during merges. For backward compatibility, the default implementation implements the earlier logic of applying the merge operation to the default column of any wide-column entities. Specifically, if there is no base value or the base value is a plain key-value, the default implementation falls back to FullMergeV2. If the base value is a wide-column entity, the default implementation invokes FullMergeV2 to perform the merge on the default column, and leaves any other columns unchanged.
• Add wide column support to ldb commands (scan, dump, idump, dump_wal) and sst_dump tool's scan command

Public API Changes

• Expose more information about input files used in table creation (if any) in CompactionFilter::Context. See CompactionFilter::Context::input_start_level,CompactionFilter::Context::input_table_properties for more.
• Options::compaction_readahead_size 's default value is changed from 0 to 2MB.
• When using LZ4 compression, the acceleration parameter is configurable by setting the negated value in CompressionOptions::level. For example, CompressionOptions::level=-10 will set acceleration=10
• The NewTieredCache API has been changed to take the total cache capacity (inclusive of both the primary and the compressed secondary cache) and the ratio of total capacity to allocate to the compressed cache. These are specified in TieredCacheOptions. Any capacity specified in LRUCacheOptions, HyperClockCacheOptions and CompressedSecondaryCacheOptions is ignored. A new API, UpdateTieredCache is provided to dynamically update the total capacity, ratio of compressed cache, and admission policy.
• The NewTieredVolatileCache() API in rocksdb/cache.h has been renamed to NewTieredCache().

Behavior Changes

• Compaction read performance will regress when Options::compaction_readahead_size is explicitly set to 0
• Universal size amp compaction will conditionally exclude some of the newest L0 files when selecting input with a small negative impact to size amp. This is to prevent a large number of L0 files from being locked by a size amp compaction, potentially leading to write stop with a few more flushes.
• Change ldb scan command delimiter from ':' to '==>'.
• For non direct IO, eliminate the file system prefetching attempt for compaction read when Options::compaction_readahead_size is 0

Bug Fixes

• Fix a bug where if there is an error reading from offset 0 of a file from L1+ and that the file is not the first file in the sorted run, data can be lost in compaction and read/scan can return incorrect results.
• Fix a bug where iterator may return incorrect result for DeleteRange() users if there was an error reading from a file.
• Fix a bug with atomic_flush=true that can cause DB to stuck after a flush fails (#11872).
• Fix a bug where RocksDB (with atomic_flush=false) can delete output SST files of pending flushes when a previous concurrent flush fails (#11865). This can result in DB entering read-only state with error message like IO error: No such file or directory: While open a file for random read: /tmp/rocksdbtest-501/db_flush_test_87732_4230653031040984171/000013.sst.
• Fix an assertion fault during seek with async_io when readahead trimming is enabled.
• When the compressed secondary cache capacity is reduced to 0, it should be completely disabled. Before this fix, inserts and lookups would still go to the backing LRUCache before returning, thus incurring locking overhead. With this fix, inserts and lookups are no-ops and do not add any overhead.
• Updating the tiered cache (cache allocated using NewTieredCache()) by calling SetCapacity() on it was not working properly. The initial creation would set the primary cache capacity to the combined primary and compressed secondary cache capacity. But SetCapacity() would just set the primary cache capacity. With this fix, the user always specifies the total budget and compressed secondary cache ratio on creation. Subsequently, SetCapacity() will distribute the new capacity across the two caches by the same ratio.

... (truncated)

Commits

• 596718b Fix the ZStd checksum (#12005)
• c9ea61b Update HISTORY.md and version for 8.7.3
• 75fab59 Rate limiting stale sst files' deletion during recovery (#12016)
• b9938eb Update HISTORY.md and version for 8.7.2
• 611ba94 Add APIs to query secondary cache capacity and usage for TieredCache (#12011)
• cb04d40 Fix windows build errors (rdtsc and fnptr) (#12008)
• 6b230bc Fix runtime error in UpdateTieredCache due to integer underflow (#11949)
• 0847b70 Add TieredCache and compressed cache capacity change to db_stress (#11935)
• 60a6488 update HISTORY.md and version.h for 8.7.1
• dd94499 Add bounds checking to WBWIIteratorImpl and respect bounds of ReadOptions in ...
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot]

Superseded by #399.