Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is WriteBatchWithIndex supported? #836

Open
tisonkun opened this issue Nov 23, 2023 · 4 comments
Open

Is WriteBatchWithIndex supported? #836

tisonkun opened this issue Nov 23, 2023 · 4 comments

Comments

@tisonkun
Copy link

I ever wrote a Java program with rocksdbjni -

import io.korandoru.zeronos.server.record.BackendRangeResult;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Predicate;
import lombok.extern.slf4j.Slf4j;
import org.rocksdb.RocksDB;
import org.rocksdb.RocksDBException;
import org.rocksdb.RocksIterator;
import org.rocksdb.WriteBatchWithIndex;
import org.rocksdb.WriteOptions;

@Slf4j
public class RocksDBRWBatch implements WriteTxn {

    private final WriteBatchWithIndex batch;
    private final RocksDB db;

    public RocksDBRWBatch(RocksDB db) {
        this.db = db;
        this.batch = new WriteBatchWithIndex();
    }

    @Override
    public BackendRangeResult unsafeRange(Namespace ns, byte[] key, byte[] end, long limit) {
        final long bound;
        final Predicate<byte[]> isMatch;
        if (end == null) {
            bound = 1;
            isMatch = bytes -> Arrays.equals(bytes, key);
        } else {
            bound = limit > 0 ? limit : Long.MAX_VALUE;
            isMatch = bytes -> Arrays.compare(bytes, end) < 0;
        }

        final List<byte[]> keys = new ArrayList<>();
        final List<byte[]> values = new ArrayList<>();
        final RocksIterator it = batch.newIteratorWithBase(db.newIterator());
        final byte[] fixedKey = ns.fixKey(key);
        for (it.seek(fixedKey); it.isValid(); it.next()) {
            final byte[] bytes = ns.unfixKey(it.key());
            if (!isMatch.test(bytes)) {
                break;
            }
            keys.add(bytes);
            values.add(it.value());
            if (keys.size() >= bound) {
                break;
            }
        }
        return new BackendRangeResult(keys, values);
    }

    @Override
    public void unsafePut(Namespace ns, byte[] key, byte[] value) {
        final byte[] bytes = ns.fixKey(key);
        try {
            batch.put(bytes, value);
        } catch (RocksDBException e) {
            log.atError().addKeyValue("namespace", ns).log("failed to write to the backend", e);
            throw new UncheckedIOException(new IOException(e));
        }
    }

    @Override
    public void close() throws Exception {
        try (final WriteOptions options = new WriteOptions()) {
            db.write(options, batch);
        } finally {
            batch.close();
        }
    }
}

New I found that to port it to Rust, the concept WriteBatchWithIndex mismatches. I wonder if I miss something in this crate or just this crate is yet to support this.

@tisonkun
Copy link
Author

It seems exposed in the upstream - facebook/rocksdb#12091 (comment)

@tillrohrmann
Copy link

I would also be interested in this feature. I am wondering whether there is something fundamentally missing for it or whether it mainly needs someone exposing it.

@tisonkun
Copy link
Author

tisonkun commented Apr 20, 2024

@zaidoon1 do you pick up this issue? Or are you open to review a patch that export these types and functions?

@zaidoon1
Copy link
Contributor

zaidoon1 commented Apr 20, 2024

@tisonkun I typically work on things that I need to use due to time constraints. This feature is not one of them, however, If you create a pr, I'm happy to review it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants