Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can`t put objects larger than 1Mb to KV store #1134

Open
roman-mozil opened this issue May 10, 2024 · 0 comments
Open

Can`t put objects larger than 1Mb to KV store #1134

roman-mozil opened this issue May 10, 2024 · 0 comments
Labels
defect Suspected defect such as a bug or regression

Comments

@roman-mozil
Copy link

Observed behavior

Java SDK throws error when I try to put 1.2 Mb String value into KV store

Expected behavior

I can put values larger than 1 Mb to KV

Server and client version

Nats is deployed using the latest Docker-compose(Logs: Starting nats-server Version: 2.10.12).
Also, the issue reproduces in Kubernetes cluster. For that, I use the latest helm chart with enabled “jetstream” option.
Nats CLI: v. 0.1.4

Java SDK:
implementation group: 'io.nats', name: 'jnats', version: '2.17.5'
implementation group: 'io.nats', name: 'java-nats-streaming', version: '2.2.3'

Host environment

No response

Steps to reproduce

I created KV bucket using Nats CLI command nats kv add --max-value-size=9999999999 --storage=file exampleBucket
(I tried with “--storage=memory” option and got the same result)

Output:

Configuration:
          Bucket Name: exampleBucket
         History Kept: 1
        Values Stored: 0
           Compressed: false
   Backing Store Kind: JetStream
          Bucket Size: 0 B
  Maximum Bucket Size: unlimited
   Maximum Value Size: 1.3 GiB
          Maximum Age: unlimited
     JetStream Stream: KV_exampleBucket
              Storage: File

Extra bucket details from Java application:

KeyValue kv = nc.keyValue(exArgs.bucket);
System.out.println(kv.getStatus());

Output:

KeyValueStatus{"entryCount":1,"byteCount":67,"config":{"name":"exampleBucket","maxBucketSize":-1,"ttl":"PT0S","storageType":"file","replicas":1,"placement":{},"isCompressed":false,"maxHistoryPerKey":1,"maxValueSize":1410065407,"sources":[]}}

Code example to put obj:

kv.put(STRING_KEY, largeFile);
kv.put(STRING_KEY, largeFile.getBytes());

Error that I got:

java.lang.IllegalArgumentException: Message payload size exceed server configuration 1211480 vs 1048576
    at io.nats.client.impl.NatsConnection.checkPayloadSize(NatsConnection.java:906)
    at io.nats.client.impl.NatsConnection.requestFutureInternal(NatsConnection.java:1182)
    at io.nats.client.impl.NatsConnection.requestInternal(NatsConnection.java:1123)
    at io.nats.client.impl.NatsJetStreamImpl.makeInternalRequestResponseRequired(NatsJetStreamImpl.java:241)
    at io.nats.client.impl.NatsJetStream.publishSyncInternal(NatsJetStream.java:155)
    at io.nats.client.impl.NatsJetStream.publish(NatsJetStream.java:83)
    at io.nats.client.impl.NatsKeyValue._write(NatsKeyValue.java:237)
    at io.nats.client.impl.NatsKeyValue.put(NatsKeyValue.java:141)
    at com.nats.kv.app.NatsKeyValueFull.main(NatsKeyValueFull.java:95)

I also tried GO SDK to put the same 1.2 MB String object to KV and this attempt was unsuccessful too.

The large object(large_text.txt) that I use is attached.
large_text.txt

P.S.
Java SDK create bucket operation with parameters:

KeyValueConfiguration kvc = KeyValueConfiguration.builder()
    .name("exampleBucket")
    .description("Example Description")
    .maxHistoryPerKey(5)
    .storageType(StorageType.File)
    .maxValueSize(9999999999L)
    .build();
KeyValueStatus kvs = kvm.create(kvc);

Fails with error:

io.nats.client.JetStreamApiException: invalid JSON [10025]
    at io.nats.client.api.ApiResponse.throwOnHasError(ApiResponse.java:82)
    at io.nats.client.impl.NatsJetStreamImpl.createAndCacheStreamInfoThrowOnError(NatsJetStreamImpl.java:154)
    at io.nats.client.impl.NatsJetStreamManagement.addOrUpdateStream(NatsJetStreamManagement.java:68)
    at io.nats.client.impl.NatsJetStreamManagement.addStream(NatsJetStreamManagement.java:48)
    at io.nats.client.impl.NatsKeyValueManagement.create(NatsKeyValueManagement.java:51)
    at com.nats.kv.app.NatsKeyValueFull.main(NatsKeyValueFull.java:68)
@roman-mozil roman-mozil added the defect Suspected defect such as a bug or regression label May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
defect Suspected defect such as a bug or regression
Projects
None yet
Development

No branches or pull requests

1 participant